Add failure category filtering to report-failure-as-issue

.github/workflows/example-failure-category-filter.md

@@ -0,0 +1,73 @@
+---
+name: Example Failure Category Filter
+on:
+  workflow_dispatch:
+safe-outputs:
+  report-failure-as-issue:
+    - agent_failure           # Only report genuine agent-side failures
+    - missing_safe_outputs    # Only report when outputs are missing
+    - missing_tool            # Only report when functionality is missing
+    - missing_data            # Only report when required data is unavailable
+    # Excluded categories (won't create issues):
+    # - report_incomplete: Infrastructure/tool failures
+    # - inference_access_error: AI server transient errors
+    # - ai_credits_rate_limit_error: AI rate limits
+    # - mcp_policy_error: MCP policy violations
+  create-issue:
+---
+
+# Example: Failure Category Filtering
+
+This workflow demonstrates the `report-failure-as-issue` category filtering feature with both inclusion and exclusion syntax.
+
+## Context
+
+For scheduled workflows that frequently encounter transient infrastructure failures:
+- Docker registry timeouts
+- AI server 5xx errors
+- Firewall startup failures
+- MCP image pull intermittent failures
+
+Traditional `report-failure-as-issue: false` suppresses ALL failure reports, including genuine agent bugs.
+
+## Solution
+
+Use category filtering to only report actionable failures:
+
+### Inclusion Syntax (include only these categories)
+
+```yaml
+safe-outputs:
+  report-failure-as-issue:
+    - agent_failure
+    - missing_safe_outputs
+    - missing_tool
+    - missing_data
+```
+
+### Exclusion Syntax (exclude these categories, report all others)
+
+```yaml
+safe-outputs:
+  report-failure-as-issue:
+    - "!inference_access_error"        # Exclude AI server transient errors
+    - "!ai_credits_rate_limit_error"   # Exclude AI rate limits
+    - "!report_incomplete"             # Exclude infrastructure failures
+    - "!mcp_policy_error"              # Exclude MCP policy violations
+```
+
+### Mixed Syntax (include these, but not those)
+
+```yaml
+safe-outputs:
+  report-failure-as-issue:
+    - agent_failure                    # Include agent failures
+    - missing_safe_outputs             # Include missing outputs
+    - "!unknown_model_ai_credits"      # But exclude unknown model AI credits
+```
+
+This prevents noise while preserving actionable signals.
+
+## Task
+
+Create an issue summarizing this feature.

actions/setup/js/handle_agent_failure.cjs

@@ -2505,6 +2505,40 @@ async function main() {
     const unknownModelAICredits = unknownModelAICreditsFromAudit || (unknownModelAICreditsFromOutput && agentConclusion === "failure");
     const pushRepoMemoryResult = process.env.GH_AW_PUSH_REPO_MEMORY_RESULT || "";
     const reportFailureAsIssue = process.env.GH_AW_FAILURE_REPORT_AS_ISSUE !== "false"; // Default to true
+    // Parse included categories filter for report-failure-as-issue (optional JSON array of category strings)
+    const failureCategoriesFilterRaw = process.env.GH_AW_FAILURE_CATEGORIES_FILTER || "";
+    let failureCategoriesFilter = null;
+    if (failureCategoriesFilterRaw) {
+      try {
+        failureCategoriesFilter = JSON.parse(failureCategoriesFilterRaw);
+        if (!Array.isArray(failureCategoriesFilter)) {
+          core.warning(`GH_AW_FAILURE_CATEGORIES_FILTER is not an array, ignoring: ${failureCategoriesFilterRaw}`);
+          failureCategoriesFilter = null;
+        } else {
+          core.info(`Failure categories include filter enabled: ${failureCategoriesFilter.join(", ")}`);
+        }
+      } catch (parseError) {
+        core.warning(`Failed to parse GH_AW_FAILURE_CATEGORIES_FILTER, ignoring: ${getErrorMessage(parseError)}`);
+        failureCategoriesFilter = null;
+      }
+    }
+    // Parse excluded categories filter for report-failure-as-issue (optional JSON array of category strings)
+    const failureExcludedCategoriesFilterRaw = process.env.GH_AW_FAILURE_EXCLUDED_CATEGORIES_FILTER || "";
+    let failureExcludedCategoriesFilter = null;
+    if (failureExcludedCategoriesFilterRaw) {
+      try {
+        failureExcludedCategoriesFilter = JSON.parse(failureExcludedCategoriesFilterRaw);
+        if (!Array.isArray(failureExcludedCategoriesFilter)) {
+          core.warning(`GH_AW_FAILURE_EXCLUDED_CATEGORIES_FILTER is not an array, ignoring: ${failureExcludedCategoriesFilterRaw}`);
+          failureExcludedCategoriesFilter = null;
+        } else {
+          core.info(`Failure categories exclude filter enabled: ${failureExcludedCategoriesFilter.join(", ")}`);
+        }
+      } catch (parseError) {
+        core.warning(`Failed to parse GH_AW_FAILURE_EXCLUDED_CATEGORIES_FILTER, ignoring: ${getErrorMessage(parseError)}`);
+        failureExcludedCategoriesFilter = null;
+      }
+    }
     // Feature flags: control whether missing_tool/missing_data signals trigger agent failure handling.
     // Defaults to true (new behavior); set to false to restore pre-2026 behavior where these signals
     // are only shown in output footers / separate issues without activating the failure code path.
@@ -2881,6 +2915,53 @@ async function main() {
       core.warning(`Failed to write failure categories: ${getErrorMessage(writeError)}`);
     }
 
+    // Check if failure categories match the filter (if configured)
+    // Logic:
+    // 1. If only excluded categories are specified: report all EXCEPT those categories
+    // 2. If only included categories are specified: report ONLY those categories
+    // 3. If both are specified: report categories that are included AND not excluded
+    if (failureCategoriesFilter || failureExcludedCategoriesFilter) {
+      const includeCategories = Array.isArray(failureCategoriesFilter) ? failureCategoriesFilter : [];
+      const excludeCategories = Array.isArray(failureExcludedCategoriesFilter) ? failureExcludedCategoriesFilter : [];
+      const hasIncludeFilter = includeCategories.length > 0;
+      const hasExcludeFilter = excludeCategories.length > 0;
+
+      let shouldCreateIssue = false;
+
+      if (hasIncludeFilter && hasExcludeFilter) {
+        // Both filters: must match include AND not match exclude
+        const hasIncludedCategory = failureCategories.some(cat => includeCategories.includes(cat));
+        const hasExcludedCategory = failureCategories.some(cat => excludeCategories.includes(cat));
+        shouldCreateIssue = hasIncludedCategory && !hasExcludedCategory;
+        if (!shouldCreateIssue) {
+          core.info(`Skipping failure issue creation: categories don't match filters. Categories: [${failureCategories.join(", ")}], Include: [${includeCategories.join(", ")}], Exclude: [${excludeCategories.join(", ")}]`);
+        } else {
+          core.info(`Failure categories match filters, proceeding with issue creation. Include: [${includeCategories.join(", ")}], Exclude: [${excludeCategories.join(", ")}]`);
+        }
+      } else if (hasIncludeFilter) {
+        // Only include filter: must match at least one included category
+        shouldCreateIssue = failureCategories.some(cat => includeCategories.includes(cat));
+        if (!shouldCreateIssue) {
+          core.info(`Skipping failure issue creation: no failure categories match include filter. Categories: [${failureCategories.join(", ")}], Include: [${includeCategories.join(", ")}]`);
+        } else {
+          core.info(`Failure categories match include filter, proceeding with issue creation. Matching categories: [${failureCategories.filter(cat => includeCategories.includes(cat)).join(", ")}]`);
+        }
+      } else if (hasExcludeFilter) {
+        // Only exclude filter: must NOT match any excluded category
+        const hasExcludedCategory = failureCategories.some(cat => excludeCategories.includes(cat));
+        shouldCreateIssue = !hasExcludedCategory;
+        if (!shouldCreateIssue) {
+          core.info(`Skipping failure issue creation: failure categories match exclude filter. Categories: [${failureCategories.join(", ")}], Exclude: [${excludeCategories.join(", ")}]`);
+        } else {
+          core.info(`Failure categories don't match exclude filter, proceeding with issue creation. Categories: [${failureCategories.join(", ")}]`);
+        }
+      }
+
+      if (!shouldCreateIssue) {
+        return;
+      }
+    }
+
     core.info(`Checking for existing issue with precise failure metadata for title: "${issueTitle}"`);
 
     try {

actions/setup/js/handle_agent_failure.test.cjs

@@ -4178,4 +4178,173 @@ describe("handle_agent_failure", () => {
       await expect(detectAndHandleFailureCascade("owner", "repo", 999)).resolves.toBeUndefined();
     });
   });
+
+  describe("failure categories generation", () => {
+    let buildFailureMatchCategories;
+
+    beforeEach(() => {
+      vi.resetModules();
+      ({ buildFailureMatchCategories } = require("./handle_agent_failure.cjs"));
+    });
+
+    it("returns expected categories for agent failure", () => {
+      const categories = buildFailureMatchCategories({
+        agentConclusion: "failure",
+        isTimedOut: false,
+      });
+      expect(categories).toContain("agent_failure");
+      expect(categories.length).toBeGreaterThan(0);
+    });
+
+    it("returns timed_out category", () => {
+      const categories = buildFailureMatchCategories({
+        isTimedOut: true,
+      });
+      expect(categories).toContain("timed_out");
+    });
+
+    it("returns missing_safe_outputs category", () => {
+      const categories = buildFailureMatchCategories({
+        hasMissingSafeOutputs: true,
+      });
+      expect(categories).toContain("missing_safe_outputs");
+    });
+
+    it("returns report_incomplete category", () => {
+      const categories = buildFailureMatchCategories({
+        hasReportIncomplete: true,
+      });
+      expect(categories).toContain("report_incomplete");
+    });
+
+    it("returns sorted categories", () => {
+      const categories = buildFailureMatchCategories({
+        hasMissingSafeOutputs: true,
+        isTimedOut: true,
+        hasReportIncomplete: true,
+      });
+      // Should be sorted alphabetically
+      for (let i = 1; i < categories.length; i++) {
+        expect(categories[i] >= categories[i - 1]).toBe(true);
+      }
+    });
+  });
+
+  describe("failure categories filter behavior", () => {
+    const fs = require("fs");
+    const path = require("path");
+    const os = require("os");
+
+    /** @type {string} */
+    let tmpDir;
+    /** @type {string} */
+    let promptsDir;
+
+    const setupGithubMock = () => {
+      const createIssueMock = vi.fn(async () => ({
+        data: { number: 101, html_url: "https://github.com/owner/repo/issues/101", node_id: "I_101" },
+      }));
+
+      global.github = {
+        rest: {
+          search: {
+            issuesAndPullRequests: vi.fn(async ({ q }) => {
+              if (q.includes("is:pr")) {
+                return { data: { total_count: 0, items: [] } };
+              }
+              return { data: { total_count: 0, items: [] } };
+            }),
+          },
+          issues: {
+            create: createIssueMock,
+            createComment: vi.fn(),
+            getLabel: vi.fn(),
+            addLabels: vi.fn(),
+          },
+          pulls: { get: vi.fn() },
+        },
+        graphql: vi.fn(),
+      };
+
+      return { createIssueMock };
+    };
+
+    beforeEach(() => {
+      tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "aw-failure-filter-"));
+      promptsDir = path.join(tmpDir, "gh-aw", "prompts");
+      fs.mkdirSync(promptsDir, { recursive: true });
+      fs.writeFileSync(path.join(promptsDir, "agent_failure_comment.md"), "COMMENT TEMPLATE CONTENT");
+      fs.writeFileSync(path.join(promptsDir, "agent_failure_issue.md"), "ISSUE TEMPLATE CONTENT");
+      fs.writeFileSync(path.join(promptsDir, "daily_cap_rollup_issue.md"), "Daily cap rollup issue body cap={cap} window={window_hours}");
+      fs.writeFileSync(path.join(promptsDir, "daily_cap_rollup_comment.md"), "Failure suppressed workflow={workflow_name} run={run_url} categories={summary} cap={cap} window={window_hours}h");
+      fs.writeFileSync(path.join(promptsDir, "optimize_token_consumption_context.md"), "OPTIMIZE CONTEXT guardrail={guardrail_name} run={run_url}");
+
+      process.env.RUNNER_TEMP = tmpDir;
+      process.env.GH_AW_WORKFLOW_NAME = "Test Workflow";
+      process.env.GH_AW_WORKFLOW_ID = "test-workflow";
+      process.env.GH_AW_RUN_URL = "https://github.com/owner/repo/actions/runs/123456";
+      process.env.GH_AW_AGENT_CONCLUSION = "failure";
+      process.env.GITHUB_HEAD_REF = "feature/test";
+      process.env.GITHUB_WORKSPACE = tmpDir;
+    });
+
+    afterEach(() => {
+      delete process.env.RUNNER_TEMP;
+      delete process.env.GH_AW_WORKFLOW_NAME;
+      delete process.env.GH_AW_WORKFLOW_ID;
+      delete process.env.GH_AW_RUN_URL;
+      delete process.env.GH_AW_AGENT_CONCLUSION;
+      delete process.env.GITHUB_HEAD_REF;
+      delete process.env.GITHUB_WORKSPACE;
+      delete process.env.GH_AW_FAILURE_CATEGORIES_FILTER;
+      delete process.env.GH_AW_FAILURE_EXCLUDED_CATEGORIES_FILTER;
+
+      if (tmpDir && fs.existsSync(tmpDir)) {
+        fs.rmSync(tmpDir, { recursive: true, force: true });
+      }
+    });
+
+    it.each([
+      {
+        name: "include-only filter creates issue when category matches",
+        includeFilter: ["agent_failure"],
+        excludeFilter: null,
+        shouldCreateIssue: true,
+      },
+      {
+        name: "include-only filter skips issue when category does not match",
+        includeFilter: ["missing_safe_outputs"],
+        excludeFilter: null,
+        shouldCreateIssue: false,
+      },
+      {
+        name: "exclude-only filter skips issue when category is excluded",
+        includeFilter: null,
+        excludeFilter: ["agent_failure"],
+        shouldCreateIssue: false,
+      },
+      {
+        name: "mixed include+exclude filter skips issue when excluded category is present",
+        includeFilter: ["agent_failure"],
+        excludeFilter: ["agent_failure"],
+        shouldCreateIssue: false,
+      },
+    ])("$name", async ({ includeFilter, excludeFilter, shouldCreateIssue }) => {
+      const { createIssueMock } = setupGithubMock();
+      if (includeFilter) {
+        process.env.GH_AW_FAILURE_CATEGORIES_FILTER = JSON.stringify(includeFilter);
+      }
+      if (excludeFilter) {
+        process.env.GH_AW_FAILURE_EXCLUDED_CATEGORIES_FILTER = JSON.stringify(excludeFilter);
+      }
+
+      await main();
+
+      if (shouldCreateIssue) {
+        expect(createIssueMock).toHaveBeenCalledOnce();
+      } else {
+        expect(createIssueMock).not.toHaveBeenCalled();
+      }
+    });
+  });
 });

docs/src/content/docs/reference/glossary.md

@@ -203,14 +203,41 @@ Customizable messages workflows can display during execution. Configured in `saf
 
 ### Failure Issue Reporting (`report-failure-as-issue:`)
 
-A `safe-outputs` option controlling whether workflow run failures are automatically reported as GitHub issues. Defaults to `true` when safe outputs are configured. Set to `false` to suppress failure issue creation for workflows where failures are expected or handled externally:
+A `safe-outputs` option controlling whether workflow run failures are automatically reported as GitHub issues. Defaults to `true` when safe outputs are configured.
+
+**Simple boolean (opt-out all failures):**
 
 ```yaml
 safe-outputs:
   report-failure-as-issue: false
 ```
 
-See [Safe Outputs Reference](/gh-aw/reference/safe-outputs/).
+**Category filtering (selective reporting):**
+
+Filter which failure types trigger issue creation. Categories can be included (default) or excluded (using `!` prefix):
+
+```yaml
+safe-outputs:
+  report-failure-as-issue:
+    - agent_failure           # Include: report genuine agent-side failures
+    - missing_safe_outputs    # Include: report missing outputs
+    - "!inference_access_error"  # Exclude: don't report AI server errors
+```
+
+**Exclusion-only syntax:**
+
+When only exclusions are specified, all categories except those are reported:
+
+```yaml
+safe-outputs:
+  report-failure-as-issue:
+    - "!report_incomplete"           # Exclude infrastructure failures
+    - "!ai_credits_rate_limit_error" # Exclude rate limits
+```
+
+Common categories include: `agent_failure`, `timed_out`, `missing_safe_outputs`, `report_incomplete` (infrastructure failures), `missing_tool`, `missing_data`, `inference_access_error` (AI server transient errors), `ai_credits_rate_limit_error`, and others.
+
+See [Safe Outputs Reference](/gh-aw/reference/safe-outputs/) for complete documentation.
 
 ### Failure Issue Repository (`failure-issue-repo:`)