Suppress noisy auto-filed failure issues for recurring AI-credits failures#9113
Conversation
There was a problem hiding this comment.
Pull request overview
Disables auto-filing GitHub issues for agentic workflow failures that occur after useful output is already produced (notably recurring AI-credits exhaustion), by setting safe-outputs.report-failure-as-issue: false across the affected workflows in this repo.
Changes:
- Added
safe-outputs.report-failure-as-issue: falseto the workflow source files for perf/efficiency/test improvers, PR review, and grade-tests. - Regenerated the corresponding
*.lock.ymlfiles so the compiled workflows setGH_AW_FAILURE_REPORT_AS_ISSUE: "false".
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/test-improver.md | Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false. |
| .github/workflows/test-improver.lock.yml | Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false". |
| .github/workflows/review-on-open.agent.md | Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false. |
| .github/workflows/review-on-open.agent.lock.yml | Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false". |
| .github/workflows/perf-improver.md | Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false. |
| .github/workflows/perf-improver.lock.yml | Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false". |
| .github/workflows/grade-tests-on-pr.agent.md | Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false. |
| .github/workflows/grade-tests-on-pr.agent.lock.yml | Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false". |
| .github/workflows/efficiency-improver.md | Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false. |
| .github/workflows/efficiency-improver.lock.yml | Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false". |
Copilot's findings
- Files reviewed: 10/10 changed files
- Comments generated: 0
Evangelink
left a comment
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
Review Summary
This is a clean, well-scoped configuration change. All 5 affected workflows are updated consistently: both the .md source file and the corresponding compiled .lock.yml file are modified in every case, as required by the agentic workflow guidelines.
Lock File Integrity Verification
All lock file metadata hashes are correct:
frontmatter_hashis updated for every workflow (expected — frontmatter changed).body_hashis unchanged for every workflow (correct — onlysafe-outputsfrontmatter changed, not the agent body/prompt)."strict":trueis preserved in all five lock files. ✅
Minor Observation: grade-tests.agent.md not in scope
grade-tests.agent.md (the /grade-tests slash-command variant) still carries GH_AW_FAILURE_REPORT_AS_ISSUE: "true" in its lock file and has no report-failure-as-issue: false in its source. It already has noop.report-as-issue: false, which is a partial mitigation.
Since that workflow is manually triggered (not recurring/scheduled), omitting it here is likely intentional — a human issued the command so the failure is more expected and visible. If the same AI-credit exhaustion pattern is ever observed from a manual /grade-tests run, it would be easy to follow up with a single-line addition there too.
22-Dimension Verdict Table
| # | Dimension | Status | Notes |
|---|---|---|---|
| 1 | Correctness | ✅ Pass | report-failure-as-issue: false in .md compiles to GH_AW_FAILURE_REPORT_AS_ISSUE: "false" in lock; all five pairs aligned |
| 2 | Security | ✅ N/A | No security surface changed |
| 3 | Performance | ✅ N/A | Configuration only |
| 4 | Error handling | ✅ Pass | Failure mode intentionally changed from auto-issue to silent; deliberate and well-reasoned in the PR description |
| 5 | Edge cases | ✅ N/A | No code logic involved |
| 6 | Code style | ✅ Pass | YAML formatting is consistent with existing safe-outputs: blocks |
| 7 | Testing | ✅ N/A | No tests applicable for workflow config |
| 8 | Documentation | ✅ N/A | No documentation update required |
| 9 | Breaking changes | ✅ Pass | Reduces noise; no consumer-visible contract broken |
| 10 | Public API surface | ✅ N/A | Not applicable |
| 11 | Thread safety | ✅ N/A | Not applicable |
| 12 | Memory / resource leaks | ✅ N/A | Not applicable |
| 13 | Logic bugs | ✅ Pass | No logic introduced |
| 14 | Naming conventions | ✅ Pass | Field name matches the gh aw schema (report-failure-as-issue) |
| 15 | Dependencies | ✅ N/A | No dependency changes |
| 16 | Scope discipline | ✅ Pass | Only report-failure-as-issue touched; GH_AW_MISSING_TOOL_REPORT_AS_FAILURE and GH_AW_MISSING_DATA_REPORT_AS_FAILURE deliberately left as "true" |
| 17 | Consistency | ✅ Pass | All five workflows updated uniformly |
| 18 | Intent alignment | ✅ Pass | Directly suppresses the auto-filed issues referenced in the PR (closes #9085, #9086, #9093, #9102, #9109, #9074) |
| 19 | Lock file integrity | ✅ Pass | frontmatter_hash updated; body_hash unchanged; compiler_version v0.79.4 and engine_versions preserved |
| 20 | Compilation workflow compliance | ✅ Pass | Each .md source change is accompanied by its compiled .lock.yml; strict mode maintained |
| 21 | Strict mode preserved | ✅ Pass | "strict":true in all five lock file metadata headers |
| 22 | Coverage completeness | grade-tests.agent.md (slash-command variant) is not in scope; likely intentional (manual trigger ≠ recurring), but worth keeping in mind if that workflow also starts filing noisy issues |
Overall verdict: ✅ LGTM — no blocking or major issues found. The change is correct, minimal, well-motivated, and matches the established precedent from build-failure-analysis*.md.
🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review (on PR ready) workflow. · 353.3 AIC · ⌖ 12.3 AIC · ◷
Closes #9109
Closes #9102
Closes #9093
Closes #9086
Closes #9085
Closes #9074
These agentic workflows were auto-filing noisy failure issues when they exhausted AI credits after already producing their useful output. This PR disables failure issue reporting with
safe-outputs.report-failure-as-issue: falsefor the recurring perf, efficiency, test, review, and grade-tests workflows, matching the precedent already used bybuild-failure-analysis*.md.Bumping
max-ai-creditswas considered but rejected because the workflows still produce useful PRs (for example #9108, #9101, and #9092) before running out. Only the cleanup step crashes, and that step does not need to be retried via auto-filed tracking issues.