Skip to content

Suppress noisy auto-filed failure issues for recurring AI-credits failures#9113

Merged
Evangelink merged 1 commit into
mainfrom
dev/amauryleve/fix-aw-credit-limits
Jun 14, 2026
Merged

Suppress noisy auto-filed failure issues for recurring AI-credits failures#9113
Evangelink merged 1 commit into
mainfrom
dev/amauryleve/fix-aw-credit-limits

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Closes #9109
Closes #9102
Closes #9093
Closes #9086
Closes #9085
Closes #9074

These agentic workflows were auto-filing noisy failure issues when they exhausted AI credits after already producing their useful output. This PR disables failure issue reporting with safe-outputs.report-failure-as-issue: false for the recurring perf, efficiency, test, review, and grade-tests workflows, matching the precedent already used by build-failure-analysis*.md.

Bumping max-ai-credits was considered but rejected because the workflows still produce useful PRs (for example #9108, #9101, and #9092) before running out. Only the cleanup step crashes, and that step does not need to be retried via auto-filed tracking issues.

…lures (#9109, #9102, #9093, #9086, #9085, #9074)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 14, 2026 19:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Disables auto-filing GitHub issues for agentic workflow failures that occur after useful output is already produced (notably recurring AI-credits exhaustion), by setting safe-outputs.report-failure-as-issue: false across the affected workflows in this repo.

Changes:

  • Added safe-outputs.report-failure-as-issue: false to the workflow source files for perf/efficiency/test improvers, PR review, and grade-tests.
  • Regenerated the corresponding *.lock.yml files so the compiled workflows set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".
Show a summary per file
File Description
.github/workflows/test-improver.md Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false.
.github/workflows/test-improver.lock.yml Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".
.github/workflows/review-on-open.agent.md Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false.
.github/workflows/review-on-open.agent.lock.yml Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".
.github/workflows/perf-improver.md Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false.
.github/workflows/perf-improver.lock.yml Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".
.github/workflows/grade-tests-on-pr.agent.md Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false.
.github/workflows/grade-tests-on-pr.agent.lock.yml Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".
.github/workflows/efficiency-improver.md Disables auto-filing failure issues via safe-outputs.report-failure-as-issue: false.
.github/workflows/efficiency-improver.lock.yml Compiled output updated to set GH_AW_FAILURE_REPORT_AS_ISSUE: "false".

Copilot's findings

  • Files reviewed: 10/10 changed files
  • Comments generated: 0

@Evangelink Evangelink left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

Review Summary

This is a clean, well-scoped configuration change. All 5 affected workflows are updated consistently: both the .md source file and the corresponding compiled .lock.yml file are modified in every case, as required by the agentic workflow guidelines.

Lock File Integrity Verification

All lock file metadata hashes are correct:

  • frontmatter_hash is updated for every workflow (expected — frontmatter changed).
  • body_hash is unchanged for every workflow (correct — only safe-outputs frontmatter changed, not the agent body/prompt).
  • "strict":true is preserved in all five lock files. ✅

Minor Observation: grade-tests.agent.md not in scope

grade-tests.agent.md (the /grade-tests slash-command variant) still carries GH_AW_FAILURE_REPORT_AS_ISSUE: "true" in its lock file and has no report-failure-as-issue: false in its source. It already has noop.report-as-issue: false, which is a partial mitigation.

Since that workflow is manually triggered (not recurring/scheduled), omitting it here is likely intentional — a human issued the command so the failure is more expected and visible. If the same AI-credit exhaustion pattern is ever observed from a manual /grade-tests run, it would be easy to follow up with a single-line addition there too.


22-Dimension Verdict Table

# Dimension Status Notes
1 Correctness ✅ Pass report-failure-as-issue: false in .md compiles to GH_AW_FAILURE_REPORT_AS_ISSUE: "false" in lock; all five pairs aligned
2 Security ✅ N/A No security surface changed
3 Performance ✅ N/A Configuration only
4 Error handling ✅ Pass Failure mode intentionally changed from auto-issue to silent; deliberate and well-reasoned in the PR description
5 Edge cases ✅ N/A No code logic involved
6 Code style ✅ Pass YAML formatting is consistent with existing safe-outputs: blocks
7 Testing ✅ N/A No tests applicable for workflow config
8 Documentation ✅ N/A No documentation update required
9 Breaking changes ✅ Pass Reduces noise; no consumer-visible contract broken
10 Public API surface ✅ N/A Not applicable
11 Thread safety ✅ N/A Not applicable
12 Memory / resource leaks ✅ N/A Not applicable
13 Logic bugs ✅ Pass No logic introduced
14 Naming conventions ✅ Pass Field name matches the gh aw schema (report-failure-as-issue)
15 Dependencies ✅ N/A No dependency changes
16 Scope discipline ✅ Pass Only report-failure-as-issue touched; GH_AW_MISSING_TOOL_REPORT_AS_FAILURE and GH_AW_MISSING_DATA_REPORT_AS_FAILURE deliberately left as "true"
17 Consistency ✅ Pass All five workflows updated uniformly
18 Intent alignment ✅ Pass Directly suppresses the auto-filed issues referenced in the PR (closes #9085, #9086, #9093, #9102, #9109, #9074)
19 Lock file integrity ✅ Pass frontmatter_hash updated; body_hash unchanged; compiler_version v0.79.4 and engine_versions preserved
20 Compilation workflow compliance ✅ Pass Each .md source change is accompanied by its compiled .lock.yml; strict mode maintained
21 Strict mode preserved ✅ Pass "strict":true in all five lock file metadata headers
22 Coverage completeness ⚠️ Minor grade-tests.agent.md (slash-command variant) is not in scope; likely intentional (manual trigger ≠ recurring), but worth keeping in mind if that workflow also starts filing noisy issues

Overall verdict: ✅ LGTM — no blocking or major issues found. The change is correct, minimal, well-motivated, and matches the established precedent from build-failure-analysis*.md.

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review (on PR ready) workflow. · 353.3 AIC · ⌖ 12.3 AIC ·

@Evangelink Evangelink merged commit bf92be0 into main Jun 14, 2026
29 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/fix-aw-credit-limits branch June 14, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants