Description
A single failed run of Daily Sub-Agent Model Resolution Audit consumed 11,490,620 tokens (49% of the entire fleet's daily spend) before failing at Turns=0 with no agentic output. The driver-level retry/loop burned ~11.5M tokens pre-turn with no cost guardrail intervening.
This is both a reliability and cost-quality issue: the workflow infrastructure should prevent a failing run from doubling fleet token usage.
Suggested Changes
- Add a per-run token limit / circuit-breaker that terminates a driver run if token consumption exceeds a configurable cap before any agent turn is produced (
Turns=0 after X tokens).
- Alternatively, detect the
agentic_fraction=0 + high-token signature and surface an alert / auto-terminate.
- Add the
Daily Sub-Agent Model Resolution Audit workflow as a test case with a conservative per-run cap (e.g., 2M tokens).
- Consider emitting a warning when
TokenUsage > fleet_daily_avg * 0.25 for a single run, regardless of outcome.
Files Affected
- Workflow driver / agent job execution layer
.github/workflows/daily-sub-agent-model-resolution-audit.md (add token cap frontmatter if supported)- Monitoring/alerting config
Success Criteria
- A run consuming > configurable token cap with
Turns=0 is terminated early
- The circuit-breaker fires before the run doubles fleet daily spend
- Token anomaly is surfaced in the audit report
- Existing workflows unaffected (cap only triggers on pre-turn runaway)
Source
Extracted from Agentic Workflow Audit discussion 47150 (Top anomaly — 11.49M token burn on failed run, run 29806493363)
Priority
High — cost leak; single run burned 49% of fleet daily tokens with zero output, estimated effort 1–2 days
🔍 Task mining by Discussion Task Miner - Code Quality Improvement Agent · sonnet46 76.4 AIC · ⌖ 5.54 AIC · ⊞ 7.1K · ◷
Description
A single failed run of
Daily Sub-Agent Model Resolution Auditconsumed 11,490,620 tokens (49% of the entire fleet's daily spend) before failing atTurns=0with no agentic output. The driver-level retry/loop burned ~11.5M tokens pre-turn with no cost guardrail intervening.This is both a reliability and cost-quality issue: the workflow infrastructure should prevent a failing run from doubling fleet token usage.
Suggested Changes
Turns=0after X tokens).agentic_fraction=0+ high-token signature and surface an alert / auto-terminate.Daily Sub-Agent Model Resolution Auditworkflow as a test case with a conservative per-run cap (e.g., 2M tokens).TokenUsage > fleet_daily_avg * 0.25for a single run, regardless of outcome.Files Affected
.github/workflows/daily-sub-agent-model-resolution-audit.md(add token cap frontmatter if supported)Success Criteria
Turns=0is terminated earlySource
Extracted from Agentic Workflow Audit discussion 47150 (Top anomaly — 11.49M token burn on failed run, run 29806493363)
Priority
High — cost leak; single run burned 49% of fleet daily tokens with zero output, estimated effort 1–2 days