[audit-workflows] 🔍 Agentic Workflow Audit — 2026-07-11 · 89.6% healthy, Smoke CI chronic dominant

[github-actions[bot]]

🔍 Agentic Workflow Audit — 2026-07-11

Window: 2026-07-10 22:13Z → 2026-07-11 21:05Z (~23h) · 335 runs analyzed across 7 continuation batches

Verdict: 🟢 HEALTHY-on-baseline. Overall success 89.6% (300/335), on par with the 07-04→07-06 baseline (86–90%). All 35 failures are 0-turn driver-level — zero agent-logic errors, zero missing tools, zero missing data, zero MCP failures. One chronic cluster (Smoke CI) accounts for 34% of all failures; removing it lifts the fleet to 92.8%.

📊 Health at a glance

Metric	Value
Overall success	89.6% (300/335)
Prod-main	86.3% (184/213)
Non-main (PR/dispatch)	95.1% (116/122)
Excluding Smoke CI	92.8% (300/323)
Missing tools / data / MCP	0 / 0 / 0 ✅
AI Credits (AIC)	14,553
Action-minutes	3,323
GitHub API calls	6,896
Tokens	⚠️ null (reporting gap persists)

Per-engine success: codex 30/30 = 100% ✅ · claude 52/53 = 98.1% · pi 70/72 = 97.2% · copilot 146/172 = 84.9% (all real failures) · opencode 0/3 (dev-branch, see below).

🔴 Failure clusters (35 total)

1. Smoke CI — 12/12 = 100% fail · DOMINANT (34% of all fails) · chronic recur #7

• copilot / push→main, 3.7–4.6m, 0-turn at Execute GitHub Copilot CLI
• Chronic across 7 observed windows (06-30 → 07-11), still unfixed
• Highest-ROI fix on the board: resolving/muting it lifts the fleet from 89.6% → 92.8%
• Root cause still opaque — the failing Execute-CLI step captures no stderr/exit-code

2. copilot Execute-CLI 0-turn family — ~13 fails (instant + longrun) · known: copilot-sdk-driver-failures

Agent does real work then the job fails 0-turn. This window's members:

• Daily Security Observability Report 30m · daily-experiment-report 22m · Daily Firewall Logs Collector 19m · Daily Project Performance Summary (MCP) 19m
• PR Triage Agent ×2 (~14m) · Documentation Noob Tester 11m · Contribution Check 10m · Daily Issues Report Generator 9m (chroot-node variant)
• Daily BYOK Ollama Test 7m · Smoke Copilot Sub Agents 5.5m · GitHub Remote MCP Auth Test 2.8m

Together with Smoke CI, the copilot 0-turn family is 25/35 = 71% of all fleet failures.

3. Smoke OpenCode — 3/3 fail · NEW · isolated to dev branch (no action)

• All 3 are workflow_dispatch on feature branch copilot/define-agentic-engine-syntax
• opencode is a newly-onboarded engine under active development — this is a PR-under-development signal, not a prod-main regression
• Flag to the branch author; no fleet action needed while it stays off main

4. Minor / isolated (5 fails)

• PR Sous Chef (pi) ×2 — 3.7m + 9.7m 0-turn. Minor relapse after 07-04 full recovery (19/19); pi otherwise 70/72. Watch for a wider pi/gpt-5.4 collapse.
• Design Decision Gate (claude) — 8.2m on PR branch copilot/extend-compiler-security-analysis. Only claude failure (52/53). Isolated.
• Intentional test harnesses — Daily Credit Limit Test ×2, Daily Max Ai Credits Test, [aw] Failure Investigator (38s). Credit/limit probes — failures largely by-design.

🟢 Notable improvements (absences vs prior windows)

• Avenger: 0 failures this window (was the rejig docs #1 chronic offender — err-config, 10/10 on 06-27, recur18). Second consecutive quiet window → likely still paused or root-cause addressed. Kept OPEN/WATCH (absence has been a pause before, not a fix).
• codex: 30/30 = 100% — the codex gh-aw-binary and Daily Fact 0-turn issues are quiet.
• PR Code Quality Reviewer: 0 fails / 19 runs — previously longrun-fail-heavy, recovered this window.

📈 Trend charts (30-day)

Workflow health has held a stable 86–92% band for the past week after the volatile late-June stretch (60–70% troughs on 06-20/07-02/07-03). Today's 89.6% sits right on that baseline, with the failure floor now dominated almost entirely by the chronic Smoke CI + copilot 0-turn clusters rather than novel breakage.

Cost is charted in AI Credits (AIC) because the per-run token metric has been null since ~06-19 (metrics.TokenUsage empty — a known reporting gap). Today's 14.5K AIC is a mid-range day; the 7-day moving average remains steady with no cost spike, and the highest single spenders (PR Code Quality Reviewer 818 AIC/19 runs, Go Logger Enhancement 668/1) all completed successfully.

🎯 Prioritized recommendations

1. [P1 · recur Weekly Research Report: AI Workflow Automation Landscape and Market Opportunities - August 2025 #7] Smoke CI 100%-fail on push→main. Single highest-ROI fix. Instrument the Execute GitHub Copilot CLI step to capture stderr + exit-code so the 0-turn abort stops being opaque, or mute the push trigger until fixed. Fixing it → +3.2pp fleet success.
2. [HIGH] Root-cause the copilot Execute-CLI 0-turn family (71% of fleet fails). Same diagnostic gap: no stderr/exit-code at the failing step. Distinguish instant (Smoke CI ~4m) vs longrun (19–30m) sub-variants — likely different causes (startup vs session-idle/network).
3. [LOW · WATCH] pi PR Sous Chef relapse — 2 isolated 0-turn fails after 07-04 recovery. No action unless it widens.
4. [LOW · INFO] Smoke OpenCode dev-branch — flag to copilot/define-agentic-engine-syntax author; opencode smoke path not yet green (off-main, no fleet action).
5. [LOW] Token-reporting gap — metrics.TokenUsage empty 3+ weeks. Restore per-run aggregation or officially adopt AIC as the fleet cost metric.

✅ Repo memory updated

audit-history.jsonl (+1 cycle), metrics-summary.json (+07-11 point), known-issues.json (Smoke CI recur7, copilot-sdk recur, pi relapse, Avenger→quieting, +opencode NEW; pruned 6 stale), anomalies.json, recommendations.json, workflow-trends.json — validated 51 KB / 60 KB.

References:

• §29168017929 — Daily Project Perf Summary (copilot 0-turn 19m)
• §29159239452 — Daily Security Observability Report (copilot 0-turn 30m)
• §29155738582 — Smoke CI (chronic 0-turn on push)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔍 Agentic Workflow Audit Agent · 311.6 AIC · ⌖ 43 AIC · ⊞ 7.4K · ◷

• expires on Jul 12, 2026, 1:58 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Agentic Workflow Audit Agent.

A newer discussion is available at Discussion #45139.