[detection-analysis] Detection Analysis Report — 2026-07-02 (last 24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC), intended 2026-07-01T13:00:00Z → 2026-07-02T13:00:00Z
• Data coverage: the logs fetch (start_date: -1d) hit the 60s gateway deadline; 87 runs with complete metadata (run_summary.json + aw_info.json) were analyzed. Observed runs span 2026-07-02T10:11:23.608Z → 2026-07-02T13:06:50.312Z.
• Total runs analyzed: 87
• Detection-enabled runs: 21 (24%)
• Regular runs: 66 (76%)
• Misconfigured workflows found: 5

Warning

5 workflows were flagged with detection misconfigurations — see the section below.

Metric	Regular Runs	Detection Runs
Total runs	66	21
Success rate	90.9%	90.5%
Avg tokens (input+output)	70,839	23,158
Failure count	6	2
Misconfigured count	—	5

Detection classification uses features.gh-aw-detection from each run's aw_info.json (present in 33 of 87 runs; absent → treated as Regular).

Comparison Chart

Success rates are effectively identical (~90.5% vs ~90.9%). Detection-enabled runs show a much lower average token footprint (~23,158 vs ~70,839) — the Regular average is inflated by high-volume smoke tests (e.g. Smoke Copilot variants at 0.7M–1.4M tokens).

Misconfigured Workflows

Workflow	Misconfiguration	Runs	Example run	Recommended fix
Copilot Agent Prompt Clustering Analysis	Detection absent on audit/analysis/report workflow	1	§28583469944	Add gh-aw-detection: true — name indicates an audit/analysis/report workflow that should run threat detection.
Smoke CI	Detection explicitly disabled on active workflow	4	§28584608104	Enable gh-aw-detection: true — this workflow has >3 runs/window and should have threat detection on.
Typist - Go Type Analysis	Detection absent on audit/analysis/report workflow	1	§28588330215	Add gh-aw-detection: true — name indicates an audit/analysis/report workflow that should run threat detection.
Daily Agentic Workflow AIC Usage Audit	Detection absent on audit/analysis/report workflow	1	§28589170830	Add gh-aw-detection: true — name indicates an audit/analysis/report workflow that should run threat detection.
GitHub MCP Structural Analysis	Detection absent on audit/analysis/report workflow	1	§28590153883	Add gh-aw-detection: true — name indicates an audit/analysis/report workflow that should run threat detection.

The most actionable case is Smoke CI — 4 runs, 0% success, with detection explicitly disabled. The remaining four are audit/analysis workflows whose names imply they should carry threat detection but currently do not.

View All Run Metrics (87 runs across 42 workflows)

Workflow	Runs	Detection	Success %	Avg tokens	Engine
Design Decision Gate 🏗️	8	—	100%	11,784	claude
Impeccable Skills Reviewer	8	—	75%	28,625	copilot
Matt Pocock Skills Reviewer	8	—	100%	32,579	copilot
PR Code Quality Reviewer	8	—	100%	24,230	copilot
Test Quality Sentinel	8	Yes	88%	16,114	copilot
PR Description Updater	6	—	100%	13,494	copilot
Smoke CI	4	Off	0%	0	copilot
PR Sous Chef	3	—	100%	18,335	pi
Agent Container Smoke Test	1	Off	100%	5,338	copilot
Auto-Triage Issues	1	Yes	100%	11,255	pi
Changeset Generator	1	Yes	100%	13,712	codex
Claude Code User Documentation Review	1	Yes	100%	89,213	claude
Constraint Solving — Problem of the Day	1	Yes	100%	7,055	copilot
Copilot Agent Prompt Clustering Analysis	1	—	100%	37,620	claude
Copilot PR Conversation NLP Analysis	1	Yes	100%	11,860	copilot
Daily A/B Testing Advisor	1	Yes	100%	46,872	pi
Daily AIC Consumption Report (Sentry + Grafana OTel)	1	Yes	100%	35,610	claude
Daily Agentic Workflow AIC Usage Audit	1	—	100%	15,656	copilot
Daily Choice Type Test	1	Yes	100%	6,473	claude
Daily Documentation Updater	1	Yes	100%	62,538	pi
Daily Go Function Namer	1	Yes	100%	26,205	pi
Daily MCP Tool Concurrency Analysis	1	Yes	100%	6,929	copilot
Daily Max Ai Credits Test	1	Yes	0%	2,974	copilot
Draft PR Cleanup	1	—	100%	4,149	copilot
Functional Pragmatist	1	—	100%	21,349	copilot
GitHub API Consumption Report Agent	1	Yes	100%	36,700	claude
GitHub MCP Structural Analysis	1	—	100%	41,448	claude
Glossary Maintainer	1	—	100%	41,538	copilot
Issue Monster	1	—	100%	18,409	pi
PR Triage Agent	1	—	100%	16,368	copilot
Package Specification Enforcer	1	—	100%	46,933	pi
Package Specification Extractor	1	—	100%	21,900	copilot
Smoke Antigravity	1	Off	100%	0	antigravity
Smoke Claude	1	Off	100%	19,936	claude
Smoke Codex	1	Off	100%	260,273	codex
Smoke Copilot	1	Off	100%	1,023,696	copilot
Smoke Copilot - AOAI (Entra)	1	Off	100%	1,409,503	copilot
Smoke Copilot - AOAI (apikey)	1	Off	100%	693,587	copilot
Smoke Gemini	1	—	100%	0	gemini
Smoke Pi	1	Off	100%	8,600	pi
Team Status	1	—	100%	10,670	copilot
Typist - Go Type Analysis	1	—	100%	64,687	claude

Detection column: Yes = gh-aw-detection: true; Off = explicitly false; — = flag absent (treated as Regular).

View Historical Trend (6 days: 2026-06-27 → 2026-07-02)

Detection-run volume has grown modestly (12→21 over the last two days) while detection success rate has held in the 80–95% band. (Trend covers 6 days; the 7-day threshold in the spec is nearly met — the line will fill in on the next run.)

Recommendations

1. Fix Smoke CI (priority): it is failing every run (0/4). Investigate the failure first, then decide on gh-aw-detection — as an active, repeatedly-run workflow it should have detection enabled once green.
2. Enable detection on audit/analysis workflows: add gh-aw-detection: true to Daily Agentic Workflow AIC Usage Audit, GitHub MCP Structural Analysis, Copilot Agent Prompt Clustering Analysis, and Typist - Go Type Analysis — these read/analyze repo content and are natural threat-detection candidates.
3. No detection-job failures observed: all 21 detection-enabled runs completed their detection job successfully — no type-3 issues this window.
4. No inconsistent-state workflows: no workflow alternated detection on/off within the window.

References:

• §28584608104 — Smoke CI (detection off, failing)
• §28589170830 — Daily Agentic Workflow AIC Usage Audit
• §28584538845 — Design Decision Gate (detection job success reference)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔎 Detection Analysis Report · 191.1 AIC · ⌖ 34 AIC · ⊞ 7.2K · ◷

• expires on Jul 5, 2026, 5:42 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #43202.