Impact Efficiency Report - 2026-06-16

[github-actions]

Executive Summary

In the last 180 days, two high-value bug fixes in the sample-replay pipeline and one improvement-tier linter addition delivered 205 total outcome value from 3 accepted PR outcomes. The bug fixes — repairing push_to_pull_request_branch branch derivation and multi-repo checkout manifest lookup — each scored 85 (objective: bug). PR Description Updater carries 100% of attributable delivered value (IE = 0.234 value/AIC). Documentation workflows consumed 35.4M + 1.3M model tokens with zero accepted outcomes — all 4 documentation PRs were closed without merge, representing a significant efficiency gap.

---

Summary

Metric	PRs	Safe-output Issues	Combined
Analyzed with linked issue (merged/closed/excl. open)	7 (3 merged / 4 closed)	0	7
Excluded (no linked closing issue)	993 of 1,000-PR sample	—	993
Outcomes mapped to objectives	7	0	7
Unmapped outcomes	0	0	0
Accepted outcome count	3	0	3
Total outcome value	205	0	205
AI Credits	876.3 AIC (3 of 7 runs; 4 missing)	n/a	876.3 AIC
Impact Efficiency	0.234 value/AIC (partial)	n/a	0.234

---

What We Worked On

bug — Value: 85 · 2 outcomes · Total: 170 · AIC: 486.4 · IE: 0.350

• PR #37863 — Fix push_to_pull_request_branch always derives from PR head ref → root #37835 (bug, workflows, compiler)
• PR #37565 — Fix multi-repo checkout manifest-first lookup → root #37545 (bug, workflows, compiler)

⚠️ Highest AIC category: bug — 486.4 AIC, IE = 0.350. Highest return per credit.

improvement — Value: 35 · 1 outcome · Total: 35 · AIC: 389.9* · IE: ~0.090*

• PR #38185 — [linter-miner] Add execcommandwithoutcontext Go analyzer → root #38143 (improvement, quick-win, automation)

*AIC shown is from the PR Description Updater description-update run, not the linter-miner creating run (AIC unknown). True IE uncertain.

---

Which Workflows Drove That Impact

1. PR Description Updater (pr-description-caveman) — 205 value, 100% share, IE 0.234

Accepted outcomes	AIC	Top categories
3 (PRs #38185, #37863, #37565)	876.3 (runs 27230621654, 27310810669, 27094578466)	bug (170), improvement (35)

⚠️ Attribution caveat: PR #38185 was created by github-actions[bot] / linter-miner workflow; PRs #37863 and #37565 were created by a human contributor. PR Description Updater only updated their descriptions.

2. Daily Documentation Healer — 0 value, ~35.4M tokens (non-std AIC), IE 0

3 rejected PRs: #36767 (root #36275), #36371 (root #34686), #35330 (root #34686). PRs #36371 and #35330 both targeted the same root issue — duplicated effort, both rejected.

3. Daily Documentation Updater — 0 value, ~1.3M tokens (non-std AIC), IE 0

1 rejected PR: #35906 (root #35506).

Key callouts: Most value + largest share + most standard AIC: PR Description Updater. Most non-standard AIC with zero return: Daily Documentation Healer (35.4M tokens, 3 runs, 0 accepted).

---

Top Outcomes by Outcome Value

Outcome	Workflow	Type	Root / Objective	Obj. Value	Outcome Value
PR #37863 Fix push_to_pull_request_branch	PR Description Updater	PR merged	#37835 bug	85	85
PR #37565 Fix multi-repo checkout lookup	PR Description Updater	PR merged	#37545 bug	85	85
PR #38185 Add execcommandwithoutcontext linter	PR Description Updater*	PR merged	#38143 improvement	35	35
PR #36767 Document undocumented frontmatter fields	Daily Documentation Healer	PR closed	#36275 improvement	35	0
PR #36371 Fix broken doc-link anchors (attempt 1)	Daily Documentation Healer	PR closed	#34686 improvement	35	0
PR #35330 Fix broken doc-link anchors (attempt 2)	Daily Documentation Healer	PR closed	#34686 improvement	35	0
PR #35906 Document user-invokable frontmatter field	Daily Documentation Updater	PR closed	#35506 improvement	35	0

---

Unmapped Outcomes

None. All 7 analyzed outcomes had root issues with labels matching .github/objective-mapping.json. Labels cookie and deep-report present on some issues are not in the mapping and were ignored.

---

Interpretation

Accepted outcome count (3/7, 43%) is a poor cost signal — it doesn't capture the cost of 4 failed documentation PRs. Impact Efficiency (0.234 value/AIC, partial) better reflects value-per-credit:

• 🏅 Best IE: bug category at 0.350 — the two sample-replay reliability fixes returned the most value per AIC spent.
• ⚠️ Zero-return AIC: Documentation workflows consumed 35.4M + 1.3M model tokens (non-standard units) with all PRs rejected. The Daily Documentation Healer opened two PRs for the same root issue (#34686), both closed without merge — suggesting a systematic mismatch between what the workflow produces and what maintainers accept.
• 📌 Attribution gap: PR Description Updater appears as the sole workflow contributor, but it only rewrites PR descriptions. The actual value-producing workflows (linter-miner, human contributor dsyme) are not separately tracked in available AIC data.

---

Data Quality

Area	Status
PR root tracing	3/500 merged (0.6%), 4/500 closed (0.8%) had linked issues. Dataset capped at 500 each.
Safe-output label coverage	N/A — 0 safe-output outcomes in window.
Workflow attribution	PR #38185 body shows PR Description Updater but was created by linter-miner (title [linter-miner], creator github-actions[bot]). PRs #37863/#37565 created by human dsyme — technically outside "PRs created by workflow runs" scope.
AIC availability	workflow-logs.json empty (0 bytes). AIC extracted from PR body footers (live fallback). 3 runs: 876.3 AIC (standard). 4 runs: non-standard model-token units (M format, not convertible to AIC without a rate). 4 of 7 runs (57%) have missing standard AIC.
AIC source	Live fallback (PR body footer extraction) — not deterministic precomputed data.

Generated by 📊 Impact Efficiency Report · 436.5 AIC · ⌖ 14.4 AIC · ⊞ 23.1K · ◷