[ci-scan-feedback] KPI Tracker

[github-actions]

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

The threat detection results could not be parsed.

Review the workflow run logs for details.

Tracking quality of [ci-scan] (detection) and [ci-fix] (mitigation) issues, PRs, and loop-in comments since 2026-05-21. Updated every tick of ci-failure-scan-feedback.lock.yml. To raise a concern, comment here or on any [ci-scan]/[ci-fix] issue/PR; the next tick reads in-scope feedback and either opens a [ci-scan-feedback] PR with prompt edits or pushes to the existing one.

Snapshot — 2026-07-14T13:22Z

Activity (last 7d)

stream	opened	good	wrong/unmerged
Scanner KBE issues	7	12	1
Fixer [ci-fix] PRs (confident)	0	0	0
Fixer [ci-fix] PRs (help-wanted)	2	1	1
Fixer loop-in comments	3	0 engaged	—

"good" = KBEs closed completed, PRs merged, loop-ins with a maintainer reply. "wrong/unmerged" = KBEs closed not_planned/duplicate, PRs closed without merge (note: a help-wanted PR closed unmerged is expected, not necessarily a miss). Help-wanted PRs opened 7d: #130328 (merged 07-10), #130594 (closed unmerged 07-13). Loop-in comments over the 7d-opened KBE cohort: #130278, #130479, #130569; none drew a maintainer reply yet.

Scanner quality (last 30d)

metric	count	rate
KBE closures	40	—
Wrong KBE closures	6	15.0%
Maintainer rejection comments	0 (sampled)	—
Duplicate KBEs	4	—

Wrong closures: #128531, #129210 (not_planned); #128076, #129378, #129576, #130014 (duplicate).

Fixer quality (last 30d)

metric	count	rate
Fix PR closures	10	—
Fix PRs merged	2	—
Fix PRs rejected (maintainer pushback)	3	30.0%
Loop-in comments (posted / engaged)	3+ / 0 (sampled)	—
Loop-in / help-wanted mis-attributions	0	—

Merged: #129652, #130328 (both kind: help a maintainer accepted). Rejections: #130106 (@MichalStrehovsky — NativeAOT already-rooted no-op), #129360 (@MihaZupan — codegen workaround), and new this window #130594 (@vcsjones — "This is not the right fix and we need to understand what is going on before proceeding ... after investigating the AZL configuration"). The first two remain covered by existing ci-failure-fix.md Step 5.2 guards; #130594 is a newly identified failure mode — the fixer weakened a crypto test's PlatformKeySizeRequirements (dropping the IsSymCryptOpenSsl branch) to accommodate an Azure Linux / SymCrypt-OpenSSL provider difference under active owner investigation. This tick's [ci-scan-feedback] PR adds a gate for it.

Outage signals (analyzed CI)

signal	threshold	24h	7d	status
New-KBE burst	day > 2x trailing 30d median (min 3)	0 / median 2	peak day 3	🟢
Build-break spike	>= 2 in any 24h	0	0	🟢
Multi-pipeline outage (distinct legs proxy)	>= 3 distinct legs in 24h	0 distinct	3 distinct (peak day)	🟢
KBE re-filed after maintainer close	any in 7d	0	0	🟢
Scanner wrong-closure rate (30d)	>= 30% with scan_closed_30d >= 10	—	15.0%	🟢
Fixer rejection rate (30d)	>= 30% with fix_closed_30d >= 5	—	30.0%	🔴

details: Fixer rejection rate — 3 of 10 [ci-fix] PRs closed in 30d drew maintainer "wrong fix" pushback: #130594 (crypto platform-config, @vcsjones), #130106 (NativeAOT no-op rooting), #129360 (codegen workaround). Only #130594 is a new failure mode; a targeted gate ships in this tick's [ci-scan-feedback] PR.

This tick shipped one prompt edit

• ci-failure-fix.md Step 5.1.2 (new). Adds a security/crypto platform-expectation gate: when the only candidate change would weaken a System.Security.Cryptography* test's platform-capability expectations (KeySizes/PlatformKeySizeRequirements, IsSymCryptOpenSsl/provider-detection branches, supported-algorithm flags) to accommodate a distro/provider config (Azure Linux, SymCrypt-OpenSSL, FIPS), the fixer routes to a loop-in comment instead of a speculative PR. Directly targets the [ci-fix] Needs review: apply KMAC key-size requirements on SymCrypt-OpenSSL (refs #129939) #130594 rejection.
• The other two 30d rejections need no new rule — both already map to existing Step 5.2 guards (no NativeAOT-rooting no-ops, no codegen workarounds).

Note

This tracker snapshot was generated by the CI Outer-Loop Failure Scanner — Feedback agentic workflow. Content is AI/Copilot-generated; metrics are computed from the public [ci-scan]/[ci-fix] artifact universe and workflow-run metadata.

Generated by CI Outer-Loop Failure Scanner — Feedback · 238.9 AIC · ⊞ 19K · ◷

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.