[safe-output-health] 🏥 Safe Output Health Report - 2026-07-11

[github-actions[bot]]

Executive Summary

• Period: Last 24 hours (window ~03:1xZ–05:0xZ, 15 run directories)
• Runs Analyzed: 15
• Workflows Active: 15 (all schedule-triggered)
• Safe Output Jobs Executed: 15
• Safe Output Jobs Failed (hard): 1 (production)
• Safe Output Messages Actuated: 0 (READ-ONLY window — every run SafeItemsCount=0)
• Error Clusters Identified: 1 new (in-scope)
• Run-level safe_outputs job success rate: 93.3% (14/15)

This was a read-only window (contrast with the write-rich 07-10): all 15 runs emitted zero safe-output messages, zero noop, zero missing_tool. One production safe_outputs job hard-failed — breaking the recent clean read-only streak.

Safe Output Job Statistics

Job Type	Executed	Failed	Success Rate
safe_outputs (all runs)	15	1	93.3%
Messages actuated (any type)	0	0	n/a (read-only)

No create_issue / add_comment / create_pull_request / review-comment / assign_to_agent messages were actuated in this window — nothing to fail at the message level. The single failure is at the job / Process Safe Outputs step layer.

Error Clusters

Cluster 1: firewall_report_process_safe_outputs_hardfail_create_discussion_not_wired (NEW, production)

• Count: 1 occurrence
• Run: §29137608538 — Daily Firewall Logs Collector and Reporter (schedule, copilot/claude-sonnet-4.6)
• Job conclusions: activation ✅ · agent ✅ · detection ✅ · upload_assets ⏭ skipped · safe_outputs ❌ FAILURE · conclusion ✅
• Failed step: Process Safe Outputs (sibling Upload Safe Outputs Items succeeded)
• Metrics: SafeItemsCount=0, NoopCount=0, MissingToolCount=0, ErrorCount=0, WarningCount=0
• Sample error:

safe_outputs job → step "Process Safe Outputs" → conclusion: failure
audit MCP tool: error_count=1 / "Workflow Failed" (generic; exact step stdout NOT surfaced)

• Root cause (strong hypothesis, from prompt inspection): prompt/config drift. The workflow's task body (daily-firewall-report.md, Step 6 "Create Discussion", Expected Output: "A GitHub discussion in the audits category") instructs the agent to create_discussion with category audits as its primary output. But the compiled safe-outputs allowlist for this run enables only upload_asset(max:3), missing_tool, missing_data, noop — there is no create_discussion handler wired. The agent's intended output type is therefore not actuatable; zero safe items and zero noop is consistent with an unbufferable create_discussion attempt with no fallback. The downstream Process Safe Outputs node then hard-failed.
• Why it surfaced now: this workflow has appeared in prior audits only as an out-of-scope agent-job failure (e.g. 06-10). Today the agent job succeeded for the first time in the audited windows, exposing what is likely a long-latent config/prompt mismatch.
• Impact: the intended daily firewall report (a discussion) is not produced, and the production daily run shows red in the Actions UI.
• Exact error: NOT RECOVERABLE — the Process Safe Outputs step stdout is not pre-bundled and the audit MCP tool surfaces only a generic error_count=1. This is the 3rd distinct in-scope JOB-layer safe_outputs failure (after 06-23 & 06-26 Changeset Generator) whose exact error could not be read — a recurring observability gap.

Root Cause Analysis

Configuration / Prompt Drift

The primary defect is a mismatch between the task instructions (ask for a discussion) and the compiled safe-outputs config (no create_discussion enabled). A workflow should never instruct a safe-output type absent from its allowlist.

Observability

Three JOB-layer safe_outputs failures in the last ~3 weeks (06-23, 06-26 Changeset Generator; 07-11 Firewall) have all had unreadable exact errors because the Process Safe Outputs step log is not pre-bundled and the audit MCP tool does not surface it. Root-causing JOB-layer failures is currently guesswork.

API / Permission / Validation Issues

None observed at the message level this window (0 messages actuated).

Recommendations

Critical / Immediate

1. Reconcile the Firewall report prompt with its config — Priority: High
   • Root cause: task instructs create_discussion but only upload_asset/noop/missing_tool/missing_data are wired.
   • Action: EITHER add create-discussion (category audits) to the Daily Firewall Logs Collector safe-outputs frontmatter and recompile the lock, OR rewrite Step 6 to use a wired output (upload_asset the report, or drop the discussion step).
   • Affected: Daily Firewall Logs Collector and Reporter (production, daily).

Bug Fixes / Process Improvements

1. Pre-bundle Process Safe Outputs step stdout/stderr on failure — so JOB-layer failures are diagnosable. (Cross-cutting; affects Changeset Generator + Firewall.)
2. Defensive handler behavior — when the agent emits a safe-output type not enabled for the workflow, Process Safe Outputs should surface safe-output type <X> not enabled (allowed: ...) and soft-skip / clean no-op rather than hard-fail the whole job (mirroring create_pull_request_review_comment missing-context soft-skip).

Work Item Plans

Work Item 1: Fix Daily Firewall report output-type drift

• Type: Bug Fix (config) · Priority: High
• Description: daily-firewall-report.md Step 6 asks for a discussion; the compiled workflow has no create_discussion handler, so the safe_outputs job hard-fails and no report is published.
• Acceptance Criteria:
  • Daily Firewall Logs Collector emits its report through a wired safe-output (either create-discussion added to config, or upload_asset/removed).
  • A subsequent scheduled run completes with safe_outputs job = success and the report is delivered.
• Technical Approach: reconcile frontmatter safe-outputs: with the prompt; recompile lock; verify on next run.
• Estimated Effort: Small

Work Item 2: Recover JOB-layer failure logs (observability)

• Type: Enhancement · Priority: Medium
• Description: Pre-bundle Process Safe Outputs step logs on failure; 3 JOB-layer failures (06-23/06-26/07-11) had unreadable exact errors.
• Acceptance Criteria:
  • On a failed safe_outputs job, the Process Safe Outputs stdout/stderr is available in the run artifacts / audit MCP output.
• Estimated Effort: Small–Medium

Work Item 3: Defensive soft-skip for disallowed safe-output types

• Type: Enhancement · Priority: Medium
• Description: Emitting a safe-output type not in the allowlist should produce a clear message and soft-skip, not an opaque job failure.
• Acceptance Criteria:
  • A disallowed-type emission yields a named error and keeps safe_outputs job = success.
• Estimated Effort: Medium

Historical Context

Trends

• Error rate: read-only window, 1 production JOB-layer hard failure (93.3% run-level success) — a regression from the clean, write-rich 07-10 (100%, 48 messages).
• Standing-open items — all NOT exercised this read-only window:
  • Changeset Generator patch-format:BUNDLE bundle-transport hardfail: Changeset Generator absent (~15 days since the 06-26 actuating occurrence). Remediation UNVALIDATED. Still the highest-priority open production signature.
  • review_path_unresolved_422 Path-variant fallback (pr_review_buffer.cjs:554): no line-anchored review comments emitted (read-only) → UNVALIDATED for the 43rd consecutive audit.
  • Smoke target_star family (review-comment / add_comment→discussion permission / add_labels-no-context): no workflow_dispatch smoke in window → latent, open.
  • LintMonster update_issue target:triggering: LintMonster present (run-29138307820) but read-only (found no lint issues) — the 06-11 production hardfail neither recurred nor was re-exercised; target:'*' fix still UNVALIDATED.
• Cleared: Documentation Noob Tester (§29140125774, out-of-scope agent-job failure, safe_outputs=success) — its 07-10 upload_assets "Push assets" step failure did not recur (upload_assets=success today).

Metrics and KPIs

• Overall safe_outputs job success rate: 93.3% (14/15)
• Most reliable: all read-only runs (14/15) cleanly reached safe_outputs=success
• Most problematic: Daily Firewall Logs Collector (production JOB-layer hardfail)
• Messages actuated: 0 (read-only window)

Next Steps

• Reconcile Daily Firewall report prompt vs safe-outputs config (Work Item 1, High)
• Pre-bundle Process Safe Outputs failure logs (Work Item 2)
• Add defensive soft-skip for disallowed safe-output types (Work Item 3)
• Continue watching for the next actuating Changeset Generator run to validate the bundle-transport remediation (highest-priority, ~15 days unexercised)

References:

• §29137608538 — Daily Firewall Logs Collector (safe_outputs JOB failure)
• §29140125774 — Documentation Noob Tester (out-of-scope agent-job failure)
• §29138307820 — LintMonster (read-only, present but not exercised)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔒 Safe Output Health Monitor · 322.7 AIC · ⌖ 17 AIC · ⊞ 9.2K · ◷

• expires on Jul 11, 2026, 9:25 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Safe Output Health Monitor.

A newer discussion is available at Discussion #45039.