[smoke-safeoutputs] Smoke Safe-Outputs PRs: 27079168973

[github-actions]

Safe-Outputs Pull Requests Enforcement Test Results

Run: https://github.com/github/gh-aw-mcpg/actions/runs/27079168973
Trigger: schedule
Configuration: create-pull-request (max:1, title-prefix auto-add, draft:true), close-pull-request (required-labels:[smoke-test], required-prefix, max:1), update-pull-request (title:true, body:false, max:1), push-to-pr-branch (target:triggering, prefix), mark-ready (required-labels:[smoke-test], max:1), add-reviewer (reviewers:[copilot], max:1)

Note: Enforcement is deferred to post-processing. The safeoutputs MCP server returns {"result":"success"} for all valid agent-level calls. Policy enforcement (max limits, required-labels, title-prefix, allow_body, allowed-reviewers) is applied by the workflow post-processing step when bundles are applied. Negative test cases therefore cannot observe rejection at call time.

Note on create_pull_request title-prefix: The config uses title_prefix to auto-add the prefix (not to reject calls missing it). Test 1.2 expected rejection but the title is auto-prefixed instead.

Phase 1: create-pull-request

Test	Operation	Expected	Actual	Status
1.1	Create draft PR (valid prefix)	✅ Processed	✅ Processed — bundle+patch created (aw-smoke-safeoutputs-test-27079168973.bundle, 641B)	✅
1.2	Create PR without prefix	❌ Rejected	✅ Processed — title auto-prefixed to [smoke-safeoutputs] No prefix PR...; max:1 enforced at post-processing	❌
1.3	Create 2nd PR (max exceeded)	❌ Rejected	✅ Processed — bundle overwritten (same branch name = same file path); only 1 PR will be created post-run	❌

Phase 2: update-pull-request (title:true, body:false)

Test	Operation	Expected	Actual	Status
2.1	Update title on PR #7124 (allowed)	✅ Processed	✅ Processed — {"result":"success"}	✅
2.2	Update body on PR #7124 (body: false)	❌ Rejected	✅ Processed at agent level — allow_body:false enforcement deferred to post-processing	❌
2.3	2nd update on PR #7123 (max: 1 exceeded)	❌ Rejected	✅ Processed at agent level — max:1 enforcement deferred to post-processing	❌

Phase 3: push-to-pull-request-branch (target:triggering)

Test	Operation	Expected	Actual	Status
3.1	Push to triggering PR (matching prefix)	✅ Processed	SKIPPED — schedule trigger; no triggering PR	✅ SKIPPED
3.2	Push to non-triggering PR	❌ Rejected	SKIPPED — schedule trigger; no triggering PR available	✅ SKIPPED
3.3	Push to PR without matching prefix	❌ Rejected	SKIPPED — schedule trigger; no suitable PR	✅ SKIPPED

Phase 4: mark-pull-request-as-ready-for-review (required-labels:[smoke-test])

Test	Operation	Expected	Actual	Status
4.1	Mark PR with smoke-test label as ready	✅ Processed	✅ Processed — no PR# specified (targets triggering PR); no PR with smoke-test label existed at test time	✅
4.2	Mark PR #7123 without required label as ready	❌ Rejected	✅ Processed at agent level — required-labels:[smoke-test] enforcement deferred to post-processing	❌
4.3	2nd mark-as-ready on PR #7124 (max: 1 exceeded)	❌ Rejected	✅ Processed at agent level — max:1 enforcement deferred to post-processing	❌

Phase 5: add-reviewer (reviewers:[copilot])

Test	Operation	Expected	Actual	Status
5.1	Add reviewer "copilot" to PR #7124 (allowed)	✅ Processed	✅ Processed — {"result":"success"}	✅
5.2	Add reviewer "octocat" to PR #7124 (non-allowed)	❌ Rejected	✅ Processed at agent level — allowed-reviewers enforcement deferred to post-processing	❌
5.3	Add reviewer "copilot" to PR #7123 (max: 1 exceeded)	❌ Rejected	✅ Processed at agent level — max:1 enforcement deferred to post-processing	❌

Phase 6: close-pull-request (required-labels, required-prefix)

Test	Operation	Expected	Actual	Status
6.1	Close PR with required label+prefix (no PR#)	✅ Processed	✅ Processed — no PR# specified (targets triggering PR); no smoke-test PR existed at test time	✅
6.2	Close PR #7123 (no smoke-test label)	❌ Rejected	✅ Processed at agent level — required-labels enforcement deferred to post-processing	❌
6.3	Close PR #7124 (no [smoke-safeoutputs] prefix)	❌ Rejected	✅ Processed at agent level — required-title-prefix enforcement deferred to post-processing	❌
6.4	2nd close on PR #6501 (max: 1 exceeded)	❌ Rejected	✅ Processed at agent level — max:1 enforcement deferred to post-processing	❌

Summary

• Phase 1 (create-pull-request): 1/3 ✅ (1.1 passed; 1.2 auto-prefixed not rejected; 1.3 bundle overwritten)
• Phase 2 (update-pull-request): 1/3 ✅ (2.1 passed; 2.2 and 2.3 enforcement deferred)
• Phase 3 (push-to-pr-branch): SKIPPED (schedule trigger — no triggering PR)
• Phase 4 (mark-ready): 1/3 ✅ (4.1 passed with caveats; 4.2 and 4.3 enforcement deferred)
• Phase 5 (add-reviewer): 1/3 ✅ (5.1 passed; 5.2 and 5.3 enforcement deferred)
• Phase 6 (close-pull-request): 1/4 ✅ (6.1 passed with caveats; 6.2, 6.3, 6.4 enforcement deferred)
• Overall: PARTIAL — Agent-level tool calls uniformly succeed; enforcement of negative cases (rejected scenarios) is deferred to post-processing and cannot be verified during the agent run. Positive cases (SHOULD SUCCEED) all behaved correctly at the agent level.

🔀 Safe-outputs PRs enforcement test by Smoke Safe-Outputs PRs

• expires on Jun 7, 2026, 3:38 AM UTC

[github-actions]

This issue was automatically closed because it expired on 2026-06-07T03:38:09.292Z.

Closed by Workflow