[aw-failures] Intermittent Copilot SDK auth failure ("Personal Access Tokens are not supported") aborts PR reviewer agents at turn 1

[github-actions]

Summary

• Recommendation: Fix Copilot token selection in copilot-sdk-driver / copilot-harness so the first auth attempt presents a Copilot-compatible token instead of a Personal Access Token (PAT) — this single auth fault accounts for all observed PR-reviewer agent failures in the 6h window.
• Failure class: Authentication (isAuthenticationFailedError=true), pre-inference. Agent aborts at turn 1 with 0 tokens consumed.
• Affected workflows: PR Code Quality Reviewer (8 failures), Matt Pocock Skills Reviewer (8 failures) — 16 failures total in the last 6h.
• Coverage gap: PR Code Quality Reviewer has no tracking issue; Matt Pocock Skills Reviewer only has ephemeral per-run [aw] ... failed notices (e.g. [aw] Matt Pocock Skills Reviewer failed #37967, [aw] Matt Pocock Skills Reviewer failed #37962) that auto-close without capturing the shared root cause. No consolidated/root-cause issue exists.
• Status: ⚠️ Intermittent — the same workflows succeed on sibling copilot/* branches, so this is a token-selection/refresh race, not a configuration error.

Problem statement

The Copilot agent engine fails authentication on its first attempt with HTTP 400 from the token-check endpoint, then exhausts its retries and fails the agent job. Because the failure occurs before any model turn, the run produces no output (agent_output.json = {"items": []}), 0 token usage, and 1 turn. The two PR-review agentic workflows are the most affected; their failures interleave with successes on other PRs in the same window.

Evidence

Representative failed run: §27173437914 (PR Code Quality Reviewer). Comparator successful run: §27177131901.

1. copilot-harness log (failed run): error: 400 400 checking third-party user token: bad request: Personal Access Tokens are not supported for this endpoint, immediately followed by attempt 1 failed: ... isAuthenticationFailedError=true ... retriesRemaining=3, then Process completed with exit code 1.
2. Same signature reproduced on §27171904123 (PR Code Quality Reviewer) and §27173437929 (Matt Pocock Skills Reviewer).
3. audit-diff (base = success 27177131901, compare = fail 27173437914): failed run has turns=1, token_usage=0, and is missing the github.com:443 and o205451.ingest.us.sentry.io firewall domains the successful run reaches — confirming the agent aborts before doing real work. No MCP, firewall-policy, or network-anomaly deltas (anomaly_count=0).
4. Failed and successful runs are both pull_request events on same-repo copilot/* branches (e.g. failed copilot/fix-no-op-runs-expiry-issue, success copilot/update-hide-older-comments) — so this is not a fork-PR permission artifact.

Affected run IDs (16 failures, 2026-06-08 21:02–23:25 UTC)

PR Code Quality Reviewer: 27173437914, 27173254834, 27172824917, 27172683273, 27171904123, 27168015414, 27167064771, 27166758961

Matt Pocock Skills Reviewer: 27173437929, 27173254866, 27172824922, 27172683242, 27171904113, 27168015415, 27167064774, 27166758646

Probable root cause

On the first Copilot SDK auth attempt the harness presents a Personal Access Token to an endpoint that rejects PATs (Personal Access Tokens are not supported for this endpoint). The harness retries with the same rejected credential, so all retries fail identically and the job aborts. Intermittency across sibling copilot/* branches points to a token-acquisition/refresh race rather than a static misconfiguration.

Proposed remediation

1. In copilot-sdk-driver / copilot-harness, ensure attempt 1 uses the Copilot-compatible token rather than a PAT for the third-party-user-token check.
2. On isAuthenticationFailedError, re-acquire/refresh the token before retrying instead of replaying the same rejected credential; add backoff + jitter.
3. Add a startup assertion (or pre-agent audit check) that detects a PAT-vs-Copilot-token mismatch and fails fast with a clear message.
4. Emit a metric/log marker for isAuthenticationFailedError so this signature is trackable without log archaeology.

Success criteria / verification

1. Over a 24h window, the auth-failure rate for PR Code Quality Reviewer and Matt Pocock Skills Reviewer drops to ~0.
2. No Personal Access Tokens are not supported for this endpoint entries appear in copilot-harness logs.
3. No agent runs for these workflows terminate at turn 1 with 0 token usage due to auth.

Existing issue correlation

• PR Code Quality Reviewer: no existing tracking issue (open or closed).
• Matt Pocock Skills Reviewer: ephemeral per-run trackers [aw] Matt Pocock Skills Reviewer failed #37967, [aw] Matt Pocock Skills Reviewer failed #37962 (auto-closed) — neither captures the shared auth root cause.
• All other agentic failures in the 6h window (Issue Monster, PR Triage Agent, Daily Sentrux Report, Daily Model Inventory Checker, PR Sous Chef, Contribution Check, Daily Observability Report) already have dedicated [aw] ... failed tracking issues and are out of scope here. CJS/CGO/CI failures are standard JS/Go test CI, not agentic, and the intentionally-broken credit-limit test workflows are excluded.

References: §27173437914, §27171904123, §27177131901

Generated by 🔍 [aw] Failure Investigator (6h) · 202.5 AIC · ⌖ 12.5 AIC · ⊞ 5.3K · ◷

• expires on Jun 15, 2026, 5:40 PM UTC-08:00

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · 31.8 AIC · ⌖ 4.34 AIC · ◷