[awf] WIF auth regressed in firewall v0.27.0 — agent bypasses proxy (worked in v0.25.58)

[benissimo]

Summary

Anthropic WIF auth (engine.auth: { type: github-oidc, provider: anthropic }) worked end-to-end on gh-aw-firewall v0.25.58 but is broken on v0.27.0 — in two distinct ways depending on whether the ANTHROPIC_API_KEY placeholder workaround (from #4117) is present. v0.27.0 is a regression: the same workflow that succeeded on 0.25.58 now fails.

This is separate from #4117 (the awf agent-bypass bug, whose fix is closed but not yet in any released firewall version). On 0.25.58 the documented placeholder workaround made WIF succeed. On 0.27.0 the workaround no longer works and the no-placeholder path fails differently.

Evidence matrix

Same workflow (engine.auth github-oidc + anthropic), same federation rule, same workspace. Only the firewall version and the placeholder env var differ:

Firewall	ANTHROPIC_API_KEY placeholder	Proxy startup	awf-reflect anthropic	Result
v0.25.58	present	Anthropic=true	configured: true	✅ 200 — WIF used, placeholder ignored
v0.27.0	absent	Anthropic=false	n/a	❌ 503 Anthropic OIDC token unavailable; retry shortly
v0.27.0	present	Anthropic=true	configured: true	❌ 401 Invalid API key

(0.25.58 result is the historical proven run; see caveat at the bottom.)

The 401 case is the most diagnostic (v0.27.0 + placeholder)

The agent-side health check passes — credentials are correctly isolated:

[health-check] ✓ Anthropic credentials NOT in agent environment (correct)
[health-check] ✓ ANTHROPIC_AUTH_TOKEN is placeholder value (correct)
[health-check] ✓ Anthropic API proxy is reachable at http://172.30.0.30:10001
[health-check] ✓ All API proxy health checks passed

The proxy reflects a working Anthropic endpoint (WIF init succeeded):

{ "provider": "anthropic", "port": 10001, "base_url": "http://api-proxy:10001", "configured": true }

But the agent container bypasses the proxy and goes direct to Anthropic with the placeholder key. From the Squid access.log (proxy is 172.30.0.30; the agent container is 172.30.0.20):

172.30.0.20:55464 api.anthropic.com:443 ... CONNECT 200 TCP_TUNNEL:HIER_DIRECT api.anthropic.com:443 "-"
172.30.0.20:55480 api.anthropic.com:443 ... CONNECT 200 TCP_TUNNEL:HIER_DIRECT api.anthropic.com:443 "-"
... (one per retry)

Claude Code then reports:

API Error: 401 — Invalid API key · Fix external API key

Because the agent never gets ANTHROPIC_BASE_URL set, it sends the fake placeholder straight to api.anthropic.com → 401. This is the #4117 agent-bypass symptom, but the placeholder workaround that fixed it on 0.25.58 no longer triggers ANTHROPIC_BASE_URL injection on 0.27.0.

The 503 case (v0.27.0, no placeholder)

Without the placeholder, the proxy never sets up the Anthropic path at all:

[INFO] API proxy enabled: OpenAI=false, Anthropic=false, Copilot=false, Gemini=false
[WARN] ⚠️  API proxy enabled but no API keys found in environment

Every completion call then returns:

API Error: 503 {"error":"Anthropic OIDC token unavailable; retry shortly"}

10 retries, then timeout. (Note: AWF_AUTH_TYPE=github-oidc / AWF_AUTH_PROVIDER=anthropic / AWF_AUTH_ANTHROPIC_* are all present in the agent step env — the proxy just doesn't key off them to enable the Anthropic provider.)

Why this matters

gh-aw v0.79.4 bundles firewall v0.27.0, so the current stable gh-aw release cannot run WIF workflows at all. Teams that were on the 0.25.58 + placeholder workaround and upgrade to 0.79.4 will see their WIF workflows break.

Suggested regression guards

The thing that stings here is that 0.27.0 shipped behaving worse than 0.25.58 for WIF, and the release pipeline didn't catch it. Some ideas:

1. Add a WIF E2E smoke test to the firewall release pipeline. A workflow_dispatch workflow using engine.auth: github-oidc + provider: anthropic that makes a single /v1/messages call and asserts 200. Run it against every firewall RC before tagging.
2. Test both code paths. WIF-only (no static key) and WIF + placeholder exercise different branches of awf's ANTHROPIC_BASE_URL injection logic. Both regressed here in different ways, so both need coverage.
3. Assert at the proxy/access-log level that the agent routes through the api-proxy — i.e. no HIER_DIRECT tunnels to api.anthropic.com from the agent container IP. This catches the whole "agent bypass" class of bug ([awf] ANTHROPIC_BASE_URL not set in agent container when using engine.auth WIF (no static ANTHROPIC_API_KEY) #4117 and this one) regardless of the surface error.
4. Track a last-known-good WIF firewall version in CI and fail the release if the WIF smoke test regresses relative to it.

Caveat (full disclosure)

The 0.25.58 success is the historical proven run, which used a different service account (svac_01SuCAMo...) than the 0.27.0 failures (svac_019jzdxf...). I have not re-run 0.25.58 with the new SA. However, the failure mode is service-account-independent: the proxy's WIF init succeeds (configured: true), and the failure is the agent container bypassing the proxy and sending the placeholder key directly — which is firewall routing behavior, not an SA/federation issue. An SA misconfiguration would surface as a proxy-side WIF exchange failure, not an agent-side direct call with a fake key.

Version info

• Working: gh-aw v0.77.5 → firewall v0.25.58
• Broken: gh-aw v0.79.4 → firewall v0.27.0 (also reproduced conceptually against 0.27.1)
• Related: [awf] ANTHROPIC_BASE_URL not set in agent container when using engine.auth WIF (no static ANTHROPIC_API_KEY) #4117 (agent-bypass workaround), gh-aw#36460/#36461 (detection-job id-token:write, a separate and already-fixed compiler issue)