Support `COPILOT_MODEL=auto` in AWF validation and api-proxy resolution by Copilot · Pull Request #6474 · github/gh-aw-firewall

Copilot · 2026-07-21T12:29:29Z

AWF rejected the Copilot auto model before request handling, even though upstream now treats auto as a valid first-class model selector. This change lets auto pass host-side validation and remain intact through api-proxy resolution.

Host-side model validation
- Add auto to the accepted Copilot model catalog in src/copilot-model.ts
- Allow COPILOT_MODEL=auto in config assembly without tripping unsupported-model errors
API proxy model resolution
- Treat auto as a special pass-through in containers/api-proxy/model-resolver.js
- Preserve auto instead of rewriting it to a concrete model or applying fallback heuristics
Startup model checks
- Update startup requested-model validation to recognize auto as valid even when it does not appear in fetched provider model lists
Targeted coverage
- Add focused tests for:
  - host-side COPILOT_MODEL=auto acceptance
  - resolver pass-through behavior
  - no-rewrite behavior for request bodies containing model: "auto"
  - startup validation behavior for AWF_REQUESTED_MODEL=auto

Example:

{
  "env": {
    "COPILOT_MODEL": "auto"
  }
}

With this change, AWF accepts that configuration and forwards auto unchanged to the Copilot path.

Fixes Support for copilot auto model #6463

Copilot

Pull request overview

Adds first-class support for Copilot’s auto model selector across validation and proxy resolution.

Changes:

Accepts COPILOT_MODEL=auto.
Preserves auto through proxy resolution.
Adds targeted validation and pass-through tests.

Show a summary per file

File	Description
`src/copilot-model.ts`	Adds `auto` to supported Copilot models.
`src/copilot-model.test.ts`	Tests normalized `auto` validation.
`src/commands/validators/config-assembly-model-detection.test.ts`	Tests configuration acceptance.
`containers/api-proxy/model-resolver.js`	Implements special pass-through resolution.
`containers/api-proxy/model-resolver.test.js`	Tests resolution and body preservation.
`containers/api-proxy/server.startup-model-validation.test.js`	Tests startup validation without provider-list inclusion.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 6/6 changed files
Comments generated: 2
Review effort level: Medium

  const key = requestedModel.toLowerCase();
  const fallbackConfig = normalizeFallbackConfig(modelFallbackConfig);

+  if (key === 'auto') {


Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-07-21T15:39:06Z

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

lpcox · 2026-07-21T15:39:11Z

@copilot address review feedback

github-actions · 2026-07-21T15:41:04Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	98.74%	98.78%	📈 +0.04%
Statements	98.69%	98.72%	📈 +0.03%
Functions	99.28%	99.28%	➡️ +0.00%
Branches	94.95%	94.95%	➡️ +0.00%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/log-directory-setup.ts`	96.2% → 100.0% (+3.78%)	96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot · 2026-07-21T15:59:19Z

@copilot address review feedback

Addressed in 78ad4ef.

github-actions · 2026-07-21T16:27:28Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-07-21T16:27:29Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-21T16:27:30Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-07-21T16:27:31Z

✅ Smoke Claude passed

github-actions · 2026-07-21T16:27:31Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-07-21T16:27:32Z

📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-07-21T16:27:32Z

✅ Contribution Check completed successfully!

github-actions · 2026-07-21T16:27:33Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-07-21T16:28:09Z

❌ Smoke Copilot BYOK reports failed. BYOK mode investigation needed...

github-actions · 2026-07-21T16:30:40Z

🔬 Smoke Test Results

Test	Status
GitHub MCP	❌ Filtered by secrecy policy
GitHub.com Connectivity	✅ HTTP 200
File Write/Read	❌ Template variables not expanded (pre-step data unavailable)

Overall: FAIL — Pre-computed step data was not available (template variables unexpanded), and MCP access was filtered by secrecy policy.

📰 BREAKING: Report filed by Smoke Copilot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:30:54Z

One thing looks missing for CONTRIBUTING.md: Documentation. This PR changes user-facing model validation/handling (COPILOT_MODEL=auto / AWF_REQUESTED_MODEL=auto), so the repo docs should be updated to reflect that auto is now an accepted Copilot model selector. The contribution guide calls out both “Update documentation for any new features” and “Update documentation if needed” in the PR process.

The tests and PR description look good, but I’d add a small README/docs note so users know auto is supported.

Generated by Contribution Check for #6474 · 1.53 AIC · ⊞ 23.6K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:31:35Z

Smoke Test Results — FAIL ❌

Check	Result
Redis PING	❌ Name resolution / connection failed
PostgreSQL pg_isready	❌ No response
PostgreSQL SELECT 1	❌ Connection failed

host.docker.internal could not be resolved (DNS failure), and fallback attempts via 172.17.0.1 and localhost also failed — services appear unreachable from this runner environment.

Overall: FAIL

🔌 Service connectivity validated by Smoke Services
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:31:47Z

🔍 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Detail
1. Module Loading	✅	`otel.js` loads successfully; exports: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`
2. Test Suite	✅	59 tests passed, 0 failed (2 suites: `otel.test.js`, `otel-fanout.test.js`)
3. Env Var Forwarding	✅	`api-proxy-env-config.ts` forwards `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`, `GITHUB_AW_OTEL_TRACE_ID`, `GITHUB_AW_OTEL_PARENT_SPAN_ID`, `OTEL_SERVICE_NAME`
4. Token Tracker Integration	✅	`onUsage` callback exists in `token-tracker-http.js` (line 285, invoked at line 343)
5. OTEL Diagnostics	✅	`isEnabled()` returns `true`; file fallback to `/var/log/api-proxy/otel.jsonl` when no OTLP endpoint configured
6. Graceful Degradation	✅	Module loads without errors when no OTLP endpoint set; falls back to local NDJSON file

All 6 scenarios pass. OTEL tracing integration is fully functional.

📡 OTel tracing validated by Smoke OTel Tracing
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:31:47Z

Smoke test: Copilot Network Isolation — @lpcox

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com): HTTP 200
✅ Blocked domain (example.com): connection denied (403 from proxy)

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:31:48Z

Chroot Version Comparison ✅

All runtimes match between host and chroot environments.

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.13	✅
Node.js	v24.18.0	v24.18.0	✅
Go	go1.22.12	go1.22.12	✅

Tested by Smoke Chroot
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:32:02Z

Smoke Test: Claude Engine Validation

Check	Result
API Status	✅ PASS
GH Check	✅ PASS
File Status	✅ PASS

Overall Result: PASS ✅

Generated by Smoke Claude for #6474 · 36.7 AIC · ⊞ 3.3K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:32:34Z

@lpcox

GitHub MCP connectivity: ✅
GitHub.com connectivity: ✅
File write/read: ✅
BYOK inference: ✅

Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
Overall status: PASS

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:32:37Z

Smoke Test Results

PR Titles: Access Denied (Filtered by secrecy policy)
GitHub MCP Testing: ❌
GitHub.com Connectivity: ❌ (HTTP 000)
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:32:58Z

@lpcox
❌ Test 1 (PR Data): MCP error: filtered by secrecy policy
✅ Test 2 (GitHub.com): HTTP 200
✅ Test 3 (File I/O): 'test123'
✅ Test 4 (BYOK mode): Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)

Overall: FAIL

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:33:07Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	❌	N/A	❌ FAIL
Java	caffeine	❌	N/A	❌ FAIL
Node.js	clsx	✅	All passed	✅ PASS
Node.js	execa	✅	All passed	✅ PASS
Node.js	p-limit	✅	All passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 7/8 ecosystems passed — ❌ FAIL

❌ Failure Details

Java (gson, caffeine) — Both projects failed during mvn compile with a network error:

Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:3.4.0
from/to central (https://repo.maven.apache.org/maven2): Network is unreachable

Maven cannot reach Maven Central (repo.maven.apache.org) from this environment. The AWF firewall is blocking outbound connections to repo.maven.apache.org since it is not in the allowed domains list.

Generated by Build Test Suite for #6474 · 34.6 AIC · ⊞ 8.3K · ◷
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:33:15Z

🔬 Smoke Test: Copilot PAT Auth — PR #6474

Test	Result
GitHub MCP	✅ (PR list accessible)
GitHub.com connectivity	✅ HTTP 200
File write/read	⚠️ Template vars not expanded (pre-step skipped)

Overall: PASS ✅
Auth mode: PAT (COPILOT_GITHUB_TOKEN)

🔑 PAT report filed by Smoke Copilot PAT
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:33:28Z

Smoke test results:

GitHub reads: ❌ (authenticated wrapper unavailable in this shell)
Playwright title check: ✅
File write/read: ✅
Build (npm ci && npm run build): ✅
Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
_{Add label ready-for-aw to run again}

github-actions · 2026-07-21T16:34:51Z

🔬 Smoke Test Results — Docker Sbx

Test	Status
GitHub MCP connectivity	⚠️ filtered by secrecy policy
GitHub.com HTTP	⚠️ pre-step data unavailable (template not expanded)
File write/read	⚠️ pre-step data unavailable (template not expanded)

Overall: INDETERMINATE — pre-step outputs were not injected into the prompt (template variables unexpanded). MCP PR data filtered by secrecy policy.

/cc @lpcox

📰 BREAKING: Report filed by Smoke Docker Sbx
_{Add label ready-for-aw to run again}

Initial plan

f985f4b

Copilot AI assigned Copilot and lpcox Jul 21, 2026

Copilot started work on behalf of lpcox July 21, 2026 12:29 View session

Copilot AI linked an issue Jul 21, 2026 that may be closed by this pull request

Support for copilot auto model #6463

Closed

feat: support copilot auto model

a8ae2e5

Copilot AI changed the title ~~[WIP] Add support for copilot auto model~~ Support COPILOT_MODEL=auto in AWF validation and api-proxy resolution Jul 21, 2026

Copilot finished work on behalf of lpcox July 21, 2026 12:35

Copilot AI requested a review from lpcox July 21, 2026 12:35

lpcox marked this pull request as ready for review July 21, 2026 13:56

Copilot AI review requested due to automatic review settings July 21, 2026 13:56

Copilot started reviewing on behalf of lpcox July 21, 2026 13:57 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread containers/api-proxy/model-resolver.js Outdated

Comment thread containers/api-proxy/model-resolver.js Outdated

const key = requestedModel.toLowerCase();

const fallbackConfig = normalizeFallbackConfig(modelFallbackConfig);

if (key === 'auto') {

Potential fix for pull request finding

93ff32c

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot started work on behalf of lpcox July 21, 2026 15:50 View session

fix: allow copilot auto through ai-credit guard

78ad4ef

Copilot finished work on behalf of lpcox July 21, 2026 15:59

lpcox added the ready-for-aw label Jul 21, 2026

github-actions Bot added smoke-copilot-network-isolation Copilot network-isolation egress smoke test smoke-chroot labels Jul 21, 2026

github-actions Bot added the smoke-claude label Jul 21, 2026

lpcox temporarily deployed to aoai-model July 21, 2026 16:32 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok-aoai-entra label Jul 21, 2026

lpcox temporarily deployed to aoai-model July 21, 2026 16:32 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-pat label Jul 21, 2026

github-actions Bot mentioned this pull request Jul 21, 2026

[aw] Smoke Copilot BYOK failed #6478

Closed

lpcox merged commit 5a0f432 into main Jul 21, 2026
139 of 142 checks passed

lpcox deleted the copilot/support-for-copilot-auto-model branch July 21, 2026 16:48

github-actions Bot mentioned this pull request Jul 21, 2026

🩺 Runner Doctor UpdateRunner Doctor catalog update — B13 Squid ACL gap, B9 RHEL CA mount fix (2026-07-20–21 scan) #6483

Closed

Copilot AI mentioned this pull request Jul 21, 2026

Update Runner Doctor catalog for B13 Squid ACL gap and B9 RHEL/Amazon Linux CA mount fix #6485

Merged

This was referenced Jul 22, 2026

[aw] No-Op Runs #5883

Open

chore: update model-to-API mapping (2026-07-22) #6489

Merged

[Test Coverage] container-startup-diagnostics.ts coverage #6493

Merged

Uh oh!

Conversation

Copilot AI commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

lpcox commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

Copilot AI commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 21, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

Smoke Test Results — FAIL ❌

Uh oh!

github-actions Bot commented Jul 21, 2026

🔍 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

Chroot Version Comparison ✅

Uh oh!

github-actions Bot commented Jul 21, 2026

Smoke Test: Claude Engine Validation

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

Smoke Test Results

Uh oh!

github-actions Bot commented Jul 21, 2026

Uh oh!

github-actions Bot commented Jul 21, 2026

🏗️ Build Test Suite Results

❌ Failure Details

Uh oh!

github-actions Bot commented Jul 21, 2026

🔬 Smoke Test: Copilot PAT Auth — PR #6474

Copilot AI commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading

github-actions Bot commented Jul 21, 2026 •

edited

Loading