Skip to content

Unblock AI credits guard for Claude Fable/Mythos 5 by adding curated pricing and catalog fallback entries#5997

Merged
lpcox merged 3 commits into
mainfrom
copilot/fix-claude-fable-5-pricing
Jul 8, 2026
Merged

Unblock AI credits guard for Claude Fable/Mythos 5 by adding curated pricing and catalog fallback entries#5997
lpcox merged 3 commits into
mainfrom
copilot/fix-claude-fable-5-pricing

Conversation

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

apiProxy.maxAiCredits rejected claude-fable-5 as unknown_model_ai_credits because the model was missing from both pricing sources used by the guard. This caused workflows pinned to the model to fail at first inference despite the model being recognized elsewhere in AWF.

  • Pricing source 1: Curated AI credits table

    • Added explicit Anthropic pricing rows in containers/api-proxy/ai-credits-pricing.js for:
      • claude-fable-5
      • claude-mythos-5
    • Rates match the expected $/1M token structure used by the guard (input, cachedInput, cacheWrite, output).
  • Pricing source 2: Bundled models.dev fallback catalog

    • Updated containers/api-proxy/models.dev.catalog.json to include:
      • anthropic/claude-fable-5
      • anthropic/claude-mythos-5
    • Ensures fallback resolution works even when curated mappings lag future model additions.
  • Guard/canonical-resolution coverage

    • Extended containers/api-proxy/guards/ai-credits-guard.test.js to cover non-rejection and credit computation for claude-fable-5 (and non-rejection for claude-mythos-5) when max AI credits is active.
    • Extended containers/api-proxy/models-dev-catalog.test.js to assert bundled catalog pricing resolution for both new Claude models.
// containers/api-proxy/ai-credits-pricing.js
'claude-fable-5':  { input: 10.00, cachedInput: 1.00, cacheWrite: 12.50, output: 50.00 },
'claude-mythos-5': { input: 10.00, cachedInput: 1.00, cacheWrite: 12.50, output: 50.00 },

Copilot AI changed the title [WIP] Fix AI-credits guard to include pricing for claude-fable-5 Unblock AI credits guard for Claude Fable/Mythos 5 by adding curated pricing and catalog fallback entries Jul 7, 2026
Copilot AI requested a review from lpcox July 7, 2026 22:32
Copilot finished work on behalf of lpcox July 7, 2026 22:32
@lpcox lpcox requested a review from Copilot July 7, 2026 22:43
@lpcox lpcox marked this pull request as ready for review July 7, 2026 22:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request updates the API proxy’s AI-credits guard pricing sources so Anthropic’s claude-fable-5 and claude-mythos-5 are treated as priced/known models when apiProxy.maxAiCredits is enabled, preventing unknown_model_ai_credits rejections for workflows pinned to those models.

Changes:

  • Added curated AI-credits pricing rows for claude-fable-5 and claude-mythos-5.
  • Extended bundled models.dev catalog coverage and added tests to validate catalog-based pricing resolution for the new Claude models.
  • Added guard tests to ensure maxAiCredits-active runs do not reject these models and that credits are computed for claude-fable-5.
Show a summary per file
File Description
containers/api-proxy/ai-credits-pricing.js Adds curated $/1M token pricing for claude-fable-5 and claude-mythos-5 to prevent “unknown model” pricing gaps.
containers/api-proxy/models.dev.catalog.json Adds bundled catalog entries for anthropic/claude-fable-5 and anthropic/claude-mythos-5 to keep fallback pricing resolution working when curated tables lag.
containers/api-proxy/models-dev-catalog.test.js Adds assertions that bundled catalog pricing resolves correctly for the two new Claude models.
containers/api-proxy/guards/ai-credits-guard.test.js Adds regression coverage ensuring these models are not rejected under AWF_MAX_AI_CREDITS and validates credits computation for claude-fable-5.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/4 changed files
  • Comments generated: 0
  • Review effort level: Low

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini completed. All facets verified. 💎

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

Smoke test summary already posted; no further action needed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Contribution Check completed successfully!

PR #5997 follows the applicable CONTRIBUTING.md guidelines based on the pre-fetched metadata, diff, and contribution guide: tests are included for the pricing/catalog change, files are in appropriate containers/api-proxy locations, and the PR description is clear. No contribution-guidelines comment needed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Security Guard failed. Please review the logs for details.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude passed

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 99.00% 99.00% ➡️ +0.00%
Statements 98.95% 98.95% ➡️ +0.00%
Functions 99.72% 99.72% ➡️ +0.00%
Branches 95.44% 95.41% 📉 -0.03%

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions github-actions Bot mentioned this pull request Jul 8, 2026
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API Status ✅ PASS
GH Check ✅ PASS
File Status ✅ PASS

Overall Result: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Smoke Claude for #5997 · 35 AIC · ⊞ 3.3K ·
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode

✅ GitHub MCP connectivity verified
✅ GitHub.com HTTP 200 OK
✅ File write/read test passed
✅ Direct BYOK inference path functional

Status: PASS | Running via api-proxy → api.githubcopilot.com

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Services Connectivity

  • Redis PING: ❌ Network unreachable
  • PostgreSQL pg_isready: ❌ No response
  • PostgreSQL SELECT 1: ❌ Network unreachable

Overall: FAILhost.docker.internal (172.17.0.1) is not reachable from this runner. Service containers may not be running or the network bridge is not configured.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.13 Python 3.12.3 ❌ NO
Node.js v24.18.0 v22.23.1 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results

Test Status
GitHub.com connectivity (HTTP 200)
File write/read (smoke-test-copilot-28908045400.txt)
GitHub MCP connectivity

Overall: PASS 🎉

cc @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode — Azure OpenAI (Foundry, Entra / GitHub OIDC)\n• GitHub MCP connectivity: ✅\n• GitHub.com connectivity: ✅\n• File write/read: ✅\n• BYOK inference: ✅\n\nRunning in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra\n\nOverall: PASS\n\n@lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔍 API Proxy OTEL Tracing Smoke Test Results

Scenario Result Details
1. Module Loading ✅ Pass otel.js loads; exports: startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled
2. Test Suite ✅ Pass 59 tests pass (39 in otel.test.js, 20 in otel-fanout.test.js)
3. Env Var Forwarding ✅ Pass api-proxy-env-config.ts forwards GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, GITHUB_AW_OTEL_TRACE_ID, GITHUB_AW_OTEL_PARENT_SPAN_ID, OTEL_SERVICE_NAME
4. Token Tracker Integration ✅ Pass onUsage callback at line 283 and onSpanEnd at lines 294/316/350 in token-tracker-http.js
5. OTEL Diagnostics ✅ Pass File fallback exporter writes to /var/log/api-proxy/otel.jsonl when no OTLP endpoint; fan-out mode active when GH_AW_OTLP_ENDPOINTS set

All 5 scenarios pass. OTEL tracing integration is complete and functioning correctly.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Gemini Engine Smoke Test Results

  • GitHub MCP Testing: ❌ (Tools missing)
  • GitHub.com Connectivity: ❌ (Failed)
  • File Writing Testing: ✅
  • Bash Tool Testing: ✅

Overall Status: FAIL

Last detected PR: #5982

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

PR titles: 5986 feat: add build-tools to SHA digest pinning infrastructure; 5984 fix: add diagnostic EACCES guard to workdir setup. Results: PR list ✅; merged PRs ✅; GitHub title ✅; file write ✅; discussion query ✅; build ✅. Overall: PASS

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • awmgmcpg
  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Smoke Test: PAT Auth ✅ PASS

@lpcox — Auth mode: PAT (COPILOT_GITHUB_TOKEN)

Test Result
GitHub MCP connectivity
GitHub.com HTTP ✅ 200
File write/read

Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 PAT report filed by Smoke Copilot PAT
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

$(sed s///g /tmp/comment_body.txt)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Build Test Suite for #5997 · 51.7 AIC · ⊞ 6.9K ·
Add label ready-for-aw to run again

@lpcox lpcox merged commit 119e4d8 into main Jul 8, 2026
86 of 90 checks passed
@lpcox lpcox deleted the copilot/fix-claude-fable-5-pricing branch July 8, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment