Skip to content

fix: unknown model pricing — better error options and accurate issue title#45985

Merged
pelikhan merged 5 commits into
mainfrom
copilot/update-model-pricing-message
Jul 16, 2026
Merged

fix: unknown model pricing — better error options and accurate issue title#45985
pelikhan merged 5 commits into
mainfrom
copilot/update-model-pricing-message

Conversation

Copilot AI commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

When max-ai-credits is active and the model isn't in the built-in pricing table, the failure message had a misleading option and the generated issue title said "timed out" even when the root cause was the missing pricing.

Changes

unknown_model_ai_credits.md

  • Added Option 2: add model pricing directly in frontmatter via models.providers:
    models:
      providers:
        openai:            # github-copilot | anthropic | openai | google
          models:
            my-custom-model:
              cost:
                input: "3.75e-06"
                output: "1.5e-05"
  • Removed the incorrect defaultAiCreditsPricing: 3.0 option — wrong type (object not scalar) and wrong location (lives under apiProxy, not models)

handle_agent_failure.cjs

  • Added unknownModelAICredits to buildFailureIssueTitle with priority above isTimedOut
  • When both flags are true (unknown pricing causes the run to exhaust its time budget), the issue title is now [aw] {workflow} has unknown model pricing instead of the misleading timed out
  • Passes the flag at the call site

Tests

  • Added case for unknownModelAICredits in buildFailureIssueTitle parameterized tests
  • Added explicit test asserting unknownModelAICredits beats isTimedOut when both are set

Copilot AI and others added 3 commits July 16, 2026 12:00
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Add Option 2 in unknown_model_ai_credits.md showing how to add
  model pricing via models.providers in frontmatter
- Remove the incorrect defaultAiCreditsPricing: 3.0 sample (Option 3)
  and renumber remaining options
- Add unknownModelAICredits flag to buildFailureIssueTitle with
  higher priority than isTimedOut, so the issue title accurately
  reflects the model pricing error instead of showing 'timed out'
- Pass unknownModelAICredits at the call site
- Add test cases for the new flag and priority behavior

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title fix: update unknown model pricing message and issue title fix: unknown model pricing — better error options and accurate issue title Jul 16, 2026
Copilot AI requested a review from pelikhan July 16, 2026 12:17
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot give examples for all fields (cached...)

@pelikhan
pelikhan marked this pull request as ready for review July 16, 2026 12:39
Copilot AI review requested due to automatic review settings July 16, 2026 12:39
…cing example

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@copilot give examples for all fields (cached...)

Done — the cost block in Option 2 now shows all five fields with inline comments:

cost:
  input: "3.75e-06"      # $3.75 per million input tokens (required)
  output: "1.5e-05"      # $15.00 per million output tokens (required)
  cache_read: "3.75e-07" # $0.375 per million cached-read tokens (optional)
  cache_write: "4.5e-06" # $4.50 per million cache-write tokens (optional)
  reasoning: "1.5e-05"   # $15.00 per million reasoning tokens (optional, defaults to output price)

The trailing text also clarifies that only input and output are required.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request improves the “unknown model pricing” failure experience when max-ai-credits is enabled by updating the remediation guidance and ensuring the generated failure issue title reflects the true root cause (unknown pricing) rather than a secondary symptom (timeout).

Changes:

  • Update the unknown-model AI credits guidance to include frontmatter-based per-model pricing via models.providers and remove the previously misleading fallback suggestion.
  • Add unknownModelAICredits as a higher-priority signal in buildFailureIssueTitle, including propagating the flag at the call site.
  • Extend the buildFailureIssueTitle tests to cover the new title and precedence over isTimedOut.
Show a summary per file
File Description
actions/setup/md/unknown_model_ai_credits.md Updates the end-user remediation options shown for unknown model pricing failures.
actions/setup/js/handle_agent_failure.cjs Adds unknownModelAICredits handling to issue title generation and passes the flag through.
actions/setup/js/handle_agent_failure.test.cjs Adds test coverage for the new title and precedence behavior.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Low

Comment on lines +27 to 28
Use the `models.providers` field to supply per-token pricing for your custom model. Use the provider key that matches your engine (`github-copilot`, `anthropic`, `openai`, `google`):

---
```

Use the provider key matching your engine: `github-copilot` (Copilot), `anthropic` (Claude), `openai` (Codex), or `google` (Gemini).
Comment on lines 295 to +300
// Keep HTTP 400 below AI-credits signals: quota/rate-limit indicates an account-level
// budget state that should take precedence when both classes are detected.
if (options.http400ResponseError) return `[aw] ${workflowName} hit HTTP 400 bad request`;
// Unknown model pricing is a configuration error that may also trigger a timeout;
// report it explicitly so the title is not misleadingly "timed out".
if (options.unknownModelAICredits) return `[aw] ${workflowName} has unknown model pricing`;
@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category bug
Risk 🟢 Low
Score 58 (impact 25 + urgency 18 + quality 15)
Action batch_review
Batch bug-small (with #45982)

Rationale: Small targeted fix (3 files, +29/-8). Fixes misleading error message and wrong issue title for unknown model pricing. Includes tests. Batch with #45982 (similar small bug fixes).

Generated by 🔧 PR Triage Agent · 32.8 AIC · ⌖ 4.9 AIC · ⊞ 5.6K ·

@pelikhan
pelikhan merged commit fe56fc6 into main Jul 16, 2026
14 checks passed
@pelikhan
pelikhan deleted the copilot/update-model-pricing-message branch July 16, 2026 13:20
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.82.11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants