feat(usage): user-configurable per-model cost overlay (providers.<name>.modelCosts) - #1010
feat(usage): user-configurable per-model cost overlay (providers.<name>.modelCosts)#1010harryzhou2000 wants to merge 11 commits into
Conversation
|
✅ PR quality gates passed This pull request now targets The title was left unchanged. Its existing draft status has been preserved. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds ChangesProvider cost overlay
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant ConfigLoader
participant UserCostOverlays
participant CostEstimator
participant UsageSummaryCache
participant Logs
ConfigLoader->>UserCostOverlays: refreshUserCostOverlays(config)
UserCostOverlays-->>CostEstimator: active overlays and version
CostEstimator->>CostEstimator: resolve user price before built-in prices
CostEstimator-->>UsageSummaryCache: store estimate with overlay version
UsageSummaryCache->>UsageSummaryCache: reject stale overlay versions
CostEstimator-->>Logs: estimate with source "user"
Logs-->>Logs: map provider_cost_overlay to localized detail
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs-site/src/content/docs/reference/configuration/providers.md`:
- Line 69: Update the `modelCosts` documentation on line 69 to explicitly
clarify that custom or internal provider model IDs—even those absent from
built-in catalogs—are valid overlay keys for cost configuration. Add a note
stating that users can define custom prices for any model identifier regardless
of catalog presence, making clear that the feature supports providers beyond the
standard built-in offerings.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d0d55d83-385c-4beb-b349-119980acd24b
📒 Files selected for processing (16)
docs-site/src/content/docs/reference/configuration/providers.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tsgui/src/pages/Logs.tsxsrc/config.tssrc/server/auth-cors.tssrc/server/management/shared.tssrc/types.tssrc/usage/cost.tssrc/usage/user-cost-overlays.tstests/provider-cost-overlay-config.test.tstests/usage-cost.test.ts
Add providers.<name>.modelCosts (model -> input/output/cacheRead/ cacheWrite in USD per 1M tokens) so operators can price internal/custom providers whose ids do not match the compiled catalogs, or whose actual costs vary from list prices. - modelCosts rows win over the jawcode catalog and the expected-price overlay in resolveMatchedPrice (exact provider/model match; all-zero entries fall through to the catalogs). Follows ocx's flat per-model config convention (models, modelContextWindows, ...). - Rows are lifted from config at loadConfig and every persist path into a versioned registry; the estimator memo keys on that version so edits apply immediately and stale rows are never served. - New CostResult reason provider_cost_overlay is surfaced in the GUI logs detail with i18n strings. - config.json validation accepts only non-negative finite 4-tuples; the management API rejects malformed overlays; safeConfigDTO exposes the field to the dashboard. - Also replace a stray NUL byte in the price-memo cache-key template literal with a space separator. - Tests cover precedence, fall-through, registry refresh/memo invalidation, config round-trip, management validation, and DTO passthrough.
afc131d to
180b8b4
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs-site/src/content/docs/reference/configuration/providers.md`:
- Line 75: Update the modelCosts documentation to state the complete
display-price fallback order: user configuration, jawcode catalog,
expected-price overlay, then model-level vendor fallback. Clarify that an
all-zero user entry falls through to the next source in that sequence, while
preserving the existing display-only and non-billing behavior.
- Line 75: Add the modelCosts configuration row after modelMaxOutputTokens in
the Japanese, Korean, Russian, and Simplified Chinese provider pages,
documenting exact model-ID keys, all-zero entries falling back to built-in
catalogs, and display-only estimates that never affect billing.
In `@gui/src/i18n/de.ts`:
- Line 574: Update the German translation value for
logs.detail.estimate.provider_cost_overlay to “Ein vom Anbieter konfiguriertes
Preis-Overlay wurde verwendet.”, preserving alignment with the English catalog
entry.
In `@gui/src/pages/Logs.tsx`:
- Around line 45-49: Extend the GUI price-source contract used by
MatchedPriceInfo to include "user", then add the logs.detail.source.user
translation key to every locale alongside the existing source labels so
provider-configured overlay-priced rows display a localized source.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a4b0f2de-a70c-4109-a5d9-f87dcbe73c8d
📒 Files selected for processing (8)
docs-site/src/content/docs/reference/configuration/providers.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tsgui/src/pages/Logs.tsx
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 180b8b4326
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- Add JSDoc to the functions touched by this PR (loadConfig, persistConfigUnlocked, saveConfig, withRefreshedCostOverlays, safeConfigDTO, costResult, resolveMatchedPriceInner/Exact, estimateAttemptCost, validCost4, Logs key helpers) to satisfy the docstring-coverage gate. - GUI: MatchedPriceInfo.source accepts "user"; add logs.detail.source.user to all six locales so user-priced rows render a label instead of a raw key. - de.ts: fix provider_cost_overlay grammar (Ein vom Anbieter konfiguriertes Preis-Overlay ...). - safeConfigDTO: serialize only the four rate fields of modelCosts rows so extra hand-edited fields cannot leak to the dashboard; regression test added. - Docs: document the complete fallback order (user modelCosts -> jawcode catalog -> expected-price overlay -> vendor fallback) with all-zero fall-through in the providers reference, and add the modelCosts row to the ja/ko/ru/zh-cn provider pages.
|
Thanks for the review — addressed in 7a4ce62:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs-site/src/content/docs/ja/reference/configuration/providers.md`:
- Line 68: Update the Cost4 documentation row in
docs-site/src/content/docs/ja/reference/configuration/providers.md:68-68 to list
the supported input, output, cacheRead, and cacheWrite rate fields. Apply the
equivalent field list or example to
docs-site/src/content/docs/ko/reference/configuration/providers.md:68-68,
docs-site/src/content/docs/ru/reference/configuration/providers.md:78-78, and
docs-site/src/content/docs/zh-cn/reference/configuration/providers.md:67-67,
preserving each locale’s language and existing pricing guidance.
In `@gui/src/i18n/ru.ts`:
- Line 566: Update the Russian translation for
logs.detail.estimate.provider_cost_overlay at line 592 to reuse the same
terminology from logs.detail.source.user at line 566. Replace the current
"настроенная пользователем цена провайдера" phrasing with a sentence form that
incorporates "Ценовой оверлей провайдера" (keeping the "оверлей" term
consistent). This aligns the Russian translations with every other locale in the
PR, which reuses an identical term across both the price-source label and the
estimate-reason explanation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a3d12d7c-bcdf-4344-b9a0-61f4e65190d7
📒 Files selected for processing (18)
docs-site/src/content/docs/ja/reference/configuration/providers.mddocs-site/src/content/docs/ko/reference/configuration/providers.mddocs-site/src/content/docs/reference/configuration/providers.mddocs-site/src/content/docs/ru/reference/configuration/providers.mddocs-site/src/content/docs/zh-cn/reference/configuration/providers.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tsgui/src/pages/Logs.tsxsrc/config.tssrc/server/auth-cors.tssrc/server/management/shared.tssrc/usage/cost.tssrc/usage/user-cost-overlays.tstests/provider-cost-overlay-config.test.ts
…ology - ja/ko/ru/zh-cn provider references now list the four Cost4 rate fields (input, output, cacheRead, cacheWrite) with a JSON example. - ru.ts: logs.detail.estimate.provider_cost_overlay reuses the same "ценовой оверлей провайдера" term as logs.detail.source.user.
…verlay # Conflicts: # src/config.ts
|
Review readiness checklistThis PR is kept in draft until every requirement below is fulfilled. The tickable checklist has been added to your PR description — tick all four boxes there.
✅ 4/4 boxes ticked. ✅ All four boxes are ticked. |
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/server/auth-cors.ts`:
- Around line 525-539: Update sanitizeModelCosts to create out with a null
prototype so assigning the "__proto__" model ID creates an own property and
Object.keys includes it. Add a regression test using JSON.parse() to construct
the "__proto__" key and verify safeConfigDTO exposes its valid cost row.
In `@tests/usage-cost.test.ts`:
- Around line 769-812: Move the empty-provider cleanup for user cost overlays
out of the test body and into an afterEach hook scoped to the surrounding
describe block. Ensure the hook calls refreshUserCostOverlays with an empty
providers object so the blsc overlay is reset even when assertions in “registry
refresh replaces rows, bumps the version, and invalidates the memo” fail, and
update the bun:test imports to include afterEach.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d8697a9c-f003-4c14-ae6a-70292ae50aef
📒 Files selected for processing (20)
docs-site/src/content/docs/ja/reference/configuration/providers.mddocs-site/src/content/docs/ko/reference/configuration/providers.mddocs-site/src/content/docs/reference/configuration/providers.mddocs-site/src/content/docs/ru/reference/configuration/providers.mddocs-site/src/content/docs/zh-cn/reference/configuration/providers.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tsgui/src/pages/Logs.tsxsrc/config.tssrc/server/auth-cors.tssrc/server/management/shared.tssrc/types.tssrc/usage/cost.tssrc/usage/user-cost-overlays.tstests/provider-cost-overlay-config.test.tstests/usage-cost.test.ts
|
@coderabbitai resume — the dev merge (f620f7f) resolved the config.ts conflict (persistConfigUnlocked now returns boolean for generation bumping while still refreshing the cost-overlay registry). Please run the full review on the merged state. |
|
✅ Action performedReviews resumed. Full review finished. |
|
Heads-up for review: the hygiene gate flags |
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
src/server/auth-cors.ts (1)
525-539: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winPreserve the valid
__proto__model ID.Line 527 creates a normal object.
providerModelCostsConfigErroraccepts"__proto__"because it is a nonblank model ID. For a row parsed from JSON, Line 536 changesout's prototype instead of adding an own property.Object.keys(out)then omits the row, sosafeConfigDTOdoes not expose a valid configured price.Create
outwith a null prototype. Add a regression test that builds the key throughJSON.parse().Proposed fix
- const out: Record<string, ProviderCostOverlay> = {}; + const out: Record<string, ProviderCostOverlay> = Object.create(null);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/server/auth-cors.ts` around lines 525 - 539, Update sanitizeModelCosts to create out with a null prototype so assigning the valid "__proto__" model ID produces an own property and remains included in the returned configuration. Add a regression test that obtains this key via JSON.parse() and verifies the sanitized costs preserve the valid entry.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/provider-cost-overlay-config.test.ts`:
- Around line 26-30: Reset the user overlay registry in both affected test
sites: in tests/provider-cost-overlay-config.test.ts lines 26-30, call
refreshUserCostOverlays({ providers: {} } as OcxConfig) in the existing
afterEach before deleting OPENCODEX_HOME; in tests/usage-cost.test.ts lines
769-812, add a scoped afterEach that performs the same reset and remove the
test-body cleanup near line 807.
---
Duplicate comments:
In `@src/server/auth-cors.ts`:
- Around line 525-539: Update sanitizeModelCosts to create out with a null
prototype so assigning the valid "__proto__" model ID produces an own property
and remains included in the returned configuration. Add a regression test that
obtains this key via JSON.parse() and verifies the sanitized costs preserve the
valid entry.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: a262235e-fcc4-491e-a174-d1c80bdad0fb
📒 Files selected for processing (20)
docs-site/src/content/docs/ja/reference/configuration/providers.mddocs-site/src/content/docs/ko/reference/configuration/providers.mddocs-site/src/content/docs/reference/configuration/providers.mddocs-site/src/content/docs/ru/reference/configuration/providers.mddocs-site/src/content/docs/zh-cn/reference/configuration/providers.mdgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tsgui/src/pages/Logs.tsxsrc/config.tssrc/server/auth-cors.tssrc/server/management/shared.tssrc/types.tssrc/usage/cost.tssrc/usage/user-cost-overlays.tstests/provider-cost-overlay-config.test.tstests/usage-cost.test.ts
… isolation - sanitizeModelCosts builds the DTO map with a null prototype so a model id literally named __proto__ stays an own row instead of mutating the map's prototype and vanishing from Object.keys (CodeRabbit minor). - usage-cost and provider-cost-overlay-config tests reset the module-level overlay registry in afterEach so rows cannot leak across test files in a shared-process run (CodeRabbit stability). - Regression test: safeConfigDTO keeps a __proto__ model id as an own row.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5db9f883f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Codex review findings: - refreshUserCostOverlays now copies ONLY the four validated rate fields into the overlay row, so a hand-edited row carrying extra properties (e.g. a misplaced apiKey) can no longer leak through /api/logs display estimates (P1). - loadConfig/configDiagnosticsFromRaw run sanitizeModelCostsForLoad before schema validation: malformed display-price rows are dropped with a warning instead of failing the whole parse and falling back to defaults, which previously discarded otherwise valid providers. Strict rejection stays at the management/write boundary (P2). Regression tests: malformed row degradation, non-object modelCosts drop, and registry rows containing only the four rate fields.
There was a problem hiding this comment.
💡 Codex Review
Line 231 in 50cc147
Try exact provider names before collapsing user overlays
When a valid custom provider name happens to end with the Codex account-label suffix pattern, such as blsc-pabcdef, resolveMatchedPrice() has already collapsed it to blsc before this new user-overlay lookup runs, while refreshUserCostOverlays() stores the configured row under the exact provider name. In that scenario a valid providers.blsc-pabcdef.modelCosts.custom-model entry is never matched, so Logs/usage estimates stay unpriced despite the explicit override; try the exact provider against userOverlays before applying baseProviderLabel, or register both exact and collapsed keys.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
A custom provider whose name ends with the Codex account-log-label suffix pattern (e.g. blsc-pabcdef) had its provider collapsed by baseProviderLabel before the user-overlay lookup, so providers.<name>.modelCosts rows stored under the exact name never matched and estimates stayed unpriced. resolveMatchedPrice now checks the exact provider against the user overlay first, then collapses for the compiled catalogs.
|
Addressed in 0e37a89: |
|
Codex review is not enabled for this repo. Please contact the admins of this repo to enable Codex. |
…y changes - providerModelCostsConfigError now JSON-quotes and redacts secret-shaped model ids so a malformed write cannot echo a pasted key/secret back through the management API. - /api/usage summaries are cached against userCostOverlayVersion(): an overlay save invalidates the entry even when the usage log is unchanged. - refreshUserCostOverlays is a no-op when the extracted rows are byte-identical, so reloads of an unchanged config (server start, migrations, persist paths) no longer churn the version, invalidate the summary cache, or thash the cost memo. - tests: redaction + quoted-key assertions, cache-invalidation test, and an identical-refresh no-op regression test.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/api-usage.test.ts`:
- Around line 171-185: Update the test’s finally block to clear the module-level
cost overlay registry before calling server.stop(true), using the existing
overlay-reset mechanism. Keep this cleanup inside finally so it executes even
when assertions or server shutdown fail, and target the registry installed by
refreshUserCostOverlays.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: f7f99109-da20-4a5b-aaba-4eb52efe10ba
📒 Files selected for processing (7)
src/config.tssrc/server/management/logs-usage-routes.tssrc/server/management/usage-summary-cache.tssrc/usage/user-cost-overlays.tstests/api-usage.test.tstests/provider-cost-overlay-config.test.tstests/usage-cost.test.ts
…t finally block The test installs a module-level blsc overlay; reset it to empty before stopping the server so a later test (or an assertion/shutdown failure) cannot resolve user-configured prices unexpectedly.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66fc6a3d4f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
providerManagementConfigError echoed the caller-controlled provider name verbatim in the modelCosts error path even though the route has not yet validated/sanitized it. JSON-quote and redact the name (same rule as the retryOn429 branch) so a token-shaped provider name cannot serialize back through the management API. Regression test added.
|
Hi @lidge-jun — just a gentle follow-up whenever you have a moment. The PR is marked ready and everything on our side is green: all review threads (CodeRabbit + Codex connector) are resolved, and CodeRabbit, enforce-target, and label checks pass. The one remaining gate is the hygiene check's |
Closes #1009
Summary
Adds
providers.<name>.modelCosts— a per-model display price overlay (USD per1M tokens) so operators can price internal/custom providers whose ids do not
match the compiled catalogs, or whose actual costs vary from list prices.
Resolution order: user
modelCosts→ jawcode catalog → expected-price overlay →model-level vendor fallback. All-zero entries fall through. Display-time
estimation only; never billing.
Changes
src/usage/user-cost-overlays.ts(new): registry liftingproviders.<name>.modelCostsinto price-overlay rows; refreshed atloadConfigand every persist path; versioned so the estimator memo neverserves stale rows and edits apply without a restart.
src/usage/cost.ts: user overlay wins inresolveMatchedPrice(exactprovider/model match,
source: "user"); memo key now includes the overlayversion. Also replaces a stray NUL byte in the memo cache-key template
literal with a space separator.
src/config.ts:providerModelCostsConfigErrorvalidation (non-negativefinite 4-tuples) wired into the load-time schema refine.
src/server/auth-cors.ts: management API rejects malformedmodelCosts;safeConfigDTOexposes the field.src/server/management/shared.ts: newprovider_cost_overlayestimatereason.
gui/: reason label + i18n strings (en/de/ja/ko/ru/zh).UI change
Tests
via overlay, all-zero fall-through, combo fail-closed, registry refresh/memo
invalidation.
validation, DTO passthrough.
bun run typecheck, GUIeslint, and the touched suites pass (91 tests).Summary by CodeRabbit
New Features
Documentation
Localization
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I fixed all correct Codex and CodeRabbit findings.
My PR is ready for review.