feat(anthropic): opt-in Claude OAuth account pool (#294) - #578
Conversation
Add experimental, default-off routing across stored Anthropic OAuth accounts: sticky session affinity, 429 cooldown failover, and new-session lowest 5h-usage pick. Includes GUI toggle with an explicit not-battle-tested warning, management API, docs, and focused regressions.
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Actionable comments posted: 8
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs-site/src/content/docs/guides/claude-code.md`:
- Around line 15-19: The Claude account pool documentation must describe OAuth
reliability behavior consistently in both
docs-site/src/content/docs/guides/claude-code.md lines 15-19 and
docs-site/src/content/docs/reference/configuration.md lines 108-123: document
that upstream 429 responses use Retry-After, quota-reset data, or default
backoff to establish account cooldown, clear affinities, and allow rotation;
state that affinity is process-local and does not survive restart; and explain
that 401/403 credential failures quarantine the account and clear its
affinities. Update both sites with the same constraints beside their existing
pool descriptions and configuration options.
In `@gui/src/components/provider-workspace/AnthropicAccountPoolSettings.tsx`:
- Around line 141-149: Update the threshold validation in the onBlur handler to
reject decimal input rather than truncating it through parseInt. Validate the
draft as a whole-number value before converting it, preserving the existing
invalid-reset/error behavior and only calling save(true, parsed) for valid
integer thresholds from 0 through 100.
In `@gui/src/i18n/de.ts`:
- Around line 671-683: Localize the account-pool strings instead of retaining
copied English text: update the entries in gui/src/i18n/de.ts lines 671-683 with
German translations, gui/src/i18n/ja.ts lines 1039-1051 with Japanese
translations, gui/src/i18n/ko.ts lines 688-700 with Korean translations,
gui/src/i18n/ru.ts lines 1081-1093 with Russian translations, and
gui/src/i18n/zh.ts lines 688-700 with Chinese translations, covering the
warning, configuration, threshold, error, and on/off labels while preserving the
existing interpolation placeholders.
In `@src/oauth/anthropic-routing.ts`:
- Around line 134-145: Update AnthropicAccountSelectionReason and
resolveAnthropicAccountForSession to distinguish having no logged-in Anthropic
accounts from having accounts that are all cooled or rate-limited. Return the
new reason consistently in the no-account branch and the pickLowestUsage
fallthrough paths, preserving existing reasons for all other selection outcomes
so downstream callers can choose 401 versus 429 correctly.
- Around line 41-42: Prevent unbounded growth of sessionAffinity by adding a
periodic eviction sweep for entries older than AFFINITY_IDLE_TTL_MS, rather than
relying only on same-key lookups or 429 handling. Reuse the existing affinity
timestamp/expiry data and ensure the sweep covers entries written by
resolveAnthropicAccountForSession, bindAnthropicSessionAffinity, and
rotateAnthropicAccountOn429. Start and clean up the sweep with the module’s
existing lifecycle, if applicable.
- Line 1: Distinguish exhausted cooled accounts from missing accounts during
Anthropic account selection. Update resolveAnthropicAccountForSession so both
pickLowestUsage failure fallthroughs return a distinct all-cooled reason, while
reason none remains limited to no configured accounts; then update
handleResponses to return a 429 rate-limit response for all-cooled and retain
401 for none.
In `@src/server/responses/core.ts`:
- Around line 1048-1051: Update resolveAnthropicAccountForSession to distinguish
“no eligible accounts logged in” from “all pooled accounts are currently
cooled/rate-limited” instead of returning reason "none" for both. In the
response handling around selection.accountId, map the cooled-account outcome to
HTTP 429 while preserving HTTP 401 for missing authentication. Ensure callers
use the resolver’s explicit reason field to select the correct error response.
- Around line 1033-1057: Extend the Anthropic pooled-account handling in the
OAuth response flow, including the upstream error handling around the existing
Anthropic pool rotation logic, so pooled accounts receiving upstream 401 or 403
are quarantined or rotated rather than returned unchanged. Mark the selected
account as needing reauthorization or apply the pool’s existing cooldown
mechanism, clear its session affinity as appropriate, and preserve the existing
429 rotation behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 68b864db-041f-4a6e-a830-ad407637cfbd
📒 Files selected for processing (18)
docs-site/src/content/docs/guides/claude-code.mddocs-site/src/content/docs/reference/configuration.mdgui/src/components/provider-workspace/AnthropicAccountPoolSettings.tsxgui/src/components/provider-workspace/ProviderAuthPanel.tsxgui/src/i18n/de.tsgui/src/i18n/en.tsgui/src/i18n/ja.tsgui/src/i18n/ko.tsgui/src/i18n/ru.tsgui/src/i18n/zh.tssrc/oauth/anthropic-routing.tssrc/oauth/health.tssrc/providers/quota.tssrc/server/management/oauth-account-routes.tssrc/server/responses/core.tssrc/types.tssrc/usage/log.tstests/anthropic-account-pool.test.ts
| const isOAuth401ReplayProvider = (route.providerName === "xai" || route.providerName === "github-copilot" || route.providerName === "kiro") | ||
| && route.provider.authMode === "oauth"; | ||
| let sentOAuthSnapshot: OAuthAccessSnapshot | undefined; | ||
| let anthropicPoolAccountId: string | null = null; | ||
| const anthropicSessionKey = route.providerName === "anthropic" && route.provider.authMode === "oauth" | ||
| ? anthropicSessionKeyFromParts({ | ||
| sessionIdHeader: sessionIdHeaderFromRequest(req.headers), | ||
| threadIdHeader: req.headers.get("thread-id"), | ||
| promptCacheKey: typeof parsed.options.promptCacheKey === "string" ? parsed.options.promptCacheKey : null, | ||
| clientThreadId: typeof parsed._clientThreadId === "string" ? parsed._clientThreadId : null, | ||
| }) | ||
| : null; | ||
| if (route.provider.authMode === "oauth") { | ||
| try { | ||
| const resolved = await getValidAccessTokenSnapshot(route.providerName); | ||
| if (isOAuth401ReplayProvider) sentOAuthSnapshot = resolved; | ||
| route.provider = { ...route.provider, apiKey: resolved.accessToken }; | ||
| if (route.providerName === "kiro") { | ||
| // `{}` is intentional: this is an account-scoped request with no stored routing metadata. | ||
| // Only genuinely accountless adapter calls leave the context undefined and use local/env fallback. | ||
| parsed._kiroAuthContext = { ...(resolved.kiro ?? {}) }; | ||
| } | ||
| // Antigravity (cloud-code-assist) needs the discovered Cloud Code Assist project id in the | ||
| // CCA envelope; the server injects only the bare token, so pull project from the credential. | ||
| if (route.provider.googleMode === "cloud-code-assist" && !route.provider.project) { | ||
| const projectId = getOAuthCredentialProjectId(route.providerName); | ||
| if (projectId) route.provider = { ...route.provider, project: projectId }; | ||
| if (route.providerName === "anthropic" && isAnthropicAccountPoolEnabled(config)) { | ||
| const selection = resolveAnthropicAccountForSession(anthropicSessionKey, config); | ||
| if (!selection.accountId) { | ||
| return formatErrorResponse(401, "authentication_error", "No eligible Anthropic OAuth account available"); | ||
| } | ||
| const accessToken = await getValidAccessTokenForAccount("anthropic", selection.accountId); | ||
| anthropicPoolAccountId = selection.accountId; | ||
| bindAnthropicSessionAffinity(anthropicSessionKey, selection.accountId); | ||
| route.provider = { ...route.provider, apiKey: accessToken }; | ||
| logCtx.provider = formatAnthropicProviderForLog("anthropic", selection.accountId, config); | ||
| } else { |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check where/if needsReauth gets set for Anthropic OAuth accounts, and whether an
# upstream 401/403 from the Messages call itself (not just a refresh failure) reaches it.
rg -n "needsReauth" src/oauth --type=ts -C3
rg -n "resolveAccessSnapshotForAccount|getValidAccessTokenForAccount" src/oauth/index.ts -C6Repository: lidge-jun/opencodex
Length of output: 17717
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the request-path logic around the cited lines and the OAuth replay/reauth hooks.
sed -n '1010,1085p' src/server/responses/core.ts
printf '\n---\n'
sed -n '220,300p' src/oauth/index.ts
printf '\n---\n'
sed -n '330,390p' src/oauth/index.ts
printf '\n---\n'
sed -n '250,290p' src/oauth/github-copilot.ts
printf '\n---\n'
sed -n '1,220p' src/oauth/anthropic-routing.tsRepository: lidge-jun/opencodex
Length of output: 19703
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find all upstream-401/403 replay or quarantine handling, and whether Anthropic
# is mentioned anywhere outside the request path.
rg -n "forceRefreshOAuthAccessSnapshot|401|403|unauthorized|forbidden|reauth|needsReauth|Anthropic" src -g '*.ts' -C2Repository: lidge-jun/opencodex
Length of output: 50376
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the downstream error handling around the request path, especially any 401/403
# replay or Anthropic-specific quarantine logic.
sed -n '1085,1225p' src/server/responses/core.ts
printf '\n---\n'
rg -n "isOAuth401ReplayProvider|sentOAuthSnapshot|anthropicPoolAccountId|needsReauth|401|403|forceRefreshOAuthAccessSnapshot|replay" src/server/responses/core.ts src/oauth/index.ts src/oauth/anthropic-routing.ts -C3Repository: lidge-jun/opencodex
Length of output: 25444
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the later response-handling block in core.ts and any direct 401/403 branches there.
sed -n '1225,1455p' src/server/responses/core.ts
printf '\n---\n'
rg -n "response\.status === 401|response\.status === 403|markAccountNeedsReauth|markAnthropicAccountCooldown|forceRefreshOAuthAccessSnapshot|isOAuth401ReplayProvider" src/server/responses/core.ts -C3Repository: lidge-jun/opencodex
Length of output: 12618
Anthropic pooled 401/403 still needs a quarantine path src/server/responses/core.ts:1033-1057,1933-2011
anthropic is excluded from isOAuth401ReplayProvider, so a pooled Anthropic account that comes back 401/403 is just returned to the client. The existing Anthropic pool logic only rotates on 429, and needsReauth is only set on refresh-time failures in src/oauth/index.ts, so a revoked/invalid pooled token can stay affined and keep serving the same auth error on later requests.
Add an Anthropic-specific 401/403 quarantine/rotation path here, or mark the selected account needsReauth/cool it when the upstream auth error comes back.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/server/responses/core.ts` around lines 1033 - 1057, Extend the Anthropic
pooled-account handling in the OAuth response flow, including the upstream error
handling around the existing Anthropic pool rotation logic, so pooled accounts
receiving upstream 401 or 403 are quarantined or rotated rather than returned
unchanged. Mark the selected account as needing reauthorization or apply the
pool’s existing cooldown mechanism, clear its session affinity as appropriate,
and preserve the existing 429 rotation behavior.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c0aa964594
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!selection.accountId) { | ||
| return formatErrorResponse(401, "authentication_error", "No eligible Anthropic OAuth account available"); | ||
| } | ||
| const accessToken = await getValidAccessTokenForAccount("anthropic", selection.accountId); |
There was a problem hiding this comment.
When the pool selects an expired, non-active Anthropic account whose credential has source: "local-cli", this call reaches refreshAnthropicAccountWithLock, which can adopt the current global Claude CLI credential without verifying that it belongs to the selected slot and then persist it there. The existing quota path explicitly refuses this operation for background local-CLI rows in src/providers/quota.ts:345-363 because it can persist a mismatched identity. A pooled request can therefore corrupt auth.json and run the session as the wrong account; fail closed for these background slots or make CLI reconciliation identity-safe before routing.
AGENTS.md reference: AGENTS.md:L83-L89
Useful? React with 👍 / 👎.
| ? anthropicSessionKeyFromParts({ | ||
| sessionIdHeader: sessionIdHeaderFromRequest(req.headers), | ||
| threadIdHeader: req.headers.get("thread-id"), | ||
| promptCacheKey: typeof parsed.options.promptCacheKey === "string" ? parsed.options.promptCacheKey : null, |
There was a problem hiding this comment.
For Claude Desktop requests without metadata.user_id, claude-messages.ts generates prompt_cache_key from the model, system prompt, and tools; src/claude/inbound.ts:454-474 explicitly defines this as a shared cache cohort rather than a per-session identifier. Treating it as an affinity key here pins unrelated Desktop conversations with the same configuration to one account, preventing independent new-session quota selection and allowing one conversation's 429 failover to rebind the entire cohort. Preserve the key provenance through the internal replay and omit system-derived keys from pool affinity.
Useful? React with 👍 / 👎.
| if (threshold > 0) { | ||
| const activeScore = activeOk ? usageScore(set.activeAccountId) : UNKNOWN_USAGE_SCORE; | ||
| if (activeOk && activeScore < threshold) { |
There was a problem hiding this comment.
On a cold start, or before the dashboard has probed per-account quota, usageScore returns 100 for the active account. This branch consequently treats unknown usage as having crossed the default 80% threshold and pickLowestUsage chooses the first eligible account when all scores are tied, silently overriding the user's active selection without evidence that its quota is high. Add an explicit unknown-usage guard, as the Codex pool does, so quota-based switching waits for a real cached 5-hour value.
Useful? React with 👍 / 👎.
| const nextEnabled = json.enabled === true; | ||
| const nextThreshold = typeof json.autoSwitchThreshold === "number" ? json.autoSwitchThreshold : 80; | ||
| setState({ enabled: nextEnabled, threshold: nextThreshold }); | ||
| setDraft(String(nextThreshold > 0 ? nextThreshold : 80)); |
There was a problem hiding this comment.
When the API returns autoSwitchThreshold: 0, this converts the draft to "80" even though zero is the documented setting that disables quota-based picking. With the pool enabled, merely focusing and blurring the displayed field causes the blur handler to save 80, and toggling the pool also substitutes 80 at line 103. Keep 0 in the draft and pass the stored threshold unchanged when toggling.
Useful? React with 👍 / 👎.
| const key = sessionKey?.trim(); | ||
| if (!key) return; | ||
| sessionAffinity.set(key, { accountId, lastUsedAt: now }); |
There was a problem hiding this comment.
In a long-running proxy receiving many distinct session IDs, every new ID inserts an entry here, but expired entries are removed only if that exact ID is requested again. One-shot conversations therefore remain forever and let sessionAffinity grow without limit. Prune expired entries and enforce an LRU/size cap on insertion, analogous to the 2,048-entry bound in src/codex/routing.ts.
Useful? React with 👍 / 👎.
| while ( | ||
| upstreamResponse.status === 429 | ||
| && anthropicPoolAccountId | ||
| && isAnthropicAccountPoolEnabled(config) | ||
| ) { | ||
| const nextAccountId = rotateAnthropicAccountOn429( |
There was a problem hiding this comment.
If every account returns 429 and the advertised cooldown expires while the next upstream attempt is in flight—for example, Retry-After: 1 with responses taking over one second—the previously failed account becomes eligible again and this loop rotates back to it. Because there is no per-request attempted-account set or retry limit, the request can cycle indefinitely and keep issuing upstream calls. Track accounts attempted by this request and stop after each eligible account has been tried once.
Useful? React with 👍 / 👎.
| if (!selection.accountId) { | ||
| return formatErrorResponse(401, "authentication_error", "No eligible Anthropic OAuth account available"); |
There was a problem hiding this comment.
After a request receives 429 from all accounts, each account is placed in cooldown and the original request correctly surfaces 429. An immediate client retry then gets selection.accountId === null and this branch returns a fatal 401 authentication error even though the credentials are valid and only temporarily cooling; Claude Code may consequently stop retrying or prompt for login. Distinguish the all-cooling state from missing authentication and return 429 with a suitable Retry-After, or admit a controlled probe.
Useful? React with 👍 / 👎.
| type="checkbox" | ||
| checked={enabled} | ||
| disabled={loading || saving || loadError || accountCount < 2} |
There was a problem hiding this comment.
If the user enables the pool with two accounts and later removes one, accountCount < 2 disables the checkbox even though it remains checked. The dashboard then offers no way to turn the experimental feature off without calling the API or editing configuration manually. Apply the account-count restriction only when enabling, and always allow an enabled pool to be disabled.
Useful? React with 👍 / 👎.
| // Opt-in Anthropic OAuth account pool (#294): cool the failed account and retry once | ||
| // with another eligible OAuth account. Disabled by default. | ||
| while ( | ||
| upstreamResponse.status === 429 | ||
| && anthropicPoolAccountId | ||
| && isAnthropicAccountPoolEnabled(config) |
There was a problem hiding this comment.
The new Anthropic OAuth recovery runs only around the initial upstream response. When the Anthropic terminal guard opens its hidden continuation after an end_turn without the promised tool call, the separate loop at src/server/responses/core.ts:2132-2148 handles API-key 429 rotation but has no OAuth-account equivalent; a continuation 429 is therefore emitted as an in-stream error even when another eligible OAuth account exists. Reuse the bounded account failover path for that continuation fetch as well.
Useful? React with 👍 / 👎.
| return null; | ||
| } | ||
|
|
||
| void setActiveAccount(PROVIDER, next).catch(() => { /* best-effort dashboard sync */ }); |
There was a problem hiding this comment.
This changes the global active account as soon as a candidate is selected, before the caller resolves its token or receives any successful retry. If that candidate has an expired or revoked credential, or itself returns 429, the failed/cooling account remains the persisted active selection; disabling the pool or using a non-pooled path then immediately targets that bad account. Delay the dashboard promotion until token resolution and the retry have succeeded, or roll it back on failure.
Useful? React with 👍 / 👎.
Return 429 when all accounts are cooling, bound per-request failover, skip unsafe local-cli refresh, and keep affinity/GUI/docs aligned with the reliability contract.
Summary
GET/PUT /api/oauth/accounts/pool, clear-cooldown).Design notes
Test plan
bun test tests/anthropic-account-pool.test.tsbun run typecheckbun run lint:gui(after gui install)Out of scope / follow-ups
Closes nothing yet (roadmap item; leave issue open until maintainers accept).
Summary by CodeRabbit