Design: tool-def deferral coverage (detect deferral gaps, restore/tune native deferral, proxy for non-native agents)

[AVSRPA1KR]

Follow-up to the acting-layer epic (#602). This is the deferral piece called out in the epic's non-goals as a separate, evidence-gated track. Design issue first, then PR, same flow the epic used.

Scope update: the proxy fallback for non-native agents (originally part 3) has been split into its own design issue — request-path (Class C) code warrants a separate review, and it now also covers a provider-agnostic gateway (Anthropic + OpenAI), which is beyond this issue's analyzer/config scope. This issue is now scoped to detection (part 1) and native config restore/tune (part 2).

Motivation

Tool-def deferral is no longer a missing capability on the happy path: Claude Code ships tool search enabled by default — MCP tool defs are deferred and discovered on demand, and the API mechanism behind it (defer_loading + a tool search tool) is GA on the Claude API, with the prefix untouched and prompt caching preserved by design (per current docs at https://code.claude.com/docs/en/mcp and https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool).

The real gap is coverage:

1. Deferral silently turns off. Claude Code disables tool search on Vertex AI and whenever ANTHROPIC_BASE_URL points to a non-first-party host (docs: "most proxies don't forward tool_reference blocks"). Users behind any gateway/proxy, or with a stale ENABLE_TOOL_SEARCH=false, are paying full tool-def cost on every turn with no signal that anything changed.
2. Deferral config has tunable waste even when on. alwaysLoad: true servers that are rarely invoked, and threshold mode (auto:N) left at defaults, both leave measurable prefix cost on the table.
3. Non-native agents have no deferral at all. Cursor, Cline, Aider and similar load every def every turn — addressed in the separate proxy design issue, not here.

None of this is routing. No model changes, no per-turn classification, no rewriting of anything the user wrote.

Captured evidence (real usage, Opus 4.x)

Measured deferred vs non-deferred on the same workloads, in separate sessions each:

• 20-turn tool-heavy session: all defs loaded every turn $1.31 vs deferred $0.90 (~31% less, no model change).
• Two-turn "chat → one tool lookup" flow: $0.96 non-deferred vs $0.35 deferred (~64% less).
• Cache behavior matches the documented design: deferred defs are excluded from the prefix; a discovered tool comes back as a tool_reference block appended inline, so the cached head stays byte-stable.

Addressing the epic's evidence gate directly:

Gate	Answer
Pre-turn classification accuracy	Not applicable — deferral needs no classification. That concern belongs to routing, which stays out of scope here.
Cache-hit impact	Positive by documented design (prefix untouched); acceptance criteria below make it measured per user, not asserted.

Scope

1. Detection (analyzer-side, always safe)

A new optimize finding family, mcp-deferral-gaps:

• Deferral-off detection. Deterministic transcript signal: sessions carrying meaningful MCP tool-def overhead with zero ToolSearch tool invocations across the window. Config corroboration where readable: ENABLE_TOOL_SEARCH=false in settings/env, a non-first-party ANTHROPIC_BASE_URL, Vertex AI, or a Claude Code version predating the feature. Each cause gets its own message, because the fixes differ.
• alwaysLoad hygiene. Servers or tools pinned with alwaysLoad: true (server-level in .mcp.json, or "anthropic/alwaysLoad": true in a tool's _meta) whose observed call rate doesn't justify permanent prefix residency. Reuses the per-server schema-token overhead the mcp-low-coverage detector already computes. Threshold starts conservative (flag only clear cases, e.g. under 1 call per 5 sessions) and tunes on the detector's own data.
• Threshold tuning. Where auto mode is in use, report whether the observed def volume suggests a tighter auto:N.
• Output: per-server estimated tokens/session in the prefix, observed call rate, estimated savings, and the detected cause. Pure reporting; ships regardless of what follows.

2. Restore/tune native deferral (config-class, journaled)

Where the agent supports deferral natively, the fix is a config edit — exactly the class of action #603/#604 already handle. New plan kinds in src/act/plans.ts:

• defer-enable: remove a stale ENABLE_TOOL_SEARCH=false (settings.json env or shell config), or set ENABLE_TOOL_SEARCH=true explicitly for a user behind a tool_reference-capable proxy who wants deferral back. Never set true blind behind an unverified proxy — the docs are explicit that requests fail on proxies that don't support tool_reference blocks, so the plan preview must state this and the finding must distinguish "proxy verified capable" from "unknown proxy".
• defer-alwaysload: remove or add alwaysLoad: true on specific servers in .mcp.json, per the hygiene finding. Requires Claude Code v2.1.121+; the plan checks the installed version and marks itself not-appliable below it. Note alwaysLoad: true also blocks startup on that server's connection (5s cap) — the preview says so.
• defer-threshold: set ENABLE_TOOL_SEARCH=auto:N in settings.json env.

All through runAction(): backup, journal, undo, dry-run preview. ENABLE_TOOL_SEARCH is read at process start, so every defer-* preview states that the change takes effect on the next session. Per #605's rule, verify the exact settings surface against live docs at implementation time; the field names above were verified against current docs but the protocol evolves.

Measurement

act report (#606) integration: defer-* actions capture a baseline (tool-def tokens/session, cache-write tokens/session, cache-hit rate over trailing 14 days) and report realized deltas post-apply, with the same confidence markers. Deferral's whole pitch is cache economics, so cache-hit rate before/after is a first-class reported number — if a deferral change ever hurts hit rate in practice, the report should say so.

Non-goals

• No per-turn model routing, no request classification, no param rewriting. (Separate future track, gated on its own evidence.)
• No reimplementing deferral where it exists natively — native path always preferred. A proxy for agents without native deferral (or gateways that break it) is a separate design issue.
• Nothing on by default. Detection findings are informational; both fix paths require explicit commands.
• No cross-provider anything in this issue. (OpenAI ships its own tool search on GPT-5.4+ via the Responses API; that path — different wire protocol, different cache economics — is handled in the separate proxy/gateway design, not here.)

Acceptance

• optimize surfaces deferral gaps with per-server evidence, detected cause, and estimated savings, with zero behavior change for users who ignore it.
• A user whose deferral was silently off (stale env / disabling config) can see why, apply the fix where safe, and undo to a byte-identical config.
• defer-* actions are journaled, dry-run previewable, and act undo restores byte-identical files; act report shows measured tokens/session and cache-hit deltas after the window.

[AVSRPA1KR]

@iamtoruk Have opened #630 (detection) and #631 (plans) for #614 — scoped it to parts 1 & 2. Splitting the proxy into its own design, which i will raise soon, as its scope is mutually exclusive to parts 1 & 2. Would love your review on these.

[iamtoruk]

Parts 1 (deferral-gap detection, shipped in #631, optimize.ts) and 2 (config plans, act/plans.ts) are merged; the proxy fallback for non-native agents is now tracked separately in #768 per the scope update. Closing this design issue as its merged scope is complete.