You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a reactive compaction path: when a provider call fails with a context-length/overflow error, automatically run compaction and replay the turn once, instead of surfacing a hard failure to the user.
Kit currently only compacts proactively — runTurn checks ShouldCompact() before building context (pkg/kit/kit.go). ErrContextOverflow exists in pkg/kit/errors.go but nothing recovers from it: if the token estimate undercounts (the char/4 heuristic inevitably will) and the provider rejects the request, the turn simply fails.
opencode has both paths (compactIfNeeded before every provider turn, and compactAfterOverflow in its LLM runner after a provider context-overflow error). The reactive path is the safety net that makes an imprecise estimator acceptable in practice.
Motivation / Use Case
Heuristic token counting will always drift from real tokenizer counts, especially with tool-heavy traffic, non-English text, and images. Without recovery, drift = hard turn failure.
A single huge mid-turn tool result can overflow context even when the turn started well under the limit — proactive checks at turn start cannot catch this.
In the agent loop error handling (pkg/kit/kit.go):
Detect context-overflow errors from the provider (map provider-specific errors to ErrContextOverflow).
Run compactInternal(ctx, opts, "", true) — reuse existing machinery.
Rebuild context and replay the failed turn once (single retry guard to avoid loops).
On replay, strip media/image attachments from the replayed user message (replace with text placeholders) to avoid immediate re-overflow — opencode does this.
If compaction still cannot fit the context, fail with a clear ErrContextOverflow message (e.g. "conversation too large to compact — exceeds model context limit") rather than a raw provider error.
Optional follow-ons (small, could be included here or split):
Mid-turn ShouldCompact() check between agentic steps, not just at turn start.
After an auto-compaction mid-task, inject a synthetic continuation nudge ("Continue if you have next steps, or stop and ask for clarification") the way opencode does, so the agent resumes cleanly.
Checklist
I've searched existing issues and this hasn't been requested yet
This feature aligns with Kit's design philosophy (TUI-first, extension-based)
Feature Description
Add a reactive compaction path: when a provider call fails with a context-length/overflow error, automatically run compaction and replay the turn once, instead of surfacing a hard failure to the user.
Kit currently only compacts proactively —
runTurnchecksShouldCompact()before building context (pkg/kit/kit.go).ErrContextOverflowexists inpkg/kit/errors.gobut nothing recovers from it: if the token estimate undercounts (the char/4 heuristic inevitably will) and the provider rejects the request, the turn simply fails.opencode has both paths (
compactIfNeededbefore every provider turn, andcompactAfterOverflowin its LLM runner after a provider context-overflow error). The reactive path is the safety net that makes an imprecise estimator acceptable in practice.Motivation / Use Case
Proposed Implementation
In the agent loop error handling (
pkg/kit/kit.go):ErrContextOverflow).compactInternal(ctx, opts, "", true)— reuse existing machinery.ErrContextOverflowmessage (e.g. "conversation too large to compact — exceeds model context limit") rather than a raw provider error.Optional follow-ons (small, could be included here or split):
ShouldCompact()check between agentic steps, not just at turn start.Checklist