Skip to content

feat: auto-compact and retry on context overflow errors #85

Description

@ezynda3

Feature Description

Add a reactive compaction path: when a provider call fails with a context-length/overflow error, automatically run compaction and replay the turn once, instead of surfacing a hard failure to the user.

Kit currently only compacts proactivelyrunTurn checks ShouldCompact() before building context (pkg/kit/kit.go). ErrContextOverflow exists in pkg/kit/errors.go but nothing recovers from it: if the token estimate undercounts (the char/4 heuristic inevitably will) and the provider rejects the request, the turn simply fails.

opencode has both paths (compactIfNeeded before every provider turn, and compactAfterOverflow in its LLM runner after a provider context-overflow error). The reactive path is the safety net that makes an imprecise estimator acceptable in practice.

Motivation / Use Case

  • Heuristic token counting will always drift from real tokenizer counts, especially with tool-heavy traffic, non-English text, and images. Without recovery, drift = hard turn failure.
  • A single huge mid-turn tool result can overflow context even when the turn started well under the limit — proactive checks at turn start cannot catch this.
  • Related: feat: improve compaction token estimation and adaptive budgets #83 improves the estimator, but estimation can never be exact; this issue makes overflow non-fatal.

Proposed Implementation

In the agent loop error handling (pkg/kit/kit.go):

  1. Detect context-overflow errors from the provider (map provider-specific errors to ErrContextOverflow).
  2. Run compactInternal(ctx, opts, "", true) — reuse existing machinery.
  3. Rebuild context and replay the failed turn once (single retry guard to avoid loops).
  4. On replay, strip media/image attachments from the replayed user message (replace with text placeholders) to avoid immediate re-overflow — opencode does this.
  5. If compaction still cannot fit the context, fail with a clear ErrContextOverflow message (e.g. "conversation too large to compact — exceeds model context limit") rather than a raw provider error.

Optional follow-ons (small, could be included here or split):

  • Mid-turn ShouldCompact() check between agentic steps, not just at turn start.
  • After an auto-compaction mid-task, inject a synthetic continuation nudge ("Continue if you have next steps, or stop and ask for clarification") the way opencode does, so the agent resumes cleanly.

Checklist

  • I've searched existing issues and this hasn't been requested yet
  • This feature aligns with Kit's design philosophy (TUI-first, extension-based)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions