Epic: Compaction Update

[riatzukiza]

Summary

Compaction currently loses important conversation context and instructions, producing regressions like memory loss after compaction, rules not being followed, and poor continuation in BUILD mode. This epic addresses boundary selection, rules preservation, summary quality, pruning, and observability so conversations remain coherent and safe after compaction.

Problem Statement

• Boundary detection uses first prior assistant summary as a hard cut (message-v2.ts:672-680), which can discard too much or wrong context and makes the model forget the current task.
• Rules and user constraints (e.g. no git push) are not reliably preserved in the conversational context after compaction.
• The current summary prompt is too light to act as a handoff; it lacks detailed structure for continuing work (files, errors/fixes, pending tasks, last action).
• Pruning is underutilized and protects the exact region where most tokens accrue (tool calls between last two user turns), leading to premature full compaction instead of selective deletion.
• Minimal observability: hard to see which messages/parts are included, and token usage breakdown for tool calls.

Related Issues

• compact causes llm to forget convo #2234 compact causes LLM to forget conversation (memory loss after compact)
• Session automatically compacted, destroying the entire working context #2945 Session automatically compacted, destroys working context
• Model in BUILD mode does not have enough context to continue after compaction #3031 BUILD mode lacks context after compaction (needs handoff)
• Soft compaction / AI global workspace metabolism #3032 Soft compaction / better metabolism (selective removal, keep recent context)
• Agent no follow rules after compact session #3099 Rules not followed after compact (AGENTS.md and constraints)
• TypeError: undefined is not an object (evaluating 'toSummarize.at(-1).info') #2559 Edge case empty slice during summarize (robustness)
• BUG: @ai-sdk/openai 2.0.50 breaks OpenAI OAuth plugin requiring store: false (missing providerOptions in compaction) #3118 ProviderOptions missing in compaction path (token/usage)
• I can not configure the limit for litellm #2787 includeUsage needed for openai-compatible (usage accounting)

Goals

• Preserve critical context for seamless continuation in BUILD mode.
• Keep user rules/constraints alive in the conversation post-compaction.
• Reduce token pressure via smarter pruning before resorting to full compaction.
• Improve transparency (what was kept/dropped and why).

Non‑Goals

• UI redesign of the session view.
• Provider-specific tuning beyond ensuring usage accounting and stable headers/options.

Plan / Tasks

• Boundary: replace filterCompacted with smarter selection that preserves the last N turns intact (configurable), includes the last assistant message, and then compacts earlier history only.
  • File: packages/opencode/src/session/message-v2.ts:672-680
  • Add defensive check that all included messages have info.sessionID === input.sessionID (belt-and-suspenders).
  • Emit debug logs: selected message IDs, last summary index, preserved turn count.
• Rules Preservation: ensure user constraints survive compaction.
  • Minimal: require the summary to explicitly include a "Rules & Constraints" section capturing active directives (AGENTS.md, permission-sensitive rules, user instructions).
  • Stretch: introduce a pinned RulesPart to persist constraints across summaries (design spike; may land later).
• Summary Prompt: upgrade to a detailed handoff format (primary intent, files touched, code snippets, errors/fixes, pending tasks, current work, rules, optional next step). Use a compact but structured template.
  • File: packages/opencode/src/session/compaction.ts (user prompt around L169-L178)
  • Ensure consistency with SystemPrompt.summarize and custom instructions.
• Pruning Improvements: make pruning more effective before full compaction.
  • Revisit PRUNE_PROTECT/PRUNE_MINIMUM and turns-protection strategy; optionally configurable.
  • Prefer deleting old tool outputs earlier; consider a soft-compaction approach that drops tool call outputs selectively.
  • Add /status or log output that shows tokens by category (user/assistant/tool/reasoning) to guide tuning.
• Provider Options & Usage
  • Ensure providerOptions (headers, includeUsage) flow in compaction path (see BUG: @ai-sdk/openai 2.0.50 breaks OpenAI OAuth plugin requiring store: false (missing providerOptions in compaction) #3118, I can not configure the limit for litellm #2787).
  • Verify accurate usage accounting during compaction and pruning.
• Tests
  • Unit: boundary selection (empty slice, multiple summaries, preserve last N turns).
  • Unit: rules preservation (rules section present in summary).
  • Unit: pruning selection logic.
  • Integration: BUILD-mode continuation across compaction retains current task context and rules.
• Telemetry/Logging
  • Log token inputs for compaction, before/after counts.
  • Log pruning decisions and compaction boundaries.

Definition of Done

• After compaction, the assistant reliably continues the task in BUILD mode without re-asking for context.
• User rules/constraints remain in force and visible in the conversation context.
• Selective pruning reduces token usage without harming recency.
• Added tests pass and prevent regressions in boundary selection and rules persistence.
• Logs clearly show messages kept/dropped and token breakdown.

Acceptance Tests

• 8+ hour session with dozens of compactions retains rules (e.g., no git push without permission) and continues coding flows.
• Last N turns are preserved verbatim; summary contains a clear handoff and rules.
• No cross-session contamination; compaction operates strictly within sessionID.