feat: implement context-window pruning middleware for semantic workspaces - #295
feat: implement context-window pruning middleware for semantic workspaces#295muntasir-396 wants to merge 3 commits into
Conversation
WalkthroughThis change adds ChangesMeeting Context Pruning
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant UserMessage
participant AIService
participant ContextPruningService
participant LLM
UserMessage->>AIService: meeting-related query
AIService->>ContextPruningService: prune summaries with maxTokens 1500
ContextPruningService-->>AIService: optimized summaries
AIService->>LLM: send formatted meetingContext
Possibly related issues
Possibly related PRs
Suggested labels: Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/services/ai_service.dart`:
- Around line 75-90: Move the meeting retrieval, ContextPruningService.prune
call, and MeetingFormatter.formatMeetingSummaries invocation from the pre-try
section into the existing try block in generateChatResponse. Ensure exceptions
from retrieval, pruning, formatting, or runtime casts are caught by the existing
error handler and produce the standard error result, while preserving the
current meetingContext behavior on successful non-empty results.
In `@lib/services/context_pruning_service.dart`:
- Around line 12-19: Update _calculateKeywordOverlap to normalize each query and
text word by removing punctuation and excluding empty tokens before constructing
queryWords and textWords. Preserve the existing intersection-based overlap
calculation and empty-set handling after normalization.
- Around line 87-94: Update the removed-chunk accounting in the context-pruning
loop so each skipped chunk is counted exactly once: either retain the per-chunk
increment in the loop and remove the aggregate addition after it, or eliminate
the increment and rely on the aggregate calculation. Keep chunksRemoved equal to
the actual number of removed chunks.
- Around line 5-8: Update ContextPruningService.prune() to budget the fully
rendered meeting context rather than raw summary text: include AIService’s
prepended content and MeetingFormatter.formatMeetingSummaries() additions such
as titles, dates, separators, and truncation markers when evaluating each
candidate. Recompute token estimates from the final rendered string, or reserve
equivalent overhead before accepting the last chunk, while preserving the
maxTokens limit.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 419c651f-716e-4f22-9bdb-e87bd4ec22c0
📒 Files selected for processing (2)
lib/services/ai_service.dartlib/services/context_pruning_service.dart
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
lib/services/context_pruning_service.dart (2)
66-106: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUse the same scope for original and final token metrics.
originalTokensexcludes the reserved header, whilecurrentTokensalways includes it—even when no meeting context is returned. A no-pruning run can report a negative reduction. Compute both values from equivalent rendered contexts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/services/context_pruning_service.dart` around lines 66 - 106, Update the token accounting around the pruning method’s originalTokens and currentTokens metrics to use equivalent rendered contexts, including the reserved “Relevant meeting information” header only when that context is returned. Ensure no-pruning results cannot report a negative reduction, while preserving the existing chunk selection and truncation behavior.
72-94: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy liftBudget the fully combined candidate after every mutation.
Each chunk is measured as
formatMeetingSummaries([chunk]), but the final formatter adds inter-chunk separators. The partial path also adds formatter metadata and the truncation marker without remeasuring, while Line 93 assumes it exactly fills the remaining budget. The rendered meeting context can still exceedmaxTokens.Build and measure
header + formatMeetingSummaries(candidateChunks)before accepting a chunk; shrink partial text until that rendered candidate fits.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/services/context_pruning_service.dart` around lines 72 - 94, The chunk selection logic in the pruning method must budget the fully rendered candidate, not individual chunks. Before accepting or mutating a chunk, build the candidate combined context using the formatter header and formatMeetingSummaries(candidateChunks), measure its rendered token count, and only accept it when it fits maxTokens; for partial chunks, iteratively shrink the summary text (including metadata and truncation marker) until the complete rendered candidate fits, rather than assuming remainingTokens can be added directly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/services/meeting_formatter_test.dart`:
- Around line 124-136: Update MeetingFormatter.formatMeetingSummary to validate
action_items entries before treating them as maps, skipping null or otherwise
malformed items while continuing to format valid entries. Replace the test’s
throwsA expectation with an assertion for the formatter’s safe fallback output,
while preserving normal behavior for valid action items.
---
Outside diff comments:
In `@lib/services/context_pruning_service.dart`:
- Around line 66-106: Update the token accounting around the pruning method’s
originalTokens and currentTokens metrics to use equivalent rendered contexts,
including the reserved “Relevant meeting information” header only when that
context is returned. Ensure no-pruning results cannot report a negative
reduction, while preserving the existing chunk selection and truncation
behavior.
- Around line 72-94: The chunk selection logic in the pruning method must budget
the fully rendered candidate, not individual chunks. Before accepting or
mutating a chunk, build the candidate combined context using the formatter
header and formatMeetingSummaries(candidateChunks), measure its rendered token
count, and only accept it when it fits maxTokens; for partial chunks,
iteratively shrink the summary text (including metadata and truncation marker)
until the complete rendered candidate fits, rather than assuming remainingTokens
can be added directly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6534048a-0192-4d45-8295-082367e16f5c
📒 Files selected for processing (3)
lib/services/ai_service.dartlib/services/context_pruning_service.darttest/services/meeting_formatter_test.dart
| test('throws Error when action items list contains null (Malformed input)', () { | ||
| final Map<String, dynamic> summary = <String, dynamic>{ | ||
| 'action_items': <dynamic>[null], | ||
| }; | ||
|
|
||
| expect( | ||
| () => MeetingFormatter.formatMeetingSummary( | ||
| title: 'Malformed Sync', | ||
| date: '2026-07-25 at 10:00', | ||
| summary: summary, | ||
| ), | ||
| throwsA(isA<Error>()), // Catches NoSuchMethodError or TypeError depending on Dart version | ||
| ); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Do not codify a crash for malformed action items.
This accepts a runtime failure when externally supplied summary data contains a null/non-map item. MeetingFormatter indexes every item as a map, so one malformed entry aborts formatting the entire meeting context. Validate or skip invalid entries in the formatter, then assert a safe fallback here instead.
As per path instructions, **/*.dart requires proper input validation and error handling.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/services/meeting_formatter_test.dart` around lines 124 - 136, Update
MeetingFormatter.formatMeetingSummary to validate action_items entries before
treating them as maps, skipping null or otherwise malformed items while
continuing to format valid entries. Replace the test’s throwsA expectation with
an assertion for the formatter’s safe fallback output, while preserving normal
behavior for valid action items.
Source: Path instructions
Closes #278
Description
This PR introduces a context-window pruning middleware to optimize meeting context before sending it to Gemini.
Changes
ContextPruningServiceAIServicebefore prompt constructionBenefits
Summary by CodeRabbit