feat: implement 5 claude-mem gap features (privacy, reaper, bun-path, relevance, progressive disclosure)

[clopca]

Summary

Implements 5 features identified as gaps from the claude-mem comparison:

Features

• Privacy Utility — Shared stripPrivateBlocks + redactSensitive utility used by both tool-capture and chat-capture hooks. Chat messages now get private blocks stripped and sensitive content redacted before storage.
• Orphan Reaper — Runs at plugin startup before daemon start, cleans up stale PID files from crashed sessions.
• Bun PATH Resolution — Scans Bun.which() → common install paths (~/.bun/bin/bun, /usr/local/bin/bun, /opt/homebrew/bin/bun) → fallback. Cached for process lifetime.
• Relevance Scoring — Weighted formula (recency 40%, type importance 30%, session affinity 20%, token efficiency 10%) that sorts observations by relevance when building context.
• Progressive Disclosure — Tool descriptions now explicitly guide the AI agent through the 3-layer search→timeline→recall workflow.

Also includes

• Dashboard stats API field name alignment fix
• Package version bump to 0.5.3

Verification

• 509 tests pass (up from 445, +64 new tests)
• Typecheck clean
• Build succeeds

Summary by CodeRabbit

• New Features

  • Multi-layer memory access with relevance-based scoring for improved retrieval.
  • Daemon orphan reaper for safer background process cleanup.
  • Privacy utilities for robust sensitive-data redaction.

• Bug Fixes

  • Improved Bun executable resolution and caching for more reliable startup.

• Documentation

  • Expanded tool descriptions clarifying the multi-layer memory workflow.

• Chores

  • Removed per-date recentActivity from dashboard stats.
  • Renamed stats fields: tokensSaved → totalTokensSaved, avgObservationSize → averageObservationSize.
  • Package version bump.

• Tests

  • Added comprehensive tests for relevance scoring, reaper, Bun path resolution, and privacy utilities.

[coderabbitai]

📝 Walkthrough

Walkthrough

Removes per-date recentActivity from dashboard stats; adds relevance scoring and sorting for progressive context; introduces Bun path resolution with caching; implements orphan daemon reaper; refactors privacy redaction into utilities; updates HTTP server dashboard fields and allows injected dashboardDir; adds comprehensive tests and bumps package version.

Changes

Cohort / File(s)	Summary
Memory relevance & context src/context/relevance.ts, src/context/progressive.ts, src/context/builder.ts	Adds relevance scoring (recency, type importance, session affinity, token efficiency), exportable scoring APIs, optional scoringContext in progressive context builder (sorts observations by relevance when provided), and updates context text to describe a 3-layer memory model.
Tools — descriptions src/tools/recall.ts, src/tools/timeline.ts	Updates mem-recall and mem-timeline descriptive strings to reference the 3-layer memory workflow; no logic changes.
Daemon & Bun runtime src/utils/bun-path.ts, src/daemon/manager.ts	Adds Bun executable resolution with candidate paths, cached resolver, and test helper; daemon startup now uses cached resolved Bun path.
Daemon reaper src/daemon/reaper.ts	New module exposing ReapResult and reapOrphanDaemons(dbPath) to detect and remove stale/corrupt PID files, track reaped count and errors, and log actions.
Privacy utilities & hook refactor src/utils/privacy.ts, src/hooks/tool-capture.ts	Adds stripPrivateBlocks and redactSensitive utilities; tool-capture delegates redaction/stripping to these helpers and removes inline redaction loop.
HTTP server & dashboard types src/servers/http-server.ts, dashboard/src/types.ts	HTTP server: adds optional dashboardDir in DashboardDeps, renames stats fields (tokensSaved→totalTokensSaved, avgObservationSize→averageObservationSize), and uses injected dashboardDir fallback; dashboard types remove recentActivity from StatsResponse.
Package & deps package.json	Version bumped (0.4.2 → 0.5.3) and open-mem: ^0.5.0 added; minor formatting changes.
Tests added/updated tests/context/relevance.test.ts, tests/daemon/reaper.test.ts, tests/utils/bun-path.test.ts, tests/utils/privacy.test.ts, tests/servers/http-server.test.ts	Adds comprehensive tests for relevance scoring, progressive context behavior, daemon reaper, Bun path resolution (and cache), privacy utilities; updates HTTP server tests to new stat field names.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~55 minutes

Possibly related PRs

• feat: v0.5 — Hybrid search scaling, background worker, and web dashboard #2: Directly related change removing recentActivity from StatsResponse in dashboard types and aligning server payload fields.

Poem

🐰 I nibble through mems, three layers in sight,
Scoring the old, bringing order to light.
I find bun paths, sweep orphaned pids clean,
Strip secret burrows where no eyes have been.
Hop, test, and ship — a small rabbit's delight.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.26% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title accurately summarizes the main changeset by identifying all five key features being implemented: privacy utilities, orphan reaper, bun-path resolution, relevance scoring, and progressive disclosure.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/claude-mem-gaps

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 7

🤖 Fix all issues with AI agents

In `@src/context/relevance.ts`:
- Around line 15-22: Remove the unused recentFileContext property from the
ScoringContext interface: edit the ScoringContext declaration to drop
recentFileContext?: string[] (leaving currentSessionId?: string and now: Date),
update any type references or imports that mentioned ScoringContext if they
assumed that field, run/type-check tests to ensure no usages remain, and remove
any related comments/docs or tests referencing recentFileContext so the codebase
no longer declares an unused scoring field.

In `@src/daemon/reaper.ts`:
- Around line 50-52: The reaper currently calls removePid(pidPath) which
swallows unlinkSync errors, so result.reaped is incremented and a success log is
emitted even if deletion failed; change the logic in reaper.ts to either call
fs.unlinkSync(pidPath) directly inside the existing try/catch so any exception
is caught and pushed into result.errors before deciding to increment
result.reaped and log, or—if you keep removePid—immediately verify removal with
fs.existsSync(pidPath) and only increment result.reaped and log when the file no
longer exists; make sure to reference removePid, pidPath, result.reaped and
update the catch to record the error (e.g., push to result.errors) rather than
silently ignoring failures.

In `@src/servers/http-server.ts`:
- Around line 267-268: The dashboardDir assignment uses new URL(...).pathname
which can produce URL-encoded segments; change the logic that computes
dashboardDir (the dashboardDir variable initialization and the fallback new URL
call) to convert the file URL to a proper filesystem path using a URL-to-path
helper (e.g., import and use fileURLToPath from 'node:url' or Bun.fileURLToPath)
so that injectedDashboardDir remains unchanged but the fallback uses
fileURLToPath(new URL(..., import.meta.url)); ensure references to
injectedDashboardDir and dashboardDir remain the same.

In `@src/utils/privacy.ts`:
- Around line 20-25: The loop compiles user/operator-supplied patterns into
RegExp directly (patterns and new RegExp(...) in src/utils/privacy.ts), which
risks ReDoS; validate and sanitize each pattern before compiling: enforce a max
pattern length (e.g., <= 200), reject patterns that contain nested/adjacent
quantifiers or common catastrophic constructs (heuristic check for sequences
like (.+)+, (a+)+, [^)]*+, backreferences like \1, or nested groups with +/*/?)
and skip or log them, and only then call new RegExp(pattern, "g") and run
result.replace; additionally mark these patterns come from
config.sensitivePatterns so validation occurs when reading that config to
fail-fast.

In `@tests/context/relevance.test.ts`:
- Around line 22-33: The helper makeEntry currently returns an ObservationIndex
but omits the required importance property; update makeEntry to include a
default importance:number (e.g., importance: 0 or another sensible default) in
the returned object and ensure any test call sites that rely on overrides either
pass importance or rely on the new default; locate the function makeEntry in
tests/context/relevance.test.ts and add the importance field to the returned
object so it matches the ObservationIndex type.

In `@tests/utils/bun-path.test.ts`:
- Line 40: The test currently sets process.env.BUN_INSTALL via
"process.env.BUN_INSTALL = undefined as unknown as string", which creates the
literal string "undefined" and breaks getCandidatePaths(); change these to
actually remove the env var using delete process.env.BUN_INSTALL (apply the same
fix for the other occurrences noted around the test at the locations
corresponding to lines 55 and 80) so getCandidatePaths() sees the variable as
unset rather than the string "undefined".
- Around line 34-58: The test manipulates global state (Bun.which,
process.env.BUN_INSTALL, and process.env.PATH) but currently restores
process.env.PATH outside the finally, which can leak an empty PATH on assertion
failure; move the saved originalPath and its restoration into the finally block
alongside restoring (Bun as { which: typeof Bun.which }).which = originalWhich
and process.env.BUN_INSTALL to ensure resolveBunPath test fully cleans up even
when assertions fail (remove the out-of-finally PATH restore).

🧹 Nitpick comments (5)

tests/daemon/reaper.test.ts (1)

19-29: Prefer the static import over dynamic require for unlinkSync.

Line 22 uses require("node:fs") to get unlinkSync, but existsSync, mkdirSync, and writeFileSync are already statically imported on Line 7. Add unlinkSync to that import and use it directly here.

Also, tmpDir() directories are never cleaned up. Consider using rmSync(dir, { recursive: true }) in afterEach for a tidier teardown.

Proposed cleanup

-import { existsSync, mkdirSync, writeFileSync } from "node:fs";
+import { existsSync, mkdirSync, rmSync, unlinkSync, writeFileSync } from "node:fs";

+let cleanupDirs: string[] = [];
+
 function tmpDir(): string {
 	const dir = `/tmp/open-mem-reaper-test-${randomUUID()}`;
 	mkdirSync(dir, { recursive: true });
+	cleanupDirs.push(dir);
 	return dir;
 }

 afterEach(() => {
 	for (const p of cleanupPaths) {
 		try {
-			const { unlinkSync } = require("node:fs");
 			unlinkSync(p);
 		} catch {
 			// file may not exist
 		}
 	}
 	cleanupPaths = [];
+	for (const d of cleanupDirs) {
+		try {
+			rmSync(d, { recursive: true });
+		} catch {
+			// dir may not exist
+		}
+	}
+	cleanupDirs = [];
 });

src/servers/http-server.ts (1)

278-283: Consider logging swallowed errors in the empty catch blocks.

Both catch blocks silently discard all errors, including unexpected ones (e.g., permission denied). At minimum, logging at debug/warn level would aid troubleshooting without affecting the fallback flow.

Also applies to: 290-297

src/utils/bun-path.ts (1)

47-60: Consider logging which path was resolved for debuggability.

When the daemon fails to start, knowing which Bun binary was resolved (and how — Bun.which, candidate scan, or bare fallback) would be valuable for troubleshooting. A debug-level log here would help operators diagnose path issues without adding runtime overhead.

src/context/relevance.ts (2)

64-83: Docstring says "exponential decay" but implementation is a step function.

The comment on line 66 says "Compute recency score using exponential decay" but the implementation uses discrete buckets (1.0 / 0.8 / 0.5 / 0.2). Consider updating the doc to say "discrete buckets" or "step function" to avoid confusion.

---

164-175: scoreObservation is recomputed on every comparison during sort.

Array.sort calls the comparator O(n log n) times, and each call invokes scoreObservation for both a and b (including new Date() parsing in the sub-scores). For large observation sets, pre-computing scores into a Map and looking them up in the comparator would be significantly more efficient.

♻️ Pre-compute scores before sorting

 export function sortByRelevance(
 	entries: ReadonlyArray<ObservationIndex>,
 	context: ScoringContext,
 ): ObservationIndex[] {
+	const scores = new Map<string, number>();
+	for (const entry of entries) {
+		scores.set(entry.id, scoreObservation(entry, context));
+	}
 	return [...entries].sort((a, b) => {
-		const scoreA = scoreObservation(a, context);
-		const scoreB = scoreObservation(b, context);
+		const scoreA = scores.get(a.id)!;
+		const scoreB = scores.get(b.id)!;
 		// Descending — higher score first
 		if (scoreB !== scoreA) return scoreB - scoreA;
 		// Tie-break: more recent first
 		return new Date(b.createdAt).getTime() - new Date(a.createdAt).getTime();
 	});
 }

[greptile-apps]

Greptile Overview

Greptile Summary

Implements 5 feature enhancements bridging gaps identified from claude-mem comparison. The changes are well-architected with comprehensive test coverage (+64 new tests, 509 total passing).

Major Features:

• Privacy Utility — Shared stripPrivateBlocks() and redactSensitive() functions used by both tool-capture and chat-capture hooks to sanitize content before storage
• Orphan Reaper — Daemon lifecycle management that cleans up stale PID files at plugin startup, handling crashed sessions gracefully
• Bun PATH Resolution — Fallback-based resolver that checks Bun.which() → BUN_INSTALL → common install paths → fallback, cached for process lifetime to handle GUI environments
• Relevance Scoring — Weighted formula (40% recency, 30% type importance, 20% session affinity, 10% token efficiency) for intelligent observation prioritization in context building
• Progressive Disclosure — Enhanced tool descriptions explicitly guide AI agents through 3-layer workflow: search → timeline → recall

Minor Fixes:

• Dashboard stats API field naming aligned with TypeScript interface (totalTokensSaved, averageObservationSize)
• Dashboard directory now injectable for testing flexibility

All implementations follow the existing codebase patterns with proper error handling, edge case coverage, and clean separation of concerns.

Confidence Score: 5/5

• This PR is safe to merge with no concerns
• All five features are well-implemented with comprehensive test coverage (509 tests passing, +64 new tests). Code follows existing patterns, includes proper error handling, and has no breaking changes. The privacy utilities add security value, the reaper improves stability, and relevance scoring enhances UX. Type checking passes and all changes are backward-compatible.
• No files require special attention

Important Files Changed

Filename	Overview
src/utils/privacy.ts	New utility for stripping <private> blocks and redacting sensitive patterns. Clean implementation with proper error handling for invalid regex patterns.
src/daemon/reaper.ts	New orphan daemon PID cleanup utility. Safely handles stale PIDs, corrupt files, and running processes without killing them. Well-tested edge cases.
src/utils/bun-path.ts	Bun executable resolver with fallback logic. Checks Bun.which(), then common install paths, with process-lifetime caching. Handles GUI apps without shell PATH.
src/context/relevance.ts	Weighted relevance scoring system (40% recency, 30% type importance, 20% session affinity, 10% token efficiency). Well-documented with sensible defaults and tie-breaking logic.
src/context/progressive.ts	Adds optional relevance-based sorting to context builder. Observations sorted by score when scoringContext provided, preserving chronological order otherwise.
src/daemon/manager.ts	Uses resolveBunPathCached() instead of hardcoded "bun" for daemon spawning. Ensures daemon can start in environments without Bun in PATH.
src/servers/http-server.ts	Aligns stats API field names (totalTokensSaved, averageObservationSize) with dashboard types. Adds dashboardDir injection for testing flexibility.

Sequence Diagram

sequenceDiagram
    participant Plugin as Plugin Init
    participant Reaper as Orphan Reaper
    participant Privacy as Privacy Utils
    participant ToolHook as Tool Capture
    participant ChatHook as Chat Capture
    participant BunPath as Bun Path Resolver
    participant Daemon as Daemon Manager
    participant Context as Context Builder
    participant Relevance as Relevance Scorer
    
    Plugin->>Reaper: reapOrphanDaemons(dbPath)
    Reaper->>Reaper: Check PID file
    alt PID file exists and process dead
        Reaper->>Reaper: Remove stale PID
    end
    Reaper-->>Plugin: ReapResult
    
    Plugin->>BunPath: resolveBunPathCached()
    BunPath->>BunPath: Try Bun.which("bun")
    alt Bun.which fails
        BunPath->>BunPath: Scan common install paths
    end
    BunPath-->>Plugin: Bun executable path
    
    Plugin->>Daemon: start(bunPath)
    Daemon->>Daemon: Spawn daemon with resolved path
    
    Note over ToolHook,ChatHook: Runtime - Tool & Chat Capture
    
    ToolHook->>Privacy: stripPrivateBlocks(output)
    Privacy-->>ToolHook: Sanitized output
    ToolHook->>Privacy: redactSensitive(output, patterns)
    Privacy-->>ToolHook: Redacted output
    
    ChatHook->>Privacy: stripPrivateBlocks(message)
    Privacy-->>ChatHook: Sanitized message
    ChatHook->>Privacy: redactSensitive(message, patterns)
    Privacy-->>ChatHook: Redacted message
    
    Note over Context,Relevance: Context Injection - Progressive Disclosure
    
    Context->>Relevance: sortByRelevance(observations, scoringContext)
    Relevance->>Relevance: Calculate weighted scores<br/>(recency 40%, type 30%,<br/>session 20%, efficiency 10%)
    Relevance-->>Context: Sorted observations
    Context->>Context: Apply token budget with<br/>3-layer disclosure guidance
    Context-->>Plugin: Context string with prioritized observations

Loading