Reconcile cache with filesystem on get-projects so sessions stop going missing - #60
Merged
Merged
Conversation
…g missing The session cache is only rebuilt from the filesystem on a cold start (populateCacheViaWorker runs only when the cache is completely empty). Once the cache has any rows, a project folder that changed while the app was closed — or that was created before the build which first indexed it — is never re-scanned, so its sessions (and whole worktrees) silently disappear from the sidebar. The transcripts are intact on disk; only the cache is stale. Make get-projects reconcile first: re-index folders that are new or whose newest .jsonl is newer than what we last indexed. The check is a cheap, stat-only gate (getFolderIndexMtimeMs vs cache_meta.indexMtimeMs), so it's effectively free when nothing changed and only does real work for folders that actually need it. The previously dead populateCacheFromFilesystem() (defined and exported but never called) is repurposed into this gated reconcileCacheFromFilesystem().
ymajoros
force-pushed
the
fix/reconcile-cache-on-get-projects
branch
from
June 1, 2026 10:19
274eb72 to
edf06c0
Compare
abasiri
added a commit
to Davidb-2107/switchboard
that referenced
this pull request
Aug 1, 2026
…e test Branch was 14 behind. Two things needed fixing, both merge-order artifacts — doctly#60 and doctly#65 landed after this branch was written. schedule-runner.js conflicted, but the two changes compose rather than compete. doctly#65 made scanSchedules prefer a cache_meta folder→projectPath lookup, falling back to reading a JSONL head only for folders missing from the cache; this branch made that head read cheap. Resolved by keeping doctly#65's structure and putting readHead(…, 4096) inside its readProjectPathFromJsonl fallback, so the common path does no file read at all and the fallback no longer loads a possibly-hundreds-of-MB file to look at its first line. test/reconcile-cache.test.js failed because refreshFolder now fetches the cached row to use as resume state, and that test's fake db predates the method: ✖ reconcileCacheFromFilesystem indexes new and stale folders … getCachedSession → undefined Added getCachedSession() { return null; } to the fake, modelling a session with nothing indexed yet. Fixed in the fake rather than guarding the call site: the real db provides the method, and a guard would mask genuine wiring errors. 27/27 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #59.
Problem
Sessions and whole worktrees silently vanish from the sidebar while their transcripts are still on disk, and reopening/restarting doesn't bring them back.
get-projectsonly scans the filesystem on a cold start:Once the cache is non-empty it's never reconciled again, so folders that changed while the app was closed — or that were never indexed by the current build (
cache_meta.indexMtimeMs === 0) — never reappear. The livefs.watchonly covers changes while the app is running.Fix
Reconcile on
get-projectsbefore building from cache. The deadpopulateCacheFromFilesystem()(defined + exported, never called) is repurposed intoreconcileCacheFromFilesystem(), now stat-gated: it re-indexes only folders that are new or whose newest.jsonlis newer than what was last indexed.When nothing changed this is just a
readdir+statper folder — no transcript parsing, no DB writes — so the steady-state cost onget-projectsis negligible; real work happens only for folders that actually need it.refreshFolderalready does its own per-file mtime skip, so this composes cleanly.Why here (vs. only at startup)
Reconciling on
get-projectsis self-healing for both the "changed while closed" and "old build never indexed it" cases, and naturally re-runs if folders drift later — without a separate watcher or timer. The mtime gate keeps it cheap. Happy to move it to a one-time startup pass instead if you'd prefer.Testing
node --testpasses, including a newtest/reconcile-cache.test.jsthat asserts the gate: a never-indexed folder and a stale folder (indexMtimeMsolder than disk) get indexed, while an up-to-date folder is skipped.indexMtimeMs = 0) re-indexed and reappeared in the sidebar.Notes
Pure cache/indexing change — no schema change, no change to
session_meta(names/stars/archive) or settings.