perf(server): add targeted projection queries

[juliusmarminge]

Summary

• add targeted projection query APIs for counts, active project/thread lookups, and single-thread checkpoint context
• switch checkpoint diff queries and startup bootstrap/heartbeat paths off the full orchestration snapshot
• add backend tests covering the new targeted query paths

Validation

• bun run test src/orchestration/Layers/ProjectionSnapshotQuery.test.ts src/checkpointing/Layers/CheckpointDiffQuery.test.ts
• bun run test src/orchestration/Layers/ProjectionSnapshotQuery.test.ts src/checkpointing/Layers/CheckpointDiffQuery.test.ts src/persistence/Layers/ProjectionThreadMessages.test.ts src/orchestration/Layers/ProjectionPipeline.test.ts src/orchestration/Layers/OrchestrationEngine.test.ts src/orchestration/Layers/CheckpointReactor.test.ts src/orchestration/Layers/ProviderRuntimeIngestion.test.ts src/orchestration/Layers/ProviderCommandReactor.test.ts
• bun fmt
• bun lint
• bun typecheck (currently fails in apps/web due pre-existing @effect/atom-react / Layer.suspend errors)

---

Note

Medium Risk
Moderate risk because it changes the ProjectionSnapshotQuery service contract, adds new SQL query paths plus indexes/migration, and alters startup + diff flows to depend on the new targeted lookups.

Overview
Reduces server startup and diff-query overhead by avoiding full orchestration snapshot hydration. ProjectionSnapshotQuery now exposes targeted read APIs (getCounts, getActiveProjectByWorkspaceRoot, getFirstActiveThreadIdByProjectId, getThreadCheckpointContext) backed by narrow SQL queries.

CheckpointDiffQuery switches to getThreadCheckpointContext (and derives cwd from worktreePath ?? workspaceRoot) instead of loading the full snapshot, and startup bootstrap/telemetry paths in serverRuntimeStartup.ts are updated to use the new targeted queries; the startup heartbeat is now launched asynchronously via launchStartupHeartbeat.

Adds migration 019_ProjectionSnapshotLookupIndexes to create supporting indexes on projection_projects(workspace_root, deleted_at) and projection_threads(project_id, deleted_at, created_at), plus new/updated tests covering the targeted query behavior and ensuring full-snapshot reads are not triggered.

^{Written by Cursor Bugbot for commit 711a3e0. This will update automatically on new commits. Configure here.}

Note

Add targeted projection queries to avoid full snapshot hydration on server startup

• Adds four new methods to ProjectionSnapshotQueryShape: getCounts, getActiveProjectByWorkspaceRoot, getFirstActiveThreadIdByProjectId, and getThreadCheckpointContext, each fetching only the data needed rather than loading the full read model.
• Updates autoBootstrapWelcome and recordStartupHeartbeat in serverRuntimeStartup.ts to use these targeted queries instead of getSnapshot.
• Updates CheckpointDiffQuery.getTurnDiff in CheckpointDiffQuery.ts to fetch only the relevant thread checkpoint context via getThreadCheckpointContext.
• Adds migration 019_ProjectionSnapshotLookupIndexes.ts with indexes on projection_projects(workspace_root, deleted_at) and projection_threads(project_id, deleted_at, created_at) to support the new queries.
• The startup heartbeat now runs asynchronously via launchStartupHeartbeat (forked), so it no longer blocks the server ready event.

^{Macroscope summarized 711a3e0.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 11d85875-d0ea-4177-8b4b-80ed71697f3a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/backend-perf-queries

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: New query excludes deleted threads unlike original code
  • Removed the unintentional AND threads.deleted_at IS NULL filter from getThreadCheckpointContextThreadRow so soft-deleted threads are still found during checkpoint diff lookups, matching the original behavior.

Or push these changes by commenting:

@cursor push 1a06fbeaff

Preview (1a06fbeaff)

diff --git a/apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.ts b/apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.ts
--- a/apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.ts
+++ b/apps/server/src/orchestration/Layers/ProjectionSnapshotQuery.ts
@@ -407,7 +407,6 @@
         INNER JOIN projection_projects AS projects
           ON projects.project_id = threads.project_id
         WHERE threads.thread_id = ${threadId}
-          AND threads.deleted_at IS NULL
         LIMIT 1
       `,
   });

_{You can send follow-ups to this agent here.}

[cursor]

New query excludes deleted threads unlike original code

Medium Severity

The getThreadCheckpointContext SQL query adds AND threads.deleted_at IS NULL, but the original code path via getSnapshot() returned all threads (including deleted ones) and CheckpointDiffQuery looked up the thread by ID without any deleted_at filter. This behavioral change means checkpoint diff requests for soft-deleted threads now fail with a "Thread not found" error instead of returning the diff data.

Additional Locations (1)

• apps/server/src/checkpointing/Layers/CheckpointDiffQuery.ts#L43-L51

 

[macroscopeapp]

Approvability

Verdict: Needs human review

An open review comment identifies a potential behavioral regression: the new targeted query filters out deleted threads while the original code did not, which could cause checkpoint diff requests for soft-deleted threads to fail. This warrants human review to determine if this behavioral change is intentional.

^{You can customize Macroscope's approvability policy. Learn more.}