feat: live transcript capture pipeline by Coldaine · Pull Request #27 · Coldaine/AgentVisualCrazy

Coldaine · 2026-04-17T05:42:01Z

Summary

Implements the full live transcript capture pipeline (issue #9).

What's included

New files: \shadow-agent/src/capture/\

File	Purpose
\session-discovery.ts\	Scans ~/.claude/projects/*/.jsonl, returns most recently modified file
\ ranscript-watcher.ts\	\s.watch\ on a JSONL file with byte-offset reads; handles truncation/rotation
\incremental-parser.ts\	Stateful line buffer — splits incoming chunks on \\n, JSON-parses complete lines
\
ormalizer.ts\	Maps raw transcript entries to \CanonicalEvent[]\ (message, tool_started, tool_completed, session_started)
\�vent-buffer.ts\	Ring buffer (capacity 2000) with \push, \getRecent, \getAll, \getSince, \subscribe\
\ipc-bridge.ts\	Debounced push at 150ms + \shadow:snapshot\ / \shadow:events-since\ IPC handlers
\session-manager.ts\	Orchestrates the full pipeline with 30s re-discovery and catch-up read on session start

Modified

\start-main-process.ts\ — wires \SessionManager\ into \�pp.whenReady()\
\session-io.ts\ — fix TypeScript strict-null dialog argument
\ ests/renderer/bridge.test.ts\ — fix TypeScript cast

Tests

17 new tests in \ ests/capture.test.ts\ covering:

Incremental parser (line splitting, CRLF, invalid JSON, reset)
Normalizer (message, tool_use, tool_result, unknown type)
Event buffer (push, eviction, getRecent, getSince, subscribe, unsubscribe, clear)
Session discovery (graceful null return for missing path)

All 41 tests pass. Zero TypeScript errors.

Closes #9

Copilot

Pull request overview

Implements the live Claude transcript capture pipeline in shadow-agent/src/capture/ and wires it into the Electron main process to enable live event ingestion and IPC exposure (issue #9).

Changes:

Add capture pipeline modules: session discovery, transcript file watcher, incremental JSONL parser, entry normalizer → CanonicalEvent[], ring buffer, IPC bridge, and session manager orchestration.
Wire SessionManager startup/shutdown into Electron startMainProcess().
Add Vitest coverage for parser/normalizer/buffer/discovery and fix a couple of strict TS casts.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
shadow-agent/src/capture/session-discovery.ts	Recursively finds most recently modified JSONL/NDJSON transcript file (or supports override path).
shadow-agent/src/capture/transcript-watcher.ts	`fs.watch` + byte-offset tail reader with truncation handling.
shadow-agent/src/capture/incremental-parser.ts	Stateful JSONL line buffering + tolerant JSON parse with warnings.
shadow-agent/src/capture/normalizer.ts	Maps transcript entries into `CanonicalEvent` stream for the rest of the app.
shadow-agent/src/capture/event-buffer.ts	In-memory ring buffer with subscription and querying helpers.
shadow-agent/src/capture/ipc-bridge.ts	IPC handlers (`shadow:snapshot`, `shadow:events-since`) + debounced push (`shadow:events`).
shadow-agent/src/capture/session-manager.ts	Orchestrates discovery → catch-up read → watch → parse → normalize → buffer → IPC.
shadow-agent/src/electron/start-main-process.ts	Instantiates and starts/stops the session manager with app lifecycle.
shadow-agent/src/electron/session-io.ts	Adjusts dialog invocation to satisfy strict TS (but currently via `null!`).
shadow-agent/tests/capture.test.ts	Adds tests for parser/normalizer/buffer/discovery.
shadow-agent/tests/renderer/bridge.test.ts	Fixes TS cast to set up `globalThis.window` in tests.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 215fca1541

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Nine drift items were identified during the 2026-04-19 end-of-day review (captured in docs/third-party-readiness-briefing.html, added here). Each is addressed as a targeted edit rather than a rewrite -- the docs are mostly right, they were just out of step with what PRs #28/#29/#31/#32/#34/#35 actually landed. Changes: - architecture.md: "Phase 2 (Release Candidate)" was misleading -- Canvas2D (#26) and live capture (#27) are still on branches. Relabel to "Phase 2 (In Progress)" and state the two PRs that close the phase. Also replaces the Three.js-only background claim with the Citadel-dot-grid-preferred decision from plan-gui-rendering.md, and converts the four section-sign cross-refs to real Markdown links. - domain-gui.md / domain-events.md / domain-inference.md: unify status headers under one schema (Planned / Partial on main / Landed on main) instead of the three different styles previously in use. - history/log.md: fix ascending chronological ordering (04-18 was before 04-12) and add the missing 2026-04-19 entry covering PRs #29, #31, #32, #34, #35 merged that day. - plan-master-a-must-do.md: RepoVis archival, temp-file removal, and domain status headers were listed as TODO but were all completed 2026-04-18/19. Mark them done with audit-trail strikethrough. Also refresh the Pre-Commit/CI section (husky -> .githooks migration) and the Definition of Done (five PRs merged, four remain). - plan-master-coordination.md: remove the three empty Stream A/B/C scratchpad placeholders and replace the Round 2 Dispatch TODO list with a closeout record pointing to the promoted outputs. - plan-gui-rendering.md / plan-event-capture.md / plan-inference-engine.md: convert the remaining section-sign cross-refs into Markdown links with descriptive section names. - getting-started.md: cd into shadow-agent for test commands, note the PR #35 bisect caveat, and mention the CI/pre-commit gates. - prompts/shadow-system-prompt.json: append a 2026-04-19 iteration-log entry documenting the regeneration; regenerated docs/prompts/ shadow-system-prompt.md is in sync (npm run prompts:check passes). - docs/third-party-readiness-briefing.html: commit the HTML artifact that drove this pass so the reasoning behind these edits is auditable. Validation: 145 tests pass (npm test), npm run prompts:check clean, no lint errors. Scope note: open PRs #26, #27, #30, #33 are deliberately untouched -- those are thousand-plus-line product-feature or infrastructure branches and each deserves its own focused session. Made-with: Cursor

* docs: remediate drift between documentation and current main state Nine drift items were identified during the 2026-04-19 end-of-day review (captured in docs/third-party-readiness-briefing.html, added here). Each is addressed as a targeted edit rather than a rewrite -- the docs are mostly right, they were just out of step with what PRs #28/#29/#31/#32/#34/#35 actually landed. Changes: - architecture.md: "Phase 2 (Release Candidate)" was misleading -- Canvas2D (#26) and live capture (#27) are still on branches. Relabel to "Phase 2 (In Progress)" and state the two PRs that close the phase. Also replaces the Three.js-only background claim with the Citadel-dot-grid-preferred decision from plan-gui-rendering.md, and converts the four section-sign cross-refs to real Markdown links. - domain-gui.md / domain-events.md / domain-inference.md: unify status headers under one schema (Planned / Partial on main / Landed on main) instead of the three different styles previously in use. - history/log.md: fix ascending chronological ordering (04-18 was before 04-12) and add the missing 2026-04-19 entry covering PRs #29, #31, #32, #34, #35 merged that day. - plan-master-a-must-do.md: RepoVis archival, temp-file removal, and domain status headers were listed as TODO but were all completed 2026-04-18/19. Mark them done with audit-trail strikethrough. Also refresh the Pre-Commit/CI section (husky -> .githooks migration) and the Definition of Done (five PRs merged, four remain). - plan-master-coordination.md: remove the three empty Stream A/B/C scratchpad placeholders and replace the Round 2 Dispatch TODO list with a closeout record pointing to the promoted outputs. - plan-gui-rendering.md / plan-event-capture.md / plan-inference-engine.md: convert the remaining section-sign cross-refs into Markdown links with descriptive section names. - getting-started.md: cd into shadow-agent for test commands, note the PR #35 bisect caveat, and mention the CI/pre-commit gates. - prompts/shadow-system-prompt.json: append a 2026-04-19 iteration-log entry documenting the regeneration; regenerated docs/prompts/ shadow-system-prompt.md is in sync (npm run prompts:check passes). - docs/third-party-readiness-briefing.html: commit the HTML artifact that drove this pass so the reasoning behind these edits is auditable. Validation: 145 tests pass (npm test), npm run prompts:check clean, no lint errors. Scope note: open PRs #26, #27, #30, #33 are deliberately untouched -- those are thousand-plus-line product-feature or infrastructure branches and each deserves its own focused session. Made-with: Cursor * docs: address Copilot + Codex review feedback on drift-remediation PR Fixes all six inline comments from the automated reviewers on PR #36. Root cause for four of them was my conflation of the in-flight husky -> .githooks migration (which is stashed, not on this branch) with current reality on main. The other two were residual section-sign cross-refs that my first sweep missed. - getting-started.md: reference the actual `.husky/pre-commit` hook (not a speculative `.githooks/pre-commit`); name the CI workflow correctly (it is `CI`, defined in `.github/workflows/prompt-parity.yml`); accurately describe what the hook runs (`prompts:check` and `npm test --prefix shadow-agent`). - plan-master-a-must-do.md: drop the incorrect "hook has since migrated to .githooks/pre-commit" claim; describe the current `.husky/pre-commit` instead. Convert the remaining "under section 3" cross-ref to a clickable anchor link. - plan-master-coordination.md: replace the Stream A/B/C "section 1 / 2 / 3" cross-refs with descriptive Markdown links to the actual files and named sections. Same fix applied to the Round 2 Dispatch PR-fix bullet. - architecture.md: replace the "section 8" cross-ref with the full step heading "Optional Background Atmosphere Layer (Choose One)" as a descriptive link target. Validation: rg for section-sign notation returns zero hits; `npm run prompts:check` clean; `npm test` 145/145 pass. Made-with: Cursor

Implements the full capture pipeline for issue #9: - session-discovery.ts: scans ~/.claude/projects/**/*.jsonl for active session - transcript-watcher.ts: byte-offset fs.watch with truncation/rotation handling - incremental-parser.ts: stateful line buffer that splits JSONL chunks - normalizer.ts: maps raw transcript entries to CanonicalEvent[] - event-buffer.ts: ring buffer (capacity 2000) with subscribe/getSince/getRecent - ipc-bridge.ts: debounced push (150ms) + shadow:snapshot / shadow:events-since handlers - session-manager.ts: orchestrates discovery→watcher→parser→normalizer→buffer→IPC with 30s re-discovery and catch-up read on session start - start-main-process.ts: wires SessionManager into app.whenReady() - tests/capture.test.ts: 17 tests covering parser, normalizer, buffer, and discovery - Fix session-io.ts and bridge.test.ts TypeScript errors on this branch Closes #9 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Address capture pipeline race, lifecycle, and payload-shape issues; wire live IPC bridge methods; and align prompt artifacts and tests so PR #27 is merge-ready after rebase. Made-with: Cursor

Copilot AI review requested due to automatic review settings April 17, 2026 05:42

Copilot started reviewing on behalf of Coldaine April 17, 2026 05:42 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Apr 17, 2026

View reviewed changes

Comment thread shadow-agent/src/capture/session-manager.ts

Comment thread shadow-agent/src/capture/normalizer.ts

Comment thread shadow-agent/src/capture/ipc-bridge.ts

coderabbitai Bot mentioned this pull request Apr 17, 2026

feat: master coordination plans, architecture assessment, and finish-line closeout #33

Merged

Coldaine requested a review from Copilot April 19, 2026 00:05

Copilot started reviewing on behalf of Coldaine April 19, 2026 00:05 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Coldaine requested a review from Copilot April 19, 2026 00:17

Copilot started reviewing on behalf of Coldaine April 19, 2026 00:17 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Coldaine mentioned this pull request Apr 20, 2026

docs: remediate drift between documentation and current main state #36

Merged

5 tasks

Coldaine force-pushed the work/event-capture branch from 215fca1 to 8f9bf58 Compare April 20, 2026 11:16

Coldaine merged commit f82e2ee into main Apr 20, 2026
2 checks passed

Conversation

Coldaine commented Apr 17, 2026

Summary

What's included

New files: \shadow-agent/src/capture/\

Modified

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants