feat(storage): archived cleanup with quarantine (phase 2 of #42) - #529
Conversation
Add preview-first archived-session cleanup (oldest N%) with quarantine under CODEX_HOME/.trash and optional permanent delete. Fold Phase 1 hygiene: immutable comment, skip .trash in scan, drop dead scanStorage imports, and document Storage on the dashboard guide.
|
Tracks Phase 2 of #42 (Option A: quarantine default + permanent checkbox). Phase 1 hygiene included in the same PR. |
This comment was marked as resolved.
This comment was marked as resolved.
Strip host paths from preview JSON, harden FS/DB error mapping, localize cleanup copy, and cover permanent/busy API paths.
Require a candidate digest, stage-then-commit with full rollback, exact archived path matching, spawn-edge/FK cleanup, and mapped 409/400 errors with GUI dialog surfacing.
Localize cleanup errors by code, keep dialog focus stable, harden path/history probes, cap preview payloads, and cover mid-batch rename rollback.
Preserve unrestored stage directories on rollback/purge failure and return the relative trashDir so recovery info reaches the API.
Avoid reading/writing the ref during render so GUI eslint stays green in CI.
Re-check referenced history under the write lock, rewrite partial-purge manifests to surviving entries, and stop claiming fs_failed is a no-op.
CloseConfirm clears the permanent checkbox on cancel/escape/backdrop/stale preview, and preset buttons use locale-aware percent labels.
Match Codex delete_threads_strict across per-thread stores, fail closed on threads read errors before mutating files, and cover permanent multi-DB cleanup.
Freeze thread IDs before satellite deletes, snapshot logs/memories/goals for rollback, and keep the recovery trash path when satellite restore fails.
Snapshot logs/memories/goals first, write satellite-backup.json, then mutate. Backup-write failures leave DBs and rollouts untouched; cover single-store homes.
Acquire BEGIN IMMEDIATE on logs/memories/goals before snapshotting, persist backup before committing satellite deletes, and restore snapshotted PK rows with conflict-safe inserts. Add regression for concurrent log insert and memory-job update surviving injected state-commit failure.
… race-safe Roll back and close all acquired satellite write locks when any BEGIN IMMEDIATE fails. Restore the global consolidation job only when the live row still matches cleanup's post-delete image, and add regressions for busy final-satellite rollback and concurrent enqueue watermark updates.
…ssion Store logs and memories read-only Database instances in variables and close them in finally blocks so Windows CI does not leak file locks.
Bind digest per physical file stats, create exclusive stage dirs with private permissions, journal manifest before staging, persist consolidate post-image before memories commit, batch large SQL IN clauses, validate archive paths, and localize GUI transport errors.
Batch satellite snapshot/delete IN queries to stay under SQLite variable limits, return fs_failed when exclusive stage-dir creation fails, and split injected-failure regressions into test.each cases.
Restrict wire-forwarded _test hooks via pickWireCleanupTestHooks and align runWithDigest typing with ExecuteCleanupOptions.
Security review — PR #529 (storage phase 2 cleanup)Reviewer: automated security pass (post CodeRabbit/Codex fixes, including @lidge-jun @Ingwannu — CI is green (ubuntu/macos/windows + npm-global + react-doctor). Codex and CodeRabbit review threads are addressed. This PR is merge-ready pending your maintainer sign-off. Auth boundaryCleanup routes sit behind the same stack as the rest of
Path traversal / filesystem scope
Symlinks under Destructive FS/DB operationsDesign matches the stated threat model:
Residual (acceptable): the initial DB probe is not held for the entire operation. If Codex grabs an exclusive lock mid-flight, cleanup returns Test hook exposure (
|
| Endpoint | Disclosure |
|---|---|
POST .../preview |
Relative relPath only; strips codexHome / absPath (tested). |
POST .../cleanup errors |
Mapped error codes + fixed message strings; trashDir is relative (e.g. .trash/173…). |
GET /api/storage |
Still returns full codexHome (phase 1; unchanged by this PR). |
Error JSON is asserted not to contain absolute host paths in tests/api-storage-cleanup.test.ts. GUI renders errors as React text (no dangerouslySetInnerHTML); trashDir in i18n is server-controlled relative path.
Manifest on disk may include threadId / rolloutPath for recovery — local session metadata under 0o600 in user-owned .trash; acceptable for same-user recovery.
Race conditions
| Scenario | Mitigation |
|---|---|
| Preview → execute TOCTOU | Digest includes per-file mtimes; file set changes → stale_preview. |
| Concurrent Codex writer | Upfront probe + busy mapping; staged FS rollback on DB failure. |
| Partial permanent purge | purgeIncomplete manifest; relative trashDir in 500 response (tested). |
| Satellite DB partial commit | Snapshot + restoreSatelliteBackup; keeps stage when restore fails. |
| Concurrent satellite writes during cleanup | BEGIN IMMEDIATE write locks held through snapshot/delete; conflict-safe INSERT … ON CONFLICT DO NOTHING restore. |
GUI (Storage.tsx)
- No
dangerouslySetInnerHTML; paths shown as text in<li>/ tables. - Confirm modal + permanent checkbox before destructive execute; checkbox resets on dismiss.
- Preview list capped at 8 rows but count/bytes/digest reflect the full preview response (digest still binds full server-side set).
Findings summary
| Area | Blocker? | Notes |
|---|---|---|
| Auth / CSRF | No | Same model as existing management API. |
| Path traversal | No | Strong containment on all changed paths. |
| Destructive ops | No | Scoped to archived files; active sessions protected. |
| Test hooks | No | Env-gated; allowlisted; no privilege escalation. |
| Info disclosure | No | Cleanup responses avoid host paths; relative recovery hints only. |
| Races / recovery | No | Documented partial-failure behavior with manifest/trash. |
Optional hardening (not required for merge):
- API test that
_testis ignored whenOPENCODEX_CLEANUP_TEST_HOOKSis unset. - One-time startup warning if
OPENCODEX_CLEANUP_TEST_HOOKS=1outsideNODE_ENV=test. - Ops note that remote/LAN deployments must use API auth (already required for non-loopback).
Conclusion
PR #529 storage cleanup is well-bounded for a local-first management surface. No concrete medium+ vulnerabilities with realistic exploitability beyond the existing "whoever can call the local management API can manage the local proxy" trust model.
Merge recommendation: approve.
Ingwannu
left a comment
There was a problem hiding this comment.
Independent maintainer re-review completed on 3b0c85d2.
The three still-open Codex threads are stale against the current head:
createExclusiveStageDircreates the quarantine directory withoutrecursive, suffixes timestamp collisions, and applies private permissions;- the staging manifest is written before the first rename, then finalized before DB reconciliation;
- the memories consolidate post-image is persisted to
satellite-backup.jsonbefore that transaction commits.
Required Linux/macOS/Windows, npm-global, and React Doctor checks are green. The recorded security review covers the final cleanup routes, filesystem scope, recovery sequence, and wire test-hook gate. Approved for dev.
Inventory all 23 open issues and 14 open PRs on the axis of decisions only the maintainer can make, clustered into six bundles for later PABCD cycles. Live findings: - anthropics/claude-code#1124, cited as upstream evidence on #543, is CLOSED - #462/#401/#241 carry upstream-tracking with no upstream ticket at all - #92's encrypted_content body has no filed upstream issue either - #527 targets #526's head branch, so enforce-target fails - #529 is already merged; the earlier triage matrix still lists it in flight
Two read-only audit lenses returned 30 contradictions; the verified ones are applied here and recorded in 006_corrections.md. Factual corrections: - claude-code#1124 closed 2025-05-16, not 2025-08-10 - 13 open enhancement issues, not 11 - #529 is merged, so issue #42 phase 2 is done - #418 has no PR behind it; it was miscounted as in-flight - #528 does not contain #424's current head, so it needs a rebase - #527's retarget is manual and independent of #526 Framing corrections: - stale-needs-info.yml is absent from the default branch, so bundle C had no real deadline - the debug switch #543's reporter asked for already exists - whether #545 is our defect is answerable from code, not owner judgment - #491 is an OAuth credential change, so it is a security boundary rather than a warm-up; #533 and #447 keep their security classification too Structure: bundle G added for roadmap honesty, bundle F rescoped to the security boundary, #498 split out of bundle B, cycle order rebuilt, and the loop archetype corrected to decision-elicitation.
Summary
CODEX_HOME/.trash/<epoch>/, permanent delete only behind an explicit checkbox.sessions/are never touched. Cleanup probesstate_*.sqlitewritability first and returns409 codex_busywith zero FS changes when Codex holds the lock.threadsrows (andthread_spawn_edgeswhen present) with archived JSONL paths..trashfrom scan totals/other, remove deadscanStorageimports, document Storage on the web dashboard guide.API
POST /api/storage/cleanup/preview{ percent }POST /api/storage/cleanup{ percent, mode: "quarantine" | "permanent" }Test plan
bun test --isolate tests/storage-scanner.test.ts tests/storage-cleanup.test.ts tests/api-storage-cleanup.test.ts tests/api-storage.test.tsbun run typecheckbun x tsc -p gui --noEmit.trash+ rescan; busy Codex → refusedSummary by CodeRabbit
CODEX_HOME(read-only)..trash/no longer affect Storage totals or “largest files.”