Skip to content

test(storage): give the policy suite an explicit harness budget - #732

Merged
lidge-jun merged 1 commit into
devfrom
codex/260730-issue727-storage-policy-timeout
Jul 30, 2026
Merged

test(storage): give the policy suite an explicit harness budget#732
lidge-jun merged 1 commit into
devfrom
codex/260730-issue727-storage-policy-timeout

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Closes #727.

What breaks

tests/storage-policy.test.ts inherits bun's 5s default harness timeout. Its
seedHome() helper writes real files plus a sqlite fixture for every case it
backs — 15 call sites — and on a slow windows-latest runner those cases land at
6-8s and fail. Run 30507689929
failed five of them, then failed the same set again after a --failed rerun, so
it reproduces rather than flakes. Locally the whole file is 22 pass in ~100ms.

Every failure sat between 6s and 8s against a 5s ceiling: these are timeouts,
not assertion failures, and nothing indicates the selection logic is wrong.

What changes

One file, setDefaultTimeout(30_000) plus a comment naming the cause and the
run id. That is the same budget the sibling tests/storage-cleanup.test.ts
already gives its seeding-heavy cases — { timeout: 20_000 } at 14 sites and
{ timeout: 30_000 } at 5, with a comment saying seeding "trips bun's default
5s harness timeout". A file-level call covers all 15 seedHome sites here
without touching each case, and setDefaultTimeout(30_000) is already the
convention in grok-management-api, codex-history-provider,
claude-management-api, server-combo-failover-e2e and
subagent-fallback-handle-responses.

No src/ change. No workflow change — the job-level ceiling was already raised
to 20 minutes in #717, and this is the per-test budget underneath it.

Verification

Mechanism proven in isolation first, under CI's --isolate mode: a 6s case with
no budget fails with this test timed out after 5000ms; the identical case with
setDefaultTimeout passes at 6005ms. Same failure shape as #727.

  • bun x tsc --noEmit — exit 0
  • bun test --isolate tests/storage-policy.test.ts — 22 pass, 0 fail, 91 expect() calls

Windows behaviour is what CI has to confirm; the local run only proves the
change is inert on a fast filesystem.

Follow-up, not in scope

Windows pays roughly 2.5x for this suite versus ubuntu. Raising budgets hides
the margin, it does not close the gap. The suspect is bun test --isolate in
ci.yml (per-test process isolation is expensive on Windows), which is a
reliability tradeoff and an owner call, not something to fold into a timeout fix.
Making seedHome cheaper is the other candidate.

Summary by CodeRabbit

  • Tests
    • Increased the default test timeout to improve reliability on slower environments.
    • Reduced intermittent failures during filesystem and database setup.

seedHome() writes real files and a sqlite fixture per case. On a slow
windows-latest runner those cases take 6-8s against bun's 5s default and fail
deterministically (run 30507689929 failed the same five twice, the second time
after a --failed rerun) while a faster runner passes the identical code —
locally the file is 22 pass in ~100ms.

The sibling storage-cleanup suite already carries { timeout: 20_000 } /
{ timeout: 30_000 } on its seeding-heavy cases with a comment naming the same
cause. This file had no budget at all, so one file-level setDefaultTimeout(30_000)
covers all fifteen seedHome sites. setDefaultTimeout is the existing convention
here (grok-management-api, codex-history-provider, claude-management-api,
server-combo-failover-e2e, subagent-fallback-handle-responses all use 30_000).

Test-harness only: no src/storage change, since nothing indicates the selection
logic is wrong.

Closes #727
@github-actions github-actions Bot added the chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature). label Jul 30, 2026
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a1f41177-9b10-4f6e-aa45-2e4ca2b6cc9a

📥 Commits

Reviewing files that changed from the base of the PR and between 67c731e and e220753.

📒 Files selected for processing (1)
  • tests/storage-policy.test.ts

📝 Walkthrough

Walkthrough

The storage policy test suite now imports setDefaultTimeout and applies a file-wide 30-second timeout to accommodate slower seedHome filesystem and SQLite setup on CI runners.

Changes

Storage policy test harness

Layer / File(s) Summary
Configure test timeout
tests/storage-policy.test.ts
Imports Bun's setDefaultTimeout and sets a file-wide timeout of 30_000 ms for storage policy tests.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

Suggested labels: bug

Suggested reviewers: wibias, ingwannu

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately reflects the main change: adding an explicit harness timeout budget to the storage policy test suite.
Linked Issues check ✅ Passed The PR addresses #727 by raising the timeout in tests/storage-policy.test.ts for seeding-heavy Windows cases that exceed Bun's 5s default.
Out of Scope Changes check ✅ Passed The change stays scoped to the test harness in tests/storage-policy.test.ts and does not alter production code or workflows.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/260730-issue727-storage-policy-timeout

Comment @coderabbitai help to get the list of available commands.

@lidge-jun
lidge-jun merged commit 60e1004 into dev Jul 30, 2026
12 checks passed
@lidge-jun
lidge-jun deleted the codex/260730-issue727-storage-policy-timeout branch July 30, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant