perf: stream prune integration checks, parallel removals - #1950
Conversation
Replace the blocking `par_iter().collect()` barrier in `step_prune` with channel-based streaming. Integration checks run on a background thread via rayon, sending results through a crossbeam channel as they complete. The main thread processes results immediately for age filtering and candidate collection, overlapping with still-running checks. Phase 2 removes confirmed candidates in parallel via `into_par_iter()`, with candidates sorted by original check order for deterministic output when rayon processes items sequentially. Performance (100 BranchOnly removals, release build): - Non-dry-run: 7.8s → 2.7s (parallel removals, 485% CPU utilization) - Dry-run first output: 2.3s silence → 0.2s progress message - Dry-run total: ~0.7s → ~0.6s Co-Authored-By: Claude <noreply@anthropic.com>
codecov/patch failed (94.8% — 4 uncovered lines in new code)
worktrunk-bot
left a comment
There was a problem hiding this comment.
codecov/patch failed — 94.8% patch coverage (4 uncovered lines of 77).
The code itself looks good (clean channel pattern, correct deterministic ordering, proper cleanup). The uncovered lines are:
-
Progress message (L1549–1552): Behind
check_items.len() > 10— no test creates that many branches. This is UX-only, no logic. -
drop(tx)(L1578): Emptycheck_itemspath. Would need a test with a repo that has only the default branch and no other worktrees/branches. -
Non-dry-run
skipped_youngreport (L1744–1750): The refactoring duplicated the "Skipped X (younger than Y)" message into both dry-run and non-dry-run branches. The existingtest_prune_min_age_skips_youngandtest_prune_stale_plus_youngtests only exercise dry-run. A non-dry-run variant with--yes --min-age=1hand a young worktree would cover this.
The easiest fix for coverage is #3 — add a --yes (non-dry-run) variant of one of the existing min-age tests.
Exercises the skipped_young message in the non-dry-run removal path (L1744-1750), which was flagged as uncovered by codecov/patch. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Removes the `if !integration_refs.is_empty()` / `else { drop(tx) }`
branch. Empty `par_iter` completes immediately, so always spawning the
thread is simpler and eliminates an uncovered code path.
Co-Authored-By: Claude <noreply@anthropic.com>
The `> 10` threshold prevented any test from covering the progress message. Removing it rather than lowering the threshold avoids changing all prune test snapshots. The channel approach already provides the UX improvement (overlapped processing). Co-Authored-By: Claude <noreply@anthropic.com>
CI failed — snapshot mismatch in test_prune_removes_integrated_detached
worktrunk-bot
left a comment
There was a problem hiding this comment.
Linux CI failed: test_prune_removes_integrated_detached snapshot mismatch (log).
This test produces two parallel candidates (the detached worktree + the orphan detached-integrated branch), but doesn't pin RAYON_NUM_THREADS=1. The branch deletion finishes before the worktree removal, flipping the output order. Other multi-candidate tests (lines 144, 224, 288, 324) already pin it.
Fix in tests/integration_tests/step_prune.rs at line 197 — same pattern as test_prune_multiple:
let mut cmd = make_snapshot_cmd(
&repo,
"step",
&["prune", "--yes", "--min-age=0s"],
None
);
cmd.env("RAYON_NUM_THREADS", "1"); // deterministic output order
assert_cmd_snapshot!(cmd);Then cargo insta test --accept to update the snapshot. I can push this fix if you'd like.
…rune_removes_integrated_detached on linux) Co-Authored-By: Claude <noreply@anthropic.com>
Replace the blocking
par_iter().collect()barrier instep_prunewith channel-based streaming. Integration checks run on a background thread via rayon, sending results through a crossbeam channel as they complete. The main thread processes results as they arrive for age filtering and candidate collection, overlapping with still-running checks.Phase 2 removes confirmed candidates in parallel via
into_par_iter(). Candidates are sorted by original check order for deterministic output.Performance (100 BranchOnly removals, release build):
Tests with multiple removal candidates pin
RAYON_NUM_THREADS=1for deterministic snapshot output.