Skip to content

test(picker): release --prs loading marker on parent exit, not a timer - #3532

Merged
max-sixty merged 2 commits into
mainfrom
fix/flaky-prs-loading-marker
Jul 25, 2026
Merged

test(picker): release --prs loading marker on parent exit, not a timer#3532
max-sixty merged 2 commits into
mainfrom
fix/flaky-prs-loading-marker

Conversation

@worktrunk-bot

@worktrunk-bot worktrunk-bot commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

Problem

test_switch_picker_prs_shows_loading_marker intermittently times out on test (windows) — it flaked on #3531 (a docs-only change), which is what prompted this fix.

The test asserts a transient frame: while a mocked, delayed gh pr list is in flight, the picker header shows ↳ Loading open PRs… and the #42 row has not yet streamed in. The mock held that loading state for a fixed time (originally 3s). On a loaded Windows runner, the PTY boot sequence — boot_picker_pty's skim-ready wait plus its initial wait_for_stable, together with the async worktree-column churn (· placeholders resolving) — can consume that whole window. By the time the stabilization helper first polls for Loading open PRs, the mocked fetch has already returned and the rows have streamed in, so the marker is gone and the wait times out at STABILIZE_TIMEOUT (30s).

Fix

Per review feedback (comment), tie the marker's lifetime to the picker's, with no timing number at all — the file's "poll, don't pick a time" rule applied to the producer side.

mock-stub gains a hold_until_parent_exit response mode: the mocked gh pr list holds until wt exits instead of sleeping a fixed delay_ms. Detection needs no platform primitive (getppid / OpenProcess):

  • While wt lives, its --prs fetch thread runs the mock with a piped stdout and blocks reading that pipe to EOF, so as long as the mock neither writes a full response nor exits, the loading marker stays.
  • When the picker aborts, wt detaches the fetch thread (drop(prs_handle)) and exits, which closes the read end of the mock's stdout pipe. The mock's next write then fails with BrokenPipe (Rust ignores SIGPIPE, so the broken write surfaces as an Err rather than killing the process), and the mock exits.

Polling for that write error is a causal parent-death signal — release is tied to wt exiting, not to a timer — and it behaves identically on Unix and Windows, so there's no platform-specific FFI to get right. The one-byte probes written into stdout are never observed (this mode's contract is that the parent aborts without parsing the response, and the fetch thread drains the pipe until process exit). A 60s poll cap bounds a stuck orphan if the pipe somehow never breaks.

This removes the race the earlier 30s bump only widened: the marker is present for the entire window the poll could observe it, on any boot latency, and no orphaned sleeper is left behind — the mock exits the moment the picker does.

Testing

cargo test --test integration --features shell-integration-tests test_switch_picker_prs_shows_loading_marker passes in ~1.3s, confirming the capture-and-abort path stays fast and the mock releases promptly on wt exit. The sibling --prs tests (test_switch_picker_prs_{github,gitlab}_list, ..._rows_survive_alt_x_removal) still pass through the refactored mock_forge_env helper. Reasoning is documented inline at the mock-stub call site and in wait_for_parent_exit.

…oot lag

test_switch_picker_prs_shows_loading_marker asserts the transient
"Loading open PRs" frame is on screen while the mocked gh pr list is in
flight. The mock held it for only 3s; on a loaded Windows runner, boot
(skim-ready wait + initial wait_for_stable) plus async worktree-column
churn ate the whole window, so the stabilization helper first polled
after the rows had already streamed in and timed out at STABILIZE_TIMEOUT.

Bump the hold to 30s. It's free: exec_in_pty_capture_before_abort
snapshots and aborts the instant the marker settles, and abort does not
join the background --prs fetch (interactive teardown ends it with
drop(prs_handle)), so the picker still exits in ~1s. The orphaned mock's
gh subprocess is read-only and its tempdir file is cleaned best-effort.
@max-sixty

Copy link
Copy Markdown
Owner

can we poll here? I thought we always polled rather than picked a specific time, or at least have this guidance in our skills?

@worktrunk-bot

Copy link
Copy Markdown
Collaborator Author

Good catch — you're right to hold this against the "poll, don't pick a time" rule in tests/CLAUDE.md ("producer sleeps to line up with a deadline … every extension just makes the race wider, not correct"). Let me split what's already polling from what isn't:

The test does poll. exec_in_pty_capture_before_abort gates on Some("Loading open PRs") via send_input_awaiting_contentwait_for_stable_until, so it captures the frame the instant the marker appears and aborts — it never waits out the delay. The 30s isn't a fixed test-side sleep; it's how long the mocked gh pr list holds the transient state on screen.

What's still time-based is exactly what you're flagging: the mock's hold has to outlast worst-case picker boot, or the marker clears before the poll starts (that's the 3s Windows flake). Bumping to 30s makes that margin huge — boot is seconds, and 30s matches the file's STABILIZE_TIMEOUT ceiling, so the poll effectively can't outlive the hold — but it's still a picked number vs. boot latency, not a structural guarantee. So: defensible, not race-free.

The structurally race-free version is to make the marker live exactly as long as the picker does, with no number at all: teach mock-stub to hold the response until its parent wt exits (parent-death poll) instead of sleeping a fixed delay_ms. Then abort → wt exits → mock releases; the marker is present for the entire lifetime the poll could observe, on any boot latency, and it also drops the orphaned 30s sleeper this leaves behind.

The cost is that mock-stub is currently dependency-free, so this adds a small platform primitive (getppid on Unix, a parent-handle wait on Windows) — and the Windows path is the one that flakes, so I can't exercise it from Linux CI without care. Two ways forward:

  1. I implement the hold-until-parent-exits mode in mock-stub and switch this test to it (the real fix your question points at), or
  2. keep the bounded 30s hold and I add a short note to tests/CLAUDE.md documenting this specific transient-frame-mock pattern and why the hold must exceed worst-case boot (your "at least have this guidance in our skills").

I lean toward (1) since it's the honest answer to "can we poll here"; happy to do (2) if you'd rather not grow mock-stub's surface. Which do you prefer?

@max-sixty

Copy link
Copy Markdown
Owner

let's try it

Replace the fixed 30s mock hold with a parent-death poll so the transient
"Loading open PRs" frame lives exactly as long as the picker does, on any
hardware — the honest answer to "can we poll here" (the file's "poll, don't
pick a time" rule).

mock-stub gains a `hold_until_parent_exit` response mode: the mocked
`gh pr list` blocks until `wt` exits instead of sleeping. Detection needs no
platform primitive (`getppid`/`OpenProcess`) — while `wt` lives, its fetch
thread blocks reading the mock's piped stdout; when `wt` aborts the picker
(`drop(prs_handle)`) and exits, the pipe's read end closes and the mock's next
write returns BrokenPipe. Rust ignores SIGPIPE, so that surfaces as an Err, and
the behavior is identical on Unix and Windows. The one-byte probes are never
observed (the parent aborts without parsing the response), and a 60s poll cap
bounds a stuck orphan.

This removes the race the 30s bump only widened: no fixed hold to outguess
worst-case boot, and no orphaned sleeper left behind — the mock exits the moment
the picker does. Test finishes in ~1.3s.
@worktrunk-bot worktrunk-bot changed the title test(picker): hold --prs loading marker long enough to beat Windows boot lag test(picker): release --prs loading marker on parent exit, not a timer Jul 21, 2026
@worktrunk-bot

Copy link
Copy Markdown
Collaborator Author

Done — went with option (1), the parent-death poll, in 568163f.

mock-stub gained a hold_until_parent_exit mode: the mocked gh pr list holds until wt exits instead of sleeping a fixed delay_ms, so the Loading open PRs… marker lives exactly as long as the picker does. No timing number, and it drops the orphaned sleeper the 30s hold left behind.

The one thing I changed from what I floated above: no getppid/OpenProcess platform primitive. While wt lives, its --prs fetch thread blocks reading the mock's piped stdout to EOF; when the picker aborts (drop(prs_handle)) and wt exits, the pipe's read end closes and the mock's next write returns BrokenPipe (Rust ignores SIGPIPE, so it's an Err, not a kill). Polling that write error is the same causal parent-death signal, but it's pure std and behaves identically on Unix and Windows — so there's no platform-specific FFI to get right, and the Windows leg that was flaking is exercised by the same code the Linux leg runs. Windows CI on this push is the real confirmation.

Test finishes in ~1.3s; the three sibling --prs tests still pass through the refactored mock_forge_env. Updated the PR title/description to match the new approach.

@max-sixty
max-sixty merged commit 2036039 into main Jul 25, 2026
37 checks passed
@max-sixty
max-sixty deleted the fix/flaky-prs-loading-marker branch July 25, 2026 20:42
max-sixty added a commit that referenced this pull request Jul 27, 2026
…#3608)

## What prompted this

Getting #3605 green ran into codecov reporting a `base_commit` three
commits
older than the real merge-base. This audits whether our config causes
that.

## The cause

Codecov picks a PR's base by walking back to the newest ancestor that
has a
coverage report. It used the real merge-base for PRs #3480, #3532 and
#3602,
and a stale one for #3603 and #3605. The difference is whether the
merge-base
uploaded a report. **29 of the last 40 main commits did not.**

`ci` had one concurrency group for main pushes, and GitHub cancels the
*pending* run in a group whenever a newer one joins, even with
`cancel-in-progress: false`. So the question is how long a run holds the
group,
and a run isn't done until its slowest job is:

| job | duration on main |
|-----|------------------|
| `fast-checks` | 2 min |
| `code-coverage` | 3-4 min |
| `test (windows)` | 11 min |
| `collect affected coverage (windows)` | 110-129 min |

Each main run held the group for ~2 hours, so nearly every subsequent
main push
was cancelled while queued, taking the 4-minute coverage job with it.
Every
cancelled main run's `updated_at` lands within a second of the next
push's
`created_at`.

The 2 hours is real work, not queue: 2-5s from `created_at` to
`started_at`,
then 108 minutes inside `cargo affected collect` — 4181 tests under
`-C instrument-coverage` with a per-test LLVM profile, ~5 GB of profraw.

## The fix: one workflow per cadence

The three groups of jobs have incompatible needs, and one group was
serving all
of them.

| workflow | cadence on main | why |
|----------|-----------------|-----|
| `ci` | every commit, ~11 min | required gate + fast checks |
| `coverage` | every commit, keyed per-sha | a skipped upload leaves
later PRs on a stale base |
| `affected` | sampled, ~2 h | a DB a few commits old still anchors a
correct superset |

`affected` keeps exactly the grouping it has today, so its sampling is
unchanged and deliberate. It just no longer drags the other two along.

### Scope of the impact

The posted `codecov/patch` check scopes to the PR's own GitHub diff, so
a stale
base did **not** score PRs against other people's lines. On #3605 the
posted
91.66% is exactly `github.rs`'s 11/12, while the stale-base compare
object
reported 64/65 across 13 files. What a stale base costs:

- `codecov/project` reports "compared to \<stale sha\>"
- the patch `auto` target is the stale base's project coverage (0.02pp
here)
- the compare API object widens to `base..head`, which is what made the
  investigation look like silence

Separately, `test`/`lint`/`fast-checks` also stopped completing on main.
Nothing
load-bearing rode on that (they already ran on the PR), but it left
`tend-ci-fix` with nothing to watch, since it doesn't fire on cancelled
runs.

## Two smaller fixes

- `ignore: "**/tests/**"` compiles to `.*/tests/.*` (confirmed against
codecov's validator), which needs a leading directory and so never
matched
`tests/` itself. Inert today since `cargo llvm-cov` reports only `src/`
(verified against a downloaded `cobertura.xml`), but now correct if that
  changes. Now `tests/**`.
- `fail_ci_if_error` gated on `github.repository_owner`, which is the
*base*
repo's owner on a fork PR too, so the soft-fail its comment describes
never
  applied. It keys off the head repo now.

## Docs

The API behaviour was ours to misuse, not codecov's to explain. Three
traps,
all confirmed against the live API:

- `file_report/<path>/` 404s with `coverage info not found` because the
route
swallows the trailing slash into the path. Without it the endpoint
returns
  `line_coverage`.
- `?pullid=N` always compares the PR's **current** head. `?base=&head=`
asks
  about an earlier commit.
- the compare response has no `patch_totals` key, and `.name` is
`{base, head}` rather than a string, so a filename lookup silently
matches
  nothing.

A working recipe already existed in `running-tend`, but that skill is
scoped to
CI. `tests/CLAUDE.md` owns coverage investigation, so the queries go
there and
`running-tend` points at them instead of keeping a second copy.

Re-running the corrected query against #3605's failing commit reproduces
the
miss exactly: `src/git/remote_ref/github.rs:164`, the `gh repo
set-default`
hint, matching what the session eventually found by hand.

## This PR demonstrates it

It changes no Rust at all, only YAML and markdown. Codecov still
reported a
**10-file, 111-line patch** on its first commit, because it based the
comparison on `203603909` rather than the real merge-base `32f380a27`.
Every
main commit in between has no report:

| commit | ci run | report |
|--------|--------|--------|
| `32f380a27` | queued | no |
| `9645e3e13` | cancelled | no |
| `bcd1ffdfd` | cancelled | no |
| `8865f20ab` | cancelled | no |

Every one of those 111 patch lines belongs to somebody else's merged
commit. It
passed at 100% only because those commits are well covered.

> _This was written by Claude Code on behalf of @max-sixty_

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated-fix Automated CI fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants