Skip to content

test(patterndetector): key fixture polarity off golden.json (#497)#514

Merged
trilamsr merged 1 commit into
mainfrom
fix/497-disk-pressure-fixture
Jun 4, 2026
Merged

test(patterndetector): key fixture polarity off golden.json (#497)#514
trilamsr merged 1 commit into
mainfrom
fix/497-disk-pressure-fixture

Conversation

@trilamsr

@trilamsr trilamsr commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Closes #497.

Root cause

TestPatternDetector_NegativeFixturesEmitNoVerdicts enumerated every
non-canonical subdir under module/pkg/replay/pod_evicted/ as a
negative fixture (filter: if f.Name == \"canonical\" { continue }).
PR #484 introduced _real_world/ as a slot for anonymized,
operator-shaped positive captures (see
module/pkg/replay/pod_evicted/_real_world/README.md — "contributed
fixtures land golden verdicts alongside"). The first such fixture,
synthetic-2026-06-multi-rank-disk-pressure/, ships a 3-verdict
golden.json (2 full + 1 partial). The detector was correct; the
test's negative-set definition was wrong.

Two symptom-only fixes were possible (skip _real_world/* by path,
or add a manifest-level positive: bool), but both leave the inverse
drift hole: a positive fixture demoted to silently-skipped would
still pass.

Fix

Replace path-based filtering with golden-driven dispatch:

  • A fixture's own golden.json declares its polarity — empty []
    means "detector must emit nothing", non-empty means "detector
    must emit exactly this".
  • Negative test (NegativeFixturesEmitNoVerdicts): skips fixtures
    whose golden is non-empty; remaining set asserts zero verdicts
    (unchanged contract).
  • New positive test (PositiveFixturesMatchGolden): non-canonical
    fixtures with non-empty goldens must round-trip to that exact
    verdict slice. Closes the inverse-drift hole.

Self-aligning: future contributions under any group land in the
correct lane based on what their golden declares, not which subdir
they live in.

Verification

  • cd module && GOWORK=off go test -run 'TestPatternDetector_NegativeFixturesEmitNoVerdicts|TestPatternDetector_PositiveFixturesMatchGolden' -v ./processor/patterndetectorprocessor/ — green (3 negatives + 1 positive subtest).
  • cd module && GOWORK=off go test ./... — green, no new failures.
  • make lint — 0 issues.
  • Pre-push hooks (go vet, go mod verify, attribute-namespace-check, no-autoupdate-check) — green.

Test plan

  • Reproduced RED on origin/main before edit.
  • Both new test bodies run green locally.
  • Full module test suite green.
  • make lint green.
fix(test): pattern-detector replay test now keys fixture polarity off `golden.json` content rather than directory name, so positive `_real_world/` captures land in the positive-assertion lane and operator contributions plug in without test-code edits.

The negative-fixture sweep iterated every non-canonical fixture
and asserted zero verdicts, so when _real_world/ (which holds
positive operator-shaped captures, per its README) gained
synthetic-2026-06-multi-rank-disk-pressure, the test went red on
correct detector behaviour.

Replace path-based filtering with golden.json-keyed dispatch: a
fixture's own golden declares its polarity (empty [] = negative,
non-empty = positive). Negative test now skips fixtures whose
golden is non-empty; new positive test asserts the golden matches
verbatim — closes the inverse drift (positive fixture silently
demoted to a no-op) the original test could not catch.

Self-aligning: future contributions under any group land in the
correct lane based on what their golden declares, not which
subdir they live in.

Signed-off-by: Tri Lam <tree@lumalabs.ai>
@trilamsr trilamsr enabled auto-merge (squash) June 4, 2026 00:34
@trilamsr trilamsr merged commit ee0c14f into main Jun 4, 2026
12 checks passed
@trilamsr trilamsr deleted the fix/497-disk-pressure-fixture branch June 4, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: synthetic-2026-06-multi-rank-disk-pressure fixture mis-labelled as negative

1 participant