test(patterndetector): key fixture polarity off golden.json (#497)#514
Merged
Conversation
The negative-fixture sweep iterated every non-canonical fixture and asserted zero verdicts, so when _real_world/ (which holds positive operator-shaped captures, per its README) gained synthetic-2026-06-multi-rank-disk-pressure, the test went red on correct detector behaviour. Replace path-based filtering with golden.json-keyed dispatch: a fixture's own golden declares its polarity (empty [] = negative, non-empty = positive). Negative test now skips fixtures whose golden is non-empty; new positive test asserts the golden matches verbatim — closes the inverse drift (positive fixture silently demoted to a no-op) the original test could not catch. Self-aligning: future contributions under any group land in the correct lane based on what their golden declares, not which subdir they live in. Signed-off-by: Tri Lam <tree@lumalabs.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #497.
Root cause
TestPatternDetector_NegativeFixturesEmitNoVerdictsenumerated everynon-canonical subdir under
module/pkg/replay/pod_evicted/as anegative fixture (filter:
if f.Name == \"canonical\" { continue }).PR #484 introduced
_real_world/as a slot for anonymized,operator-shaped positive captures (see
module/pkg/replay/pod_evicted/_real_world/README.md— "contributedfixtures land golden verdicts alongside"). The first such fixture,
synthetic-2026-06-multi-rank-disk-pressure/, ships a 3-verdictgolden.json(2 full + 1 partial). The detector was correct; thetest's negative-set definition was wrong.
Two symptom-only fixes were possible (skip
_real_world/*by path,or add a manifest-level
positive: bool), but both leave the inversedrift hole: a positive fixture demoted to silently-skipped would
still pass.
Fix
Replace path-based filtering with golden-driven dispatch:
golden.jsondeclares its polarity — empty[]means "detector must emit nothing", non-empty means "detector
must emit exactly this".
NegativeFixturesEmitNoVerdicts): skips fixtureswhose golden is non-empty; remaining set asserts zero verdicts
(unchanged contract).
PositiveFixturesMatchGolden): non-canonicalfixtures with non-empty goldens must round-trip to that exact
verdict slice. Closes the inverse-drift hole.
Self-aligning: future contributions under any group land in the
correct lane based on what their golden declares, not which subdir
they live in.
Verification
cd module && GOWORK=off go test -run 'TestPatternDetector_NegativeFixturesEmitNoVerdicts|TestPatternDetector_PositiveFixturesMatchGolden' -v ./processor/patterndetectorprocessor/— green (3 negatives + 1 positive subtest).cd module && GOWORK=off go test ./...— green, no new failures.make lint— 0 issues.go vet,go mod verify,attribute-namespace-check,no-autoupdate-check) — green.Test plan
origin/mainbefore edit.make lintgreen.