refactor(patterns): introduce Detector registry seam#520
Merged
Conversation
Wave-end audit flagged the patterndetectorprocessor fanout site as an unmet refactor: ConsumeLogs hand-rolled the dispatch for every shipped detector (12 today, 7 inline + 5 wrapped), so adding pattern #13 required editing the fanout, not registering a new entry. This change introduces a minimal Detector interface + Registered slice in module/pkg/patterns/ that pins the full set, plus a detectorRunners closure list in the processor that ConsumeLogs iterates. ConsumeLogs drops from ~77 lines to 12. Adding pattern #13 = one append to each list, drift-pinned by TestRegistered_PinsAllPatterns. Detector contract is deliberately metadata-only (PatternID() string) - each detector's Evaluate signature is heterogeneous (different record shapes, different verdict types), so a uniform Evaluate method on the interface would force a lossy any-typed contract. Behavior preserved: same telemetry vocabulary, same emission order, same partial-confidence gating. Only pre-existing #497 failure (synthetic-2026-06-multi-rank-disk-pressure, fixed in Lane J) remains. Closes wave-end-audit next-wave item: pattern registry seam. Signed-off-by: Tri Lam <tree@lumalabs.ai>
Signed-off-by: Tri Lam <tree@lumalabs.ai>
4 tasks
trilamsr
added a commit
that referenced
this pull request
Jun 4, 2026
## Summary Finishes the `appendVerdict[V verdictAttrer]` consolidation that Lane M (#508 / merged as #520) started. Before this change, only 7 of 12 verdict types satisfied the `verdictAttrer` interface — the other 5 (cuda_oom, checkpointer_hang, dataloader_hang, nccl_bootstrap, silent_data_corruption) bypassed the generic emitter and called the lower-level `appendVerdictRecord` directly with hand-built `verdictCommon{...}` literals + per-pattern stamp closures. ## Root cause The half-refactor wasn't an interface limitation — it was an unfinished migration. The 5 verdict types simply never got `Common()` + `PutAttrs()` methods written for them, and the per-pattern attribute-key constants stayed in the consumer (`patterndetectorprocessor`) package instead of moving to the producer (`patterns`) package alongside the keys for the other 7 detectors. The result was three-tier indirection (runner → `appendXxxVerdict` wrapper → `appendVerdictRecord`) for half the detectors and one-tier (runner → `appendVerdict`) for the other half — inconsistent enough that adding a new pattern required a coin-flip on which template to follow. ## What changed - Adds `Common()` + `PutAttrs(pcommon.Map)` methods to the 5 remaining verdict types in `module/pkg/patterns/`. - Moves the per-pattern attribute-key constants into the canonical block in `module/pkg/patterns/verdict.go` next to the keys for the other 7 — wire-format vocabulary is now entirely owned by the producer. - Replaces the 5 inline `appendXxxVerdict` wrappers with `appendVerdict[V]` call-site uses. - Inlines `appendVerdictRecord` into `appendVerdict` (single caller after the consolidation). - Drops the now-dead `verdictCommon` type alias, `putStrIfSet` helper, and orphaned `verdictAttrConfidence` / `verdictAttrPCIeAERGPUID` constants from the processor. All 12 detector emit paths now route through one generic emitter. ## Wire-format preservation Verdict wire-format is preserved bit-for-bit: `PutAttrs` methods stamp the same keys with the same values in the same order; concrete-type `json.Marshal` still honors per-verdict json tags (because `V` is concrete at the generic call site); the diagnostic kind label flows from `VerdictCommon.Kind` set by each verdict's `Common()`. ## LoC delta +266 / -369 (-103 net) across 12 files. Net deletion because 5 wrapper functions + their per-pattern key-constant blocks collapsed into method-based stamping with shared constants. ## Closes-the-loop Wave-end-audit next-wave item #6 (full `appendVerdict` consolidation). ## Test plan - [x] `cd module && go test ./...` green (full module, including pattern-detector golden + verdict-attribute tests that pin the wire format). - [x] `make lint` green (0 issues; golangci-lint catches dead constants + unused helpers). - [x] `go vet ./...` clean. - [x] Manual diff sweep confirms wire-format equivalence: each `PutAttrs` method stamps the same key/value/order as the previous inline closure. ```release-notes NONE ``` Signed-off-by: Tri Lam <tri@maydow.com> Co-authored-by: Tri Lam <tri@maydow.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave-end audit flagged the patterndetectorprocessor fanout site as an unmet refactor:
ConsumeLogshand-rolled dispatch for every shipped detector (12 today: 7 inline + 5 wrapped), so adding pattern #13 required editing the fanout body — not registering a new entry. Past the rule-of-three by 9x.This PR introduces a minimal Detector registry seam:
module/pkg/patterns/detector.go: newDetectorinterface (PatternID() string) +Registeredslice that pins all 12 detector pointers. Each*Detectorstruct gets a one-linePatternID()method.module/pkg/patterns/detector_test.go:TestRegistered_PinsAllPatterns(exact PatternID set + count),TestRegistered_UniquePatternIDs,TestRegistered_NonEmptyPatternIDs. Drift gate — accidental drops fail in CI.patterndetector.go: introducesdetectorRunners []detectorRunnerclosure list iterated byConsumeLogs.ConsumeLogsbody drops from ~77 lines to 12. Adding pattern [general] M1.6 hardening: lint + coverage + panic recovery + pipelinebuilder #13 = one append toRegistered+ one append todetectorRunners, no fanout-site edit.Design decision: metadata-only interface
The
Detectorinterface is intentionallyPatternID() stringonly — not a uniformEvaluatemethod. Each detector's Evaluate signature is intrinsically heterogeneous (different input record shapes — events+nodeConds, ncclRecs, xidRecs+events, etc. — and different verdict types). A uniform Evaluate would force a lossyany-typed contract that the typed test suite has been fighting for 12 patterns. The closure-per-detector approach keeps the typed Evaluate calls at their concrete-runner sites while letting the registry pin identity + iteration.Behavior preservation
IncVerdictwithstring(v.Confidence)(they gate on partial); the other 5 inline detectors still pass"". The 5 wrapped runners still tick inside their own helpers (unchanged).detectorRunnersis declared in the legacy emission order.emitPodEvictedandemitIBLinkFlappreserve the!emitPartialskip.Test plan
cd module && go build ./...cleancd module && go test ./pkg/patterns/green (incl. 3 new pin tests)cd module && go test ./processor/patterndetectorprocessor/green except pre-existing test: synthetic-2026-06-multi-rank-disk-pressure fixture mis-labelled as negative #497 (TestPatternDetector_NegativeFixturesEmitNoVerdicts/synthetic-2026-06-multi-rank-disk-pressure, fixed in Lane J)make lintclean (0 issues)make vet,go mod verify, attribute-namespace-check all green (pre-push hook)LoC delta
ConsumeLogsbody: 77 → 12 lines.Closes-the-loop
Closes wave-end-audit next-wave item #2 (pattern registry seam).