docs(patterns): correlation-window semantics rationale (#367)#388
Closed
trilamsr wants to merge 1 commit into
Closed
docs(patterns): correlation-window semantics rationale (#367)#388trilamsr wants to merge 1 commit into
trilamsr wants to merge 1 commit into
Conversation
Three v1 detectors use three different correlation-window shapes: pattern #7 (dataloader_hang) one-sided look-back, pattern #11 (checkpointer_hang) asymmetric back/forward legs, pattern #13 (silent_data_corruption) symmetric job window. The shapes were chosen independently and the rationale lived only in code comments and review threads. Decision: (A) document the existing reality, do not converge. Each shape matches a distinct physical event-ordering: strict cause→symptom for #7, bidirectional log/threshold race for #11, order-free job-scoped attribution for #13. A unified asymmetric two-knob form (issue suggestion) would silently zero one leg for #7 and would not apply to #13 at all; operators would tune knobs that have no physical meaning for their pattern. Adds a "Why this correlation shape" subsection to each pattern doc and a "Correlation-window semantics" comparison table to docs/patterns/README.md cross-linking the three rationales. No code changes; no detector behavior changes. Closes #367. Signed-off-by: Tri Lam <tri@maydow.com>
Contributor
Author
This was referenced Jun 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The patterndetector ships 11 detectors with 14 time-bounded knobs. The join shape varies across patterns and the rationale lived only in code comments + PR review threads — operators tuning windows had to read source per detector. PR #346 reviewer flagged this on three named detectors; the actual scope is wider.
Root-cause finding
Five distinct join shapes exist across the 11 detectors. They are load-bearing, not bugs — each shape matches the causal physics of the signal:
effect.t - cause.t ∈ [0, window].nowor the most-recent record.Decision
Document the existing reality, do not converge. The original suggestion (unified asymmetric two-knob form) was explored and rejected: it would force
forward_window=0son the one-sided detectors (a footgun on clock skew across nodes) and would not apply to #13 at all (bounds come from the eval record itself, not an operator knob).Changes
docs/patterns/README.md— new "Correlation-window semantics" section covering all 11 detectors with the 5-shape vocabulary, per-detector predicate / anchor / default / rationale table, and a "why not converge" explanation.docs/patterns/07-dataloader-hang.md— "Why this correlation shape" subsection (one-sided lookback rationale, predicate location, default).docs/patterns/11-checkpointer-hang.md— "Why this correlation shape" subsection (asymmetric two-sided rationale, both legs explained).docs/patterns/13-silent-data-corruption.md— "Why this correlation shape" subsection (job-bounded rationale, why no*_windowknob).The README table cross-links into the three per-pattern sections via in-page anchors.
Test plan
make checkclean (gofumpt, golangci-lint 0 issues, go vet, go mod verify, attribute-namespace-check 100/100).module/pkg/patterns/*.goline numbers), defaults matchmodule/processor/patterndetectorprocessor/config.go, bound-inclusivity (>for correlation,<for freshness) confirmed by testsGOWORK=off go test ./module/pkg/patterns/... -run Windowgreen onmain.No code changes; no detector behavior changes.
Closes #367.