Overview
The recently-added TestRegistryMatchesDocumentation (pkg/linters/spec_test.go:249) was introduced to close the recurring doc-sync drift gap (#40436, #45185, #46131, #46527, #46707, #46977). It bridges the code registry to the spec_test hand-list, but it does not connect either of those to the prose docs (doc.go / README.md). The result is two internally-consistent-but-mutually-disconnected clusters. A linter added to the code side but omitted from doc.go + README.md (the classic "docs lag code" failure) still passes every test in the suite.
The four doc surfaces and which test binds which
| Edge enforced |
Test |
linters.All() (registry) ↔ documentedAnalyzers() (spec_test hand-list) |
TestRegistryMatchesDocumentation (spec_test.go:249) |
doc.go bullets ↔ README.md Subpackages table |
TestDocGo_AnalyzersMatchREADME (doc_sync_test.go:54) |
doc.go header count ↔ doc.go bullet count |
TestDocGo_CountMatchesBullets (doc_sync_test.go:22) |
This forms two disjoint components:
- Cluster A (code):
registry = documentedAnalyzers()
- Cluster B (prose):
doc.go = README.md
No test compares any member of A against any member of B. documentedAnalyzers() is a hand-maintained Go slice that is asserted to be "derived from the README" (spec_test.go:83) but that derivation is never verified against the actual README text.
Concrete failure scenario
Add a 56th linter foolinter:
- Register it in
pkg/linters/registry.go (All() → 56 entries).
- Add
{"foolinter", foolinter.Analyzer} to documentedAnalyzers() in spec_test.go (required anyway to make the import compile).
- Forget to add the bullet to
doc.go and the row to README.md.
Result:
TestRegistryMatchesDocumentation: registry(56) == documentedAnalyzers(56) → PASS
TestDocGo_AnalyzersMatchREADME: doc.go(55) == README(55) → PASS
TestDocGo_CountMatchesBullets: doc.go header(55) == bullets(55) → PASS
The suite is green while doc.go/README.md silently omit foolinter — exactly the drift class these guards were added to prevent. The symmetric case (docs updated, code list not) is equally uncaught.
Secondary gap: the label field is never validated
TestRegistryMatchesDocumentation compares d.analyzer.Name (not d.label) against registry names (spec_test.go:259-260). The label strings in documentedAnalyzers() — which are supposed to mirror the README subpackage column — are therefore checked against nothing. A typo'd or stale label (e.g. panic-in-library-code vs panicinlibrarycode) would not be caught.
Recommendation
Add one test that bridges the clusters by parsing the prose surfaces and asserting set-equality against the canonical linters.All() names. Concretely, extend the suite so that:
- The set of
doc.go bullet slugs (already parsed by docBulletRe in doc_sync_test.go:64) equals the set of registered Analyzer.Names from linters.All().
- The set of README table slugs (already parsed by
readmeTableRe) equals the registered names.
documentedAnalyzers() label values equal the parsed README slugs (validating the "derived from README" claim).
Because the parsers already exist in doc_sync_test.go, this is a small additive change: reuse the two regexes, build the slug sets, and assert each against linters.All(). That collapses the two clusters into a single fully-connected invariant so any of the four surfaces drifting fails CI.
Note: the doc slug (subpackage dir name, e.g. panic-in-library-code) differs from Analyzer.Name (e.g. panicinlibrarycode) for one linter, so the bridge test needs a small name-normalization/alias map (the same one implicitly encoded by the label→Analyzer pairing in documentedAnalyzers()).
Validation checklist
Impact
Latent (all 55 surfaces are currently consistent). Severity: low-to-medium — this is a defense-in-depth gap in a guard that has already had to be reinforced six times. Closing it makes the doc-sync invariant total rather than partial.
Effort
Small — one additional test function reusing the existing docBulletRe/readmeTableRe parsers and linters.All(); no production code change.
Sergo R61 · registry delta 43→55 since R60 · new linters audited clean: appendoneelement, bytesbufferstring, ioutildeprecated, sprintfbool.
Generated by 🤖 Sergo - Serena Go Expert · age00 304.1 AIC · ⌖ 13.7 AIC · ⊞ 5.7K · ◷
Overview
The recently-added
TestRegistryMatchesDocumentation(pkg/linters/spec_test.go:249) was introduced to close the recurring doc-sync drift gap (#40436, #45185, #46131, #46527, #46707, #46977). It bridges the code registry to the spec_test hand-list, but it does not connect either of those to the prose docs (doc.go/README.md). The result is two internally-consistent-but-mutually-disconnected clusters. A linter added to the code side but omitted fromdoc.go+README.md(the classic "docs lag code" failure) still passes every test in the suite.The four doc surfaces and which test binds which
linters.All()(registry) ↔documentedAnalyzers()(spec_test hand-list)TestRegistryMatchesDocumentation(spec_test.go:249)doc.gobullets ↔README.mdSubpackages tableTestDocGo_AnalyzersMatchREADME(doc_sync_test.go:54)doc.goheader count ↔doc.gobullet countTestDocGo_CountMatchesBullets(doc_sync_test.go:22)This forms two disjoint components:
registry=documentedAnalyzers()doc.go=README.mdNo test compares any member of A against any member of B.
documentedAnalyzers()is a hand-maintained Go slice that is asserted to be "derived from the README" (spec_test.go:83) but that derivation is never verified against the actual README text.Concrete failure scenario
Add a 56th linter
foolinter:pkg/linters/registry.go(All()→ 56 entries).{"foolinter", foolinter.Analyzer}todocumentedAnalyzers()inspec_test.go(required anyway to make the import compile).doc.goand the row toREADME.md.Result:
TestRegistryMatchesDocumentation: registry(56) == documentedAnalyzers(56) → PASSTestDocGo_AnalyzersMatchREADME: doc.go(55) == README(55) → PASSTestDocGo_CountMatchesBullets: doc.go header(55) == bullets(55) → PASSThe suite is green while
doc.go/README.mdsilently omitfoolinter— exactly the drift class these guards were added to prevent. The symmetric case (docs updated, code list not) is equally uncaught.Secondary gap: the
labelfield is never validatedTestRegistryMatchesDocumentationcomparesd.analyzer.Name(notd.label) against registry names (spec_test.go:259-260). Thelabelstrings indocumentedAnalyzers()— which are supposed to mirror the README subpackage column — are therefore checked against nothing. A typo'd or stale label (e.g.panic-in-library-codevspanicinlibrarycode) would not be caught.Recommendation
Add one test that bridges the clusters by parsing the prose surfaces and asserting set-equality against the canonical
linters.All()names. Concretely, extend the suite so that:doc.gobullet slugs (already parsed bydocBulletReindoc_sync_test.go:64) equals the set of registeredAnalyzer.Names fromlinters.All().readmeTableRe) equals the registered names.documentedAnalyzers()labelvalues equal the parsed README slugs (validating the "derived from README" claim).Because the parsers already exist in
doc_sync_test.go, this is a small additive change: reuse the two regexes, build the slug sets, and assert each againstlinters.All(). That collapses the two clusters into a single fully-connected invariant so any of the four surfaces drifting fails CI.Note: the doc slug (subpackage dir name, e.g.
panic-in-library-code) differs fromAnalyzer.Name(e.g.panicinlibrarycode) for one linter, so the bridge test needs a small name-normalization/alias map (the same one implicitly encoded by thelabel→Analyzerpairing indocumentedAnalyzers()).Validation checklist
registry.go+documentedAnalyzers()but not todoc.go/README.md.doc.go/README.mdbut not to the registry.labelindocumentedAnalyzers().Impact
Latent (all 55 surfaces are currently consistent). Severity: low-to-medium — this is a defense-in-depth gap in a guard that has already had to be reinforced six times. Closing it makes the doc-sync invariant total rather than partial.
Effort
Small — one additional test function reusing the existing
docBulletRe/readmeTableReparsers andlinters.All(); no production code change.Sergo R61 · registry delta 43→55 since R60 · new linters audited clean: appendoneelement, bytesbufferstring, ioutildeprecated, sprintfbool.