feat(sdk): publish Python + Go verdict-consumption SDKs#356
Merged
Conversation
added 9 commits
June 1, 2026 02:01
Adds verdict_envelope_schema_test.go pinning cut-criterion-2 contract before the artifact lands: schema loadable from docs/schemas/verdict- 1.0.0-rc1.json, every shipped Verdict round-trips through it, ten envelope-drift falsifiers reject, and $id matches the published CDN URL contract for external SDK pinning. Fails with file-not-found until the schema artifact is published. Signed-off-by: Tri Lam <tri@maydow.com>
Publishes docs/schemas/verdict-1.0.0-rc1.json — the canonical Verdict envelope every pattern detector emits. Per-pattern detectors layer pattern-specific fields on top (gpu_id, xid_code, kind, …); per-pattern testdata schemas pin those extensions. Envelope sets additionalProperties true so per-pattern fields validate without re-listing here. Closes v1-rc1 cut criterion 2: - Schema at docs/schemas/verdict-1.0.0-rc1.json, draft 2020-12, $id https://schema.tracecore.io/verdict/1.0.0-rc1.json (matches the CDN contract third-party SDKs key off). - Envelope-conformance + ten drift-falsifier rows already landed RED in the prior commit; this commit makes them GREEN. - docs/schemas/README.md documents versioning + evolution policy and cross-links the forthcoming docs/DEPRECATION.md (criterion 4). - docs/ATTRIBUTES.md cross-links the envelope from the pattern.verdict_json column. - docs/v1-rc1-cut-criteria.md flips criterion 2 from planned to shipped with a Landed pointer. Signed-off-by: Tri Lam <tri@maydow.com>
Signed-off-by: Tri Lam <tri@maydow.com>
Drives criterion 12 (verdict-consumption SDKs) for v1-rc1. Tests fail with 'no non-test Go files' until verdict.go + decode.go land. Signed-off-by: Tri Lam <tri@maydow.com>
Publishes module/sdk/verdict — the typed Go client for the v1.0-rc1 Verdict envelope. Decode validates against the embedded copy of docs/schemas/verdict-1.0.0-rc1.json; a sync test pins byte-equality with the canonical source. Per-pattern extensions ride through Verdict.Extras (envelope additionalProperties:true). Drives v1-rc1 cut criterion 12 (Go half). Signed-off-by: Tri Lam <tri@maydow.com>
Mirrors the Go SDK's TDD discipline: every envelope guard + additionalProperties+ shipped-pattern fixture is encoded here as a pytest row. Fails with ModuleNotFoundError until __init__.py + verdict.py + decode.py land. Drives criterion 12 (Python half). Signed-off-by: Tri Lam <tri@maydow.com>
Publishes python/tracecore_verdict — the pip-installable Python client for the v1.0-rc1 Verdict envelope. decode() validates against the package-data copy of docs/schemas/verdict-1.0.0-rc1.json; a sync test pins byte-equality with the canonical source. Per-pattern extensions ride through Verdict.extras. Drives v1-rc1 cut criterion 12 (Python half). Signed-off-by: Tri Lam <tri@maydow.com>
Adds sdk-python CI job (pytest python/tracecore_verdict) wired into the verify aggregator. Go SDK is already covered by verify-test via go.work resolution. Publishes docs/sdk/README.md as the SDK index + cross-links it from docs/README.md. Flips cut criterion 12 to shipped. Signed-off-by: Tri Lam <tri@maydow.com>
Signed-off-by: Tri Lam <tri@maydow.com>
Contributor
Author
|
Addressed reviewer findings: 🔴 Fixed (real bug): added 🟡 Fixed: drift tests now JSON-normalize (Go: json.Unmarshal + require.Equal; Python: json.loads + dict compare) — whitespace-only canonical reformatting won't false-positive. 🔵 Fixed: 9th SDC fixture added to both Go + Python round-trip suites. docs/sdk/README.md updated to '9 shipped pattern verdicts'. Acknowledged: future patterns still require manual fixture sync across Go + Python — tracked as code-gen opportunity, not blocking rc1. |
# Conflicts: # docs/schemas/README.md # docs/schemas/verdict-1.0.0-rc1.json # docs/v1-rc1-cut-criteria.md # module/pkg/patterns/verdict_envelope_schema_test.go
added 2 commits
June 1, 2026 13:09
Signed-off-by: Tri Lam <tri@maydow.com>
…eAI/tracecore into feat/sdk-verdict-v1-rc1
7 tasks
trilamsr
added a commit
that referenced
this pull request
Jun 1, 2026
…398) ## Summary Shipped-pattern verdict fixtures lived in three places — `module/sdk/verdict/decode_test.go` (Go SDK), `python/tracecore_verdict/test_decode.py` (Python SDK), and `module/pkg/patterns/verdict_envelope_schema_test.go` (envelope schema test) — each as a hand-rolled literal. Adding a pattern verdict at rc1 or v0.4 meant editing three places in lockstep; drift was inevitable (and we've already been bitten by SDK-schema drift on the `$id` field). Replace the three literals with **one canonical JSON file** at `docs/schemas/fixtures/shipped-patterns-v1.0.0-rc1.json`. All three test sites parametrise off it at test time. Adding a pattern verdict is now a one-file edit — drift is impossible by construction. ## Root cause PR #356 cut the v1.0-rc1 SDKs (criterion 12) by duplicating the shipped-verdict fixture set across Go SDK + Python SDK + envelope schema test. There was no canonical artifact for "the v1.0-rc1 shipped-verdict set"; each test author rewrote the list. This PR creates that artifact and converts each test site to a consumer. ## Why not code-gen? The issue title says "code-gen" but the simpler primitive is runtime-load: both languages parse JSON natively, so a generator step adds zero safety beyond `os.ReadFile`/`Path.read_text`, and runtime-load eliminates the "generated file out of sync with source" failure mode. The existing `generate-fixtures` Make target codes-gen `.pkl` binary fixtures (where parse-at-test-time wouldn't work); JSON does not need that machinery. ## CI gate (`make verdict-fixtures-check`) Wired into `make verify` (pre-push) and `make ci-full` (everything CI runs). It asserts: 1. The canonical file exists at `docs/schemas/fixtures/shipped-patterns-v1.0.0-rc1.json`. 2. Each of the three consumer test files still references the canonical path by string — a reintroduced hand-rolled literal in any site trips this guard. 3. Both SDK suites + the envelope-schema canonical-fixtures subtest round-trip the fixture set green. **Mutation-verified locally:** deleting the canonical file fails the gate; dropping the reference from any consumer fails the gate. The envelope test gains a new subtest `TestVerdictEnvelopeV1RC1_CanonicalShippedFixturesValidate` that validates every canonical fixture against the published envelope schema — closing the loop so an envelope-non-conforming fixture would fail before either SDK suite runs. ## Diff shape Net: -204 deletions / +332 insertions (the canonical JSON file is +146; the rest is the new envelope-validation subtest + Make target + README rows; over 100 lines of duplicated fixture literals deleted). ## Test plan - [x] `make verdict-fixtures-check` green - [x] `make verdict-fixtures-check` red when canonical file deleted (mutation test) - [x] `make verdict-fixtures-check` red when any consumer drops the reference (mutation test) - [x] `cd module && GOWORK=off go test ./sdk/verdict/ ./pkg/patterns/ -count=1` green - [x] `cd python/tracecore_verdict && python3 -m pytest .` green (22/22) - [x] `golangci-lint run ./module/sdk/verdict/... ./module/pkg/patterns/...` clean on touched files - [x] Pre-push hook (vet, mod-verify, attribute-namespace-check, no-autoupdate-check) clean ```release-notes sdk: shipped-pattern verdict fixtures consolidated to a single canonical file at `docs/schemas/fixtures/shipped-patterns-v1.0.0-rc1.json`; the Go SDK, Python SDK, and envelope schema tests now consume it directly. Adding a new pattern verdict is a one-file edit. `make verdict-fixtures-check` is the CI drift gate (issue #368). ``` Closes #368 Signed-off-by: Tri Lam <tri@maydow.com> Co-authored-by: Tri Lam <tri@maydow.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes v1-rc1 cut criterion 12 — publishes the Python + Go client SDKs operators bind against to decode tracecore-emitted verdicts.
module/sdk/verdict/—Decode(rawJSON []byte) (*Verdict, error)validates against the embedded v1.0-rc1 envelope schema and returns a typed Verdict with per-pattern extensions onVerdict.Extras.python/tracecore_verdict/—decode(raw_json: bytes | str) -> Verdictvalidates viajsonschemaand returns a typed dataclass with per-pattern extensions onVerdict.extras. Pip-installable astracecore-verdict.docs/schemas/verdict-1.0.0-rc1.json(Go via//go:embed, Python viapackage-data). A drift-detection test in each SDK fails the build if the copy strays from canonical.sdk-pythonjob wired into theverifyaggregator; Go SDK already runs viaverify-testthroughgo.workresolution.docs/sdk/README.mdindexes both SDKs + version policy; cross-linked fromdocs/README.md; criterion 12 flipped from ☐ to ☑.Install
Discipline
test(sdk/verdict): RED ...,test(tracecore_verdict): RED ...) precede GREEN. Tests assert: happy-path decode, schema-invalid rejection (6 envelope guards), malformed JSON rejection, per-pattern extras preserved, all 8 shipped patterns round-trip, missing-layers populated.TestEmbeddedSchemaMatchesCanonical/test_embedded_schema_matches_canonicalbyte-compares the embedded schema againstdocs/schemas/verdict-1.0.0-rc1.json. Drift fails the build with a copy-paste fix hint.module/sdk/verdict/v2,tracecore-verdict-v2) — never a silent breaking change.Test plan
go test -race ./module/sdk/verdict/...cleancd python/tracecore_verdict && pytest— 21/21 passcd python/tracecore_verdict && make test— passgo vet ./...cleanmake tidy-check/make license-check/make actionlint/make doc-checkcleanmake zizmorclean