Skip to content

docs(#4892): publish the Bolt driver compatibility matrix (page + badge)#5133

Merged
robfrank merged 16 commits into
mainfrom
feat/4892-bolt-compat-matrix-publish
Jul 8, 2026
Merged

docs(#4892): publish the Bolt driver compatibility matrix (page + badge)#5133
robfrank merged 16 commits into
mainfrom
feat/4892-bolt-compat-matrix-publish

Conversation

@robfrank

@robfrank robfrank commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Closes #4892. Part of epic #4882 (Bolt Driver Compatibility Certification), Group D - publication. This completes Group D, so #4882 can be closed once this merges.

What

Turns the machine-readable matrix that #4891 / #5115 already produces into the human-facing certification artifact the epic exists to deliver: a committed compatibility page and a README badge, both auto-generated from CI.

  • bolt/conformance/tools/render_matrix.py (+ test_render_matrix.py, 23 unit tests) - a pure-stdlib + PyYAML renderer that reads the nightly bolt-compat-matrix.json, spec.yaml (scenario metadata), and driver-versions.md (the column set) and emits:
    • bolt/conformance/COMPATIBILITY.md - the matrix page: every scenario (rows, grouped by the 9 feature areas) x every driver language x pinned version (15 columns), with a legend, per-scenario known_limitation footnotes, and a "Last verified" timestamp + source-run link.
    • bolt/conformance/badge.json - a shields.io endpoint (bolt drivers: green all passing / yellow partial / red N failing).
  • README badge - one shields.io endpoint badge in the existing badge row, linking to the page.
  • bolt-nightly.yml gains a publish-matrix job that regenerates the page/badge from the nightly artifact and commits them to main ([skip ci]; a nightly freshness-stamp commit, plus real result changes).
  • bolt-conformance-spec.yml runs the new renderer unit tests.

How it works

  • Two status vocabularies are kept distinct: the matrix cell runtime status (pass/fail/skip) and each scenario's spec.yaml current_status (passing/expected-fail/not-applicable/unverified). Cells resolve by combining them.
  • AC4 (non-passing cells link to their issue): a fail/expected-fail cell links the scenario's tracking_issue, falling back to the nightly bolt-compat-regression issue.
  • Badge honesty: not-applicable scenarios (today only ERR-003) are excluded from the denominator, so the badge stays green; it only goes yellow on an expected-fail and red on any real fail / missing / empty / unexpected cell.
  • Fallback / bootstrap: with no matrix artifact the renderer draws the baseline from spec.yaml current_status. That is how the committed initial page is produced (today: all-green, 5/5 passing). In the nightly, a missing artifact (crashed merge-matrix) is a no-op - it leaves the last good page in place rather than publishing a false-green, while report-regression still alerts. The push rebase-retries so a concurrent commit to main cannot red the run.

Verification

  • test_render_matrix.py: 23 tests (cell precedence, badge thresholds incl. not-applicable exclusion, fallback baseline, issue-link normalization, unavailable-column wiring, area-drift guard, pristine output), wired into bolt-conformance-spec.yml.
  • End-to-end dry-run: junit_to_matrix.py -> merge_matrix.py -> render_matrix.py composes; the committed page/badge are byte-reproducible from the renderer.
  • actionlint clean on both edited workflows.

🤖 Generated with Claude Code

@mergify

mergify Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the publication of ArcadeDB's Bolt driver compatibility matrix by adding a Python rendering script, comprehensive unit tests, a README badge, and CI workflow integration. The review feedback focuses on enhancing the robustness of the Python script by introducing defensive programming guards—such as fallback empty dictionaries and safe nested lookups—to prevent potential runtime crashes when parsing empty or malformed YAML and JSON inputs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread bolt/conformance/tools/render_matrix.py Outdated
Comment on lines +39 to +41
spec = yaml.safe_load(fh)
scenarios = []
for entry in spec.get("scenarios", []):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If spec.yaml is empty or invalid, yaml.safe_load(fh) can return None. Calling .get() on None will raise an AttributeError. Adding a fallback empty dictionary ensures robust execution.

Suggested change
spec = yaml.safe_load(fh)
scenarios = []
for entry in spec.get("scenarios", []):
spec = yaml.safe_load(fh) or {}
scenarios = []
for entry in spec.get("scenarios", []):
References
  1. Enforce defensive programming: ensure appropriate null/nil/None checks or other language-idiomatic guards exist before object property accesses to handle invalid inputs or states safely.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in b6199dc: spec = yaml.safe_load(fh) or {}, so an empty/invalid spec.yaml yields [] instead of raising AttributeError. New test loads an empty spec file and asserts load_scenarios returns [].

Comment thread bolt/conformance/tools/render_matrix.py Outdated
Comment on lines +113 to +114
runtime = (matrix.get("scenarios", {}).get(scenario["id"], {})
.get(column.language, {}).get(column.version))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If any intermediate key in the nested matrix dictionary is explicitly null in the JSON, the lookup will return None and subsequent .get() calls will raise an AttributeError. Using a safer nested lookup pattern prevents potential runtime crashes.

Suggested change
runtime = (matrix.get("scenarios", {}).get(scenario["id"], {})
.get(column.language, {}).get(column.version))
runtime = (((matrix.get("scenarios") or {}).get(scenario["id"]) or {})
.get(column.language) or {}).get(column.version)
References
  1. Enforce defensive programming: ensure appropriate null/nil/None checks or other language-idiomatic guards exist before object property accesses to handle invalid inputs or states safely.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in b6199dc with the or {} idiom guarding each level. The matrix JSON is produced by our own merge_matrix.py, which never emits null intermediates, but the guard is cheap and a new test now feeds an explicit-null intermediate ({"scenarios": {"TYPE-011": None}}) to confirm resolve_cell returns the neutral no-result glyph rather than raising.

Comment thread bolt/conformance/tools/render_matrix.py Outdated
Comment on lines +138 to +147
languages = len(matrix.get("languages", [])) or 5
if matrix.get("has_failures"):
fails = 0
for langs in matrix.get("scenarios", {}).values():
for versions in langs.values():
fails += sum(1 for status in versions.values()
if status == "fail")
fails += len(matrix.get("missing_cells", []))
fails += len(matrix.get("empty_cells", []))
fails += len(matrix.get("unexpected_cells", []))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If any of the expected keys (languages, scenarios, missing_cells, empty_cells, unexpected_cells) are explicitly null in the JSON, calling len() or .values() on them will raise a TypeError or AttributeError. Adding fallback guards ensures the badge computation is robust against partially populated or malformed JSON inputs.

Suggested change
languages = len(matrix.get("languages", [])) or 5
if matrix.get("has_failures"):
fails = 0
for langs in matrix.get("scenarios", {}).values():
for versions in langs.values():
fails += sum(1 for status in versions.values()
if status == "fail")
fails += len(matrix.get("missing_cells", []))
fails += len(matrix.get("empty_cells", []))
fails += len(matrix.get("unexpected_cells", []))
languages = len(matrix.get("languages") or []) or 5
if matrix.get("has_failures"):
fails = 0
for langs in (matrix.get("scenarios") or {}).values():
for versions in (langs or {}).values():
fails += sum(1 for status in (versions or {}).values()
if status == "fail")
fails += len(matrix.get("missing_cells") or [])
fails += len(matrix.get("empty_cells") or [])
fails += len(matrix.get("unexpected_cells") or [])
References
  1. Enforce defensive programming: ensure appropriate null/nil/None checks or other language-idiomatic guards exist before object property accesses to handle invalid inputs or states safely.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in b6199dc: languages/scenarios/missing_cells/empty_cells/unexpected_cells are all guarded with or {} / or [], and the nested .values() iterations guard langs/versions too. New test passes an all-null-valued matrix with has_failures: true and asserts a red badge is returned without raising.

@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review: Publish the Bolt driver compatibility matrix

Nice, self-contained piece of work. The renderer is pure-stdlib + PyYAML, well-factored into small pure functions, and the 23 unit tests genuinely exercise the interesting logic (cell precedence, badge thresholds, not-applicable exclusion, fallback baseline, area-drift guard). The two-vocabulary design (runtime pass/fail/skip vs spec current_status) is cleanly separated, and the no-op-on-missing-artifact behavior is the right call to avoid publishing a false green. Overall this is close to mergeable; comments below are mostly minor.

Correctness / consistency

  1. Badge vs page disagree on missing/empty cells. compute_badge folds missing_cells, empty_cells, and unexpected_cells into the red "N failing" count (render_matrix.py:145-147), but resolve_cell renders those same cells as a neutral middot ("not reported", render_matrix.py:111-112). So a red "3 failing" badge can link to a page showing zero fail marks and only a few dots, which reads as contradictory. Consider a distinct glyph for missing/unavailable cells, or wording the badge to distinguish "failing" from "no result". At minimum worth a sentence in the legend.

  2. "Last verified" timestamp defeats the "only when changed" guard. The page embeds date -u down to the minute (bolt-nightly.yml:497), so git diff --quiet (bolt-nightly.yml:500) will essentially always see a diff and commit to main every night, even when matrix content is identical. The PR description says it commits "only when changed"; in practice it is a nightly [skip ci] commit regardless. Not a bug, but if avoiding churn was the intent, the timestamp would need to be excluded from change-detection.

  3. not-applicable scenarios are not excluded from the badge fail count. The fail loop (render_matrix.py:141-144) counts any cell with status == "fail" across all scenarios. resolve_cell correctly forces ERR-003 to the dash glyph on the page regardless of runtime, but if a not-applicable scenario ever produced a fail cell in the matrix JSON, the badge would count it while the page hides it - another badge/page divergence. Low likelihood today, but the asymmetry is worth a guard.

Robustness

  1. Rebase-retry can wedge on a conflict. In publish-matrix (bolt-nightly.yml:508-515), if git pull --rebase origin main hits a conflict on the generated files (a concurrent human edit on main), the rebase halts mid-way; the next iteration push fails and the following pull --rebase errors with "rebase in progress", so all attempts burn without recovering. It does ultimately exit 1 (fails safe, alarm still fires via report-regression), but consider git rebase --abort before retrying, or git pull --rebase -X theirs since the generated files are always regenerable. The workflow concurrency group makes nightly-vs-nightly collisions impossible, so this only bites on concurrent human commits - low frequency, but currently unrecoverable within the job.

Test coverage gaps (minor)

  1. applicable_driver_versions != "all" (render_matrix.py:107-108) is entirely untested - every scenario is "all" today. That branch does a substring in check against column.band; if the field is ever a comma-joined string rather than a list, substring matches could yield false positives. A test pinning the expected container type would lock the contract.
  2. issue_url with a full URL and the standalone regression_url fallback are not directly covered; badge counting is tested for fail + missing_cells but not empty_cells/unexpected_cells.

Nits

  1. The green badge message {languages}/{languages} passing (render_matrix.py:151) is always N/N where N is the driver-language count, not scenarios - "5/5 passing" could be misread as 5 scenarios. Consider "5/5 drivers" or "all drivers passing".
  2. pip install -r requirements.txt (bolt-nightly.yml:479) is unpinned (pyyaml>=6.0); given this job has contents: write to main, a hash-pinned/constrained install would tighten the supply-chain surface, consistent with the SHA-pinned actions elsewhere in the file.

Security

No injection surface found: scenario text/footnotes render from the in-repo spec.yaml, and the downloaded artifact only contributes status strings and cell-name sets (counted or used for glyph selection, never interpolated as markdown). run-url/repo/timestamp all come from trusted GitHub context. contents: write scope is appropriate and confined to the publish job.

Good work overall - the design doc and plan are thorough and the fallback/no-op reasoning is sound.

Reviewed with Claude Code.

@codacy-production

codacy-production Bot commented Jul 8, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 128 complexity

Metric Results
Complexity 128

View in Codacy

🟢 Coverage ∅ diff coverage

Metric Results
Coverage variation Report missing for 65dddf21
Diff coverage diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (65dddf2) Report Missing Report Missing Report Missing
Head commit (a5616c9) 168116 114254 67.96%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#5133) 0 0 ∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@robfrank

robfrank commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Review response (cycle 1) - all addressed in b6199dc

1. Badge vs page divergence on missing/empty cells - Fixed. The page now renders a ## Coverage gaps section listing missing / empty / unexpected cells whenever the matrix has any, so a red "N failing" badge is explained by visible content rather than bare dots. The legend also notes that a · in a listed column means "no result for that driver:version". Badge count and page now agree.

2. Timestamp defeats "only when changed" - Fixed. The publish step now detects changes with git diff --quiet -I'^\*\*Last verified:\*\*'; a run whose matrix content is identical (only the timestamp/run-link line differs) reverts the regenerated files and skips the commit, so there is no nightly no-op commit to main.

3. not-applicable not excluded from the badge fail count - Fixed. compute_badge now builds the set of not-applicable scenario ids and skips them in the fail loop, matching the page dash treatment. New test asserts a not-applicable fail cell contributes 0 to the count.

4. Rebase-retry could wedge - Fixed. The retry now runs git pull --rebase -X theirs origin main || git rebase --abort || true: the generated files are always regenerable so our fresh render wins on conflict, and the abort guard prevents a stuck "rebase in progress".

5 & 6. Test coverage gaps - Added tests for the applicable_driver_versions != "all" band-restriction branch (pinning the list container form), issue_url with a full URL, the standalone regression_url, and badge counting of empty_cells / unexpected_cells. Suite is now 32 tests.

7. Green badge "N/N passing" ambiguity - Green message changed to "all passing" (no count to misread as scenario count).

8. Unpinned pyyaml in a contents:write job - Pinned pyyaml==6.0.2 in bolt/conformance/requirements.txt.

Gemini's three defensive null-guard suggestions (empty-YAML, null-valued-JSON) are also in this batch. All 32 tests pass with pristine output; actionlint clean on the workflow.

@@ -1 +1 @@
pyyaml>=6.0
pyyaml==6.0.2
@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review: publish the Bolt driver compatibility matrix

Overall this is a clean, well-structured change. The renderer is pure-stdlib + PyYAML, the two status vocabularies (runtime pass/fail/skip vs spec current_status) are kept properly distinct, the null-defensive (... or {}).get(...) chains are good, and the test suite is thorough (cell precedence, badge thresholds, fallback baseline, issue-link normalization, coverage gaps). docs/superpowers/ is an established location (60 existing files), so committing the plan/spec there is consistent. Nice work.

1. (Primary) The only-if-changed guard suppresses the pending -> verified transition; the page can stay "pending first nightly run" forever

In bolt-nightly.yml the change guard uses git diff --quiet -I with the regex matching the Last verified: line. The committed baseline freshness line is **Last verified:** pending first nightly run (baseline from spec.yaml current_status). and the rendered real line is **Last verified:** {timestamp} ([run](...)). Both start with **Last verified:**, so both match the ignore regex.

When the first real nightly is all-green, render_page produces byte-identical tables to the fallback baseline (fallback baseline-pass and matrix pass both render the same glyph, and with no coverage gaps there is no extra section). The ONLY differing line is the Last verified line, which -I ignores. So git diff --quiet reports no change, the script runs git checkout -- and skips the commit.

Consequences:

  • The bootstrap placeholder "pending first nightly run" is never replaced while the matrix stays all-green. The published certification artifact perpetually reads as if it was never verified against real drivers, even after many successful nightlies, which undermines the point of the page.
  • More generally, the Last verified: timestamp/run-link never refreshes on any unchanged-green night, so the freshness signal is defeated by its own dedup.

The intent (avoid a daily no-op commit) is reasonable, but the guard cannot distinguish "placeholder -> first real verification" from "pure timestamp bump." Consider letting the transition through, e.g. only ignore timestamp deltas once the page is already in the verified form, or refresh the timestamp on a lower cadence rather than never.

2. (Operational) publish-matrix pushes directly to main

The job commits and git push origin HEAD:main with the default GITHUB_TOKEN (contents: write). If main has branch protection requiring PRs/reviews or status checks, this push is rejected, the rebase-retry loop exhausts all 3 attempts and exits 1, failing the nightly every night. Worth confirming the protection rules on main allow the actions bot to push directly (or route the update through a PR). The skip-ci and -X theirs rebase handling are correct otherwise.

3. (Minor) Badge/page can disagree, and a red badge can read "0 failing"

  • compute_badge counts fail cells without applying each scenario applicable_driver_versions band restriction, whereas resolve_cell renders out-of-band cells as not-applicable. If a matrix ever reports a fail for an out-of-band column, the page shows not-applicable but the badge counts it as failing. No scenario uses band restrictions today (all "all"), so it is latent, but worth a guard when that feature is first used.
  • Because has_failures is forced true by any missing/empty/unexpected cell (correct), but not-applicable scenarios are excluded from the fail count, a not-applicable scenario emitting a fail would yield a red badge reading "0 failing". Should not happen in the pipeline, but the state is confusing.

4. (Minor) No test guards the committed page/badge against renderer drift

Nothing asserts that the committed COMPATIBILITY.md/badge.json are byte-reproducible from the current renderer + spec.yaml. A small fallback-mode regeneration test comparing against the committed files would catch accidental formatting drift (and would have surfaced issue 1 above).

None of these block core functionality; issue 1 is the one I would fix before merge since it directly affects what the published artifact says.

@robfrank

robfrank commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Review response (cycle 2) - fixed in f9ff113

1. (Primary) only-if-changed guard suppressed the pending -> verified transition - Correct, and it was a regression from the cycle-1 change-detection I added. The git diff -I ignored both the placeholder and the real Last verified: line, so an all-green first nightly (byte-identical tables) read as "no change" and the page would have stayed "pending first nightly run" forever. Reverted to a plain git diff --quiet. For a certification-freshness artifact the nightly timestamp refresh is the signal rather than noise, so the job now commits on every green night, transitions pending -> verified, and keeps Last verified current. This supersedes the cycle-1 churn optimization.

2. (Operational) publish-matrix pushes directly to main - main is currently unprotected (no required reviews/checks) and mergify[bot] already pushes [skip ci] commits to it directly, so the push works today. Added insurance anyway: after the retry loop, a persistent push failure now emits a ::warning:: and exits 0 instead of exit 1, so future branch protection (or a contended push) cannot red the nightly. The driver-compatibility signal is unaffected and report-regression still alarms on real red cells.

3. (Minor) badge/page divergence on band-restriction / "0 failing" red - Both require a pipeline-impossible state: a not-applicable or out-of-band scenario emitting a fail. merge_matrix only sets has_failures from a real fail/missing/empty/unexpected, and every scenario is applicable_driver_versions: all today, so neither can arise from the real pipeline. Left as a documented latent item to guard if band restrictions are ever introduced (that would also require passing the column set into compute_badge).

4. (Minor) no test guards the committed page/badge against renderer drift - Added CommittedArtifactsTest: it regenerates the page + badge in fallback mode and asserts byte-equality with the committed COMPATIBILITY.md/badge.json, so a renderer or spec change that was not re-published fails the suite. Suite is now 33 tests, all passing with pristine output; actionlint clean.

@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review: Bolt driver compatibility matrix publication (#5133)

Reviewed the renderer, tests, and both workflow changes. Overall this is clean, well-structured work: pure stdlib + PyYAML (no new heavyweight deps), the two status vocabularies are kept properly distinct, resolve_cell defends against None intermediates, and the CI failure modes (missing artifact = no-op, rebase-retry on push, merge-crash still alarms via report-regression) are thought through carefully. Nice test coverage too (23 focused unit tests).

One issue I think will actually break CI, plus a few minor notes.

🔴 CommittedArtifactsTest will start failing after the first nightly publish

test_render_matrix.py::CommittedArtifactsTest.test_committed_page_and_badge_are_reproducible regenerates COMPATIBILITY.md in fallback mode (no --matrix) and asserts it is byte-for-byte equal to the committed file:

```python
rc = main(["--spec", SPEC_YAML, "--versions", DRIVER_VERSIONS_MD, ...]) # fallback, no --matrix
...
self.assertEqual(regenerated_page, committed_page, "COMPATIBILITY.md is stale; ...")
```

But the `publish-matrix` job commits a page rendered with `--matrix` and a real `--timestamp`:

```yaml
python3 bolt/conformance/tools/render_matrix.py \
--matrix bolt-compat-matrix.json \
--timestamp "$(date -u +'%Y-%m-%d %H:%M UTC')" ...
```

The matrix branch of `render_page` emits `Last verified: (run)`, whereas the fallback branch emits `Last verified: pending first nightly run ...`. So once the first nightly lands its commit on `main`, the committed `COMPATIBILITY.md` no longer matches the fallback render.

`bolt-conformance-spec.yml` runs on every push/PR touching `bolt/conformance/**` and executes this test. The nightly's own commit carries `[skip ci]`, so it slips through, but the next PR that touches anything under `bolt/conformance/` will check out the nightly-updated page and `CommittedArtifactsTest` will fail with "COMPATIBILITY.md is stale" - even though nothing is actually wrong.

Note this is guaranteed, not occasional: the `date -u` timestamp changes every night, so `git diff --quiet` in `publish-matrix` is essentially never true and a fresh non-fallback page lands on `main` every run. (That also makes the "only when changed" / commit-only-on-diff wording effectively "every night".)

Suggested fixes (pick one):

  • Have the reproducibility test compare only the stable portion of the page (strip the `Last verified:` line, or just the matrix cells) rather than the whole file; or
  • Make the assertion tolerant of the timestamp line; or
  • Only run `CommittedArtifactsTest` in the bootstrap context and drop it from the recurring conformance-spec job.

Minor notes

  • Badge message wording: the PR description says green shows `N/N passing`, but `compute_badge` emits the literal `"all passing"` (and red shows `"N failing"`, no denominator). Not a bug, just worth aligning the description with the output - or adding the `N/N` if that was the intent.
  • Direct push to `main` from CI: `publish-matrix` pushes generated files straight to `main` nightly, bypassing review/branch-protection. Reasonable for a generated artifact, and the rebase-retry + graceful `::warning::` fallback is well handled - just flagging that a daily bot commit will land on `main` indefinitely (see the timestamp point above), which some teams prefer to avoid (e.g. publish to a `gh-pages`/artifact instead).
  • `git pull --rebase -X theirs` in the retry loop: correct here since the page is fully regenerated, so favoring the replayed commit on conflict is fine - just confirming it is intentional.

Nice work on the fallback/bootstrap design and the coverage-gaps accounting. The one test-stability issue above is the only thing I would consider blocking.

@robfrank

robfrank commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Review response (cycle 3) - fixed in 6acb57e

🔴 CommittedArtifactsTest would break CI after the first nightly - Correct, and it is a self-inflicted problem from the cycle-2 drift test. That test asserts the committed COMPATIBILITY.md equals a fallback-mode render, but the publish-matrix job overwrites the committed file with a matrix-mode render (real timestamp + ([run]) link, real results). Once the first nightly lands on main, the next PR touching bolt/conformance/** would run this test and fail with a spurious "stale", exactly as you traced. The committed page is a living, nightly-maintained artifact, so there is no stable "committed == fallback render" invariant to assert. Removed the test (and its now-unused path constants); the 32 behavioral unit tests already cover the renderer, and this PR\s committed page was verified correct at generation time. Cycle-3 supersedes the cycle-2 #4 suggestion that introduced it.

Badge message wording - The code has emitted all passing (green) since cycle 1; the PR description still said N/N passing. Updated the description to match.

Direct push to main / daily bot commit - Intentional: the Last verified timestamp is the freshness signal for a certification artifact, so a nightly [skip ci] refresh commit landing on main is the design (a gh-pages/artifact target was considered and documented as a deferred non-goal in the spec). The rebase-retry + graceful ::warning:: fallback keep it from ever redding the nightly.

git pull --rebase -X theirs - Intentional: the generated files are fully regenerable, so favoring the replayed commit on conflict is correct.

32 tests pass with pristine output; actionlint clean.

@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review: publish the Bolt driver compatibility matrix (page + badge)

Overall this is a clean, well-structured, and unusually well-tested change. render_matrix.py is pure stdlib + PyYAML, the functions are small and single-purpose, the two status vocabularies are kept cleanly separate, and the 23 unit tests cover the tricky bits (cell precedence, badge thresholds, not-applicable exclusion, fallback baseline, null-safety). Actions are pinned to SHAs, permissions are scoped (contents: write only on the publish job), and output is byte-reproducible. Nice work.

A few things worth considering before merge:

1. The "Last verified" timestamp defeats the "only when changed" commit guard (main concern)

The PR describes the publish step as committing "only when changed", guarded by:

if git diff --quiet -- bolt/conformance/COMPATIBILITY.md bolt/conformance/badge.json; then
  echo "Rendered matrix is byte-identical ...; nothing to publish."

But every nightly render injects a fresh --timestamp "$(date -u ...)" (and --run-url with github.run_id) into the page:

**Last verified:** {timestamp} ([run](...))

Because that line changes on every run, git diff --quiet will essentially always report a diff, so the job commits+pushes to main every single night even when no compatibility result actually changed. The "nothing to publish" branch is effectively unreachable in the nightly path. Two options:

  • If the intent is a daily freshness stamp, that is fine but the "only when changed" description is misleading, and you will get a [skip ci] commit on main every day - worth confirming that is acceptable churn.
  • If you want to commit only on real data changes, exclude the timestamp/run-url from change detection (e.g. diff only badge.json + the table region, or render a timestamp-free body for the diff check).

2. Retry loop can waste attempts after a failed rebase

git pull --rebase -X theirs origin main || git rebase --abort || true

If the rebase fails and is aborted, the loop continues and re-runs git push, which fails the same way and burns the remaining attempts without having integrated upstream. -X theirs during rebase (favoring the replayed commit) is the correct choice for a fully-generated file, so the common case is fine; this is just about the abort path degrading to no-progress retries. Minor.

3. Badge can read red "0 failing" (contradictory) in one edge case

In compute_badge, color is driven by matrix.has_failures, but the count excludes not-applicable scenarios. If the only failing cell belongs to a not-applicable scenario (e.g. ERR-003 somehow reporting fail), you get color: red with message: "0 failing". test_not_applicable_scenario_excluded_from_fail_count even locks in the "0 failing" message. Unlikely/defensive, but a red badge saying "0 failing" is confusing - consider treating a computed count of 0 as green/partial, or asserting the invariant.

4. Unverified scenarios do not downgrade the badge

compute_badge only distinguishes has_failures / expected-fail / else-green, and the matrix is None fallback returns green "all passing" regardless of any unverified scenarios (which render as a skip glyph). No impact today (everything is passing/not-applicable), but as scenarios are added an unverified one would show a skip glyph on the page while the badge still claims "all passing". Worth deciding whether unverified should push the badge to yellow/partial.

5. No CI guard that the committed page/badge stay in sync with the renderer

MainTest.test_writes_page_and_badge_in_fallback checks fallback output shape but does not assert byte-equality against the committed COMPATIBILITY.md / badge.json. A small drift-guard test (render fallback to a temp file, compare to the committed files) would catch the committed bootstrap page silently going stale if the renderer or spec.yaml changes. Minor, but cheap insurance given the "do not edit by hand" contract.

Nits

  • Pinning pyyaml==6.0.2 (from >=6.0) is good for reproducibility.
  • resolve_cell applies the applicable_driver_versions band restriction before the matrix is None check - consistent and correct, just noting the ordering is intentional.

None of these are blockers; #1 is the one I would most want confirmed as intended before merge. Good, careful work overall.

Note: I could not execute the Python test suite in this sandbox (python invocation was blocked here), so the test-behavior comments are from reading the code rather than a live run.

@robfrank

robfrank commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Review response (cycle 4) - fixed in a5616c9

1. "Last verified" timestamp defeats "only when changed" (main concern) - The daily commit is intentional: for a certification artifact the Last verified stamp is the freshness signal, so a nightly [skip ci] refresh landing on main is the design (an earlier attempt to dedup on content instead froze the page at "pending first nightly run" forever, which a prior review round caught). The wording was the real problem - the PR description said "only when changed", which is misleading. Updated the description to describe it as a nightly freshness-stamp commit. main is unprotected and mergify[bot] already pushes [skip ci] commits there, so the churn is consistent with existing practice.

2. Retry loop can waste attempts after a failed rebase - Acknowledged, kept as-is. -X theirs auto-resolves the only realistic conflict (both sides touched the fully-generated files), so the abort path is a rare degenerate case, and when it happens the loop exhausts into the non-fatal ::warning:: (exit 0) rather than looping forever or redding the nightly. Adding smarter mid-loop recovery would be complexity for a path the fail-safe already covers.

3. Badge can read red "0 failing" - Fixed. compute_badge now only returns red when the computed count is greater than 0; if has_failures was set solely by excluded not-applicable cells (the count is 0), it falls through instead of rendering a contradictory red "0 failing". Updated the test that had locked in the old "0 failing" message, and kept a separate test proving a genuine fail with null sibling keys still reads red "1 failing".

4. Unverified scenarios do not downgrade the badge - Fixed. The yellow/partial condition now covers unverified as well as expected-fail, so "all passing" is only claimed when every scenario is passing or not-applicable. No effect today (no unverified scenarios), forward-correct as scenarios are added. Added a test.

5. Add a committed-page/badge drift test - Respectfully pushing back: this exact test existed and was removed in the previous review round. It asserted the committed COMPATIBILITY.md equals a fallback-mode render, but the publish-matrix job overwrites the committed file with a matrix-mode render (real timestamp/run-link, real results), so the invariant holds only until the first nightly and then guarantees a spurious "stale" failure on the next PR touching bolt/conformance/**. The committed page is a living, nightly-maintained artifact - there is no stable "committed == fallback render" invariant to assert. Fallback output shape stays covered by MainTest.

34 tests pass with pristine output; actionlint clean; committed page/badge are unchanged by 3 and 4 (today is all-green).

@robfrank
robfrank merged commit 3b95911 into main Jul 8, 2026
16 of 17 checks passed
@robfrank
robfrank deleted the feat/4892-bolt-compat-matrix-publish branch July 8, 2026 14:05
@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review: publish the Bolt driver compatibility matrix (page + badge)

Reviewed the renderer, its tests, the workflow wiring, and the committed page/badge. This is a clean, well-factored piece of work: a pure stdlib + PyYAML renderer with a clear split between the two status vocabularies, defensive or {}/or [] guards throughout, and strong unit coverage (~35 test methods spanning cell precedence, badge thresholds, null-container safety, fallback baseline, and issue-link normalization). The two-vocabulary design and the not-applicable-excluded-from-denominator badge logic are exactly right. The committed fallback page is consistent with spec.yaml (39 scenarios, 9 areas, ERR-003 the sole not-applicable). Nice touches: the no-op-on-missing-artifact behavior avoids publishing a false green, and the rebase-retry guards the push.

A few things worth confirming or tightening:

Operational

  • Direct push to main may hit branch protection. publish-matrix does git push origin HEAD:main with the default GITHUB_TOKEN. If main requires PRs or passing status checks, this push is rejected every night and the job goes red nightly. Please confirm the bot is allowed to push directly to main; if not, this needs an exception or a PR-based publish flow.

Correctness / consistency (low severity)

  • A runtime skip on a passing scenario is invisible to the badge. resolve_cell renders such a cell as the skip glyph with no link, and compute_badge only reacts to has_failures plus scenario-level expected-fail/unverified. So a test that silently skips when it is expected to pass shows a skip on the page while the badge stays green. Does merge_matrix set has_failures for unexpected skips? If not, consider counting skip-on-passing as a coverage gap so a silent regression cannot slip past the badge.
  • compute_badge ignores applicable_driver_versions when excluding cells. The not-applicable exclusion only checks whole-scenario current_status == not-applicable, whereas resolve_cell also renders the not-applicable glyph for out-of-band columns. An out-of-band fail would therefore count toward the badge while the page shows not-applicable. Low risk (the matrix should not emit out-of-band cells), but the two paths disagree.
  • "N failing" mixes units. The count sums per-cell fail statuses with whole-column missing/empty/unexpected gaps, so "3 failing" can mean 1 failing scenario-cell plus 2 absent driver:version columns. Cosmetic, but the number is not a pure cell count.

Nits

  • Every nightly commits even when results are unchanged. The Last-verified timestamp line changes each run, so git diff --quiet always trips and a commit lands on main nightly regardless of whether any cell changed. Intended per the PR, but it adds steady git-history noise. Consider gating the commit on the matrix data changing (e.g. keep the timestamp in a separate file, or compare only the table body) so quiet nights are true no-ops.
  • Rebase-retry cannot resolve a content conflict. Because the timestamp line changes every run, a concurrent commit touching COMPATIBILITY.md makes git pull --rebase conflict rather than fast-forward, leaving the rebase mid-flight and the loop unable to recover. The bolt-nightly concurrency group makes this unlikely, but the retry only helps against a clean non-fast-forward, not a conflict.
  • Doc vs. code drift: the design/PR text says the green badge message is "5/5 passing"; the code emits "all passing" (the better choice given 39 scenarios). Worth aligning the docs.
  • No reproducibility guard test. The PR notes the committed page/badge are byte-reproducible from the renderer, but no test asserts it. A test that regenerates in fallback mode and diffs against the committed COMPATIBILITY.md/badge.json would catch accidental hand-edits or renderer drift.

None of these block the core value here, and the ACs look satisfied. The branch-protection question is the one I would resolve before relying on the nightly publish.

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.21%. Comparing base (32b1582) to head (a5616c9).
⚠️ Report is 22 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5133      +/-   ##
============================================
+ Coverage     65.84%   66.21%   +0.37%     
- Complexity      913      916       +3     
============================================
  Files          1692     1692              
  Lines        136278   136278              
  Branches      29129    29129              
============================================
+ Hits          89736    90241     +505     
+ Misses        34336    33728     -608     
- Partials      12206    12309     +103     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bolt: publish the driver compatibility matrix

2 participants