[ci] Reproducible-release pipeline + verification recipe (M3)#28
Merged
Conversation
Wire .github/workflows/release.yml triggered on v* tags. The workflow builds linux/amd64 twice at the same SHA, compares with diffoscope, and fails closed on any byte diff; second build asserts -trimpath is in the Makefile recipe and that SOURCE_DATE_EPOCH propagates into the embedded BuildDate string. A CycloneDX SBOM is generated from go.mod via cyclonedx-gomod and asserted non-empty. The binary is signed with cosign sign-blob (keyless / GitHub Actions OIDC, no long-lived secret) and smoke-verified against the GitHub identity regex. SLSA v1.0 in-toto provenance is produced via actions/attest-build-provenance and the emitted bundle's predicateType is asserted == https://slsa.dev/provenance/v1 with subject.digest matching the build digest. All third-party actions are pinned to commit SHAs (with the version annotated next to each). docs/reproducibility.md gives a third party the seven commands to re-verify a published release end-to-end (rebuild, diffoscope, cosign verify-blob, slsa-verifier, jq over the SBOM). The file is registered in scripts/doc-check.sh under a presence-plus-shell-syntax gate: doc-check fails if the file is missing, and runs `bash -n` on every fenced bash/sh block so a stray syntax break is caught at edit time instead of at re-verify time on a stranger's laptop. Mutation-tested: injecting `if then fi` makes the gate fail with the offending block number; restoring makes it pass. The reproducibility/signing/SBOM/provenance steps all live in release.yml, not Makefile or ci.yml — make ci stays at ~13s wall clock on this branch (well under the 60s budget per PRINCIPLES §10). docs/README.md gains a top-level index row for the new doc. Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
The previous gate exited at the sha256 mismatch, which left no diagnostic trail for triaging which bytes diverged between Build #1 and Build #2. Inverting the control flow: run diffoscope on a mismatch, capture its text report, then exit non-zero. On a match, run diffoscope --exit-code as the load-bearing assertion. Either way diffoscope output ends up in the job log. Also upload both binaries as a "failed-build-pair" artifact when the job fails — needed for offline triage when the on-runner diff isn't enough (e.g. comparing across two failed runs). Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
Diffoscope on test tag v0.0.0-m3test-2 surfaced the actual delta: two runtime/debug.BuildInfo entries differed across builds — vcs.modified flipped from false to true, and the +dirty suffix appeared in the embedded module version. Cascading: that fed a different action-ID into the Go linker, which changed NT_GNU_BUILD_ID, which changed the file hash. Root cause: Build #1 created build1/ inside the worktree and moved the binary into it. By the time Build #2 ran `go build`, the worktree contained untracked files (build1/tracecore_linux_amd64 + .sha256), so `git status --porcelain` was non-empty. `go build -buildvcs=true` (default) reads that and sets vcs.modified=true for Build #2. Fix: build each iteration into `mktemp -d` outside the source tree. The worktree stays clean; Go's VCS probe sees identical state on both runs; build IDs match; binaries match. The canonical artifact is then staged from BUILD1_DIR into ./release/ for the rest of the workflow. Failure-triage upload still grabs both builds when the gate trips. Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
Test tag #3 confirmed the reproducibility fix lands: both build digests matched (f744dd584b5e…). The step still tripped because diffoscope-minimal in ubuntu-latest's apt repo doesn't recognise --exit-code; the assertion was returning 2 (argparse error) instead of 0. The default exit status (0 = no diff, 1 = diff) carries the same signal, so just drop the flag. Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
…rage Four parallel reviews landed seven actionable changes: - Cold rebuild: both builds now use isolated $(mktemp -d) GOCACHE dirs so build #2 can't pass by replaying build #1's cached object files. The assertion we want is cold-vs-cold byte-equality — which is what a third party with a fresh checkout reproduces. - Cosign cert-identity-regexp tightened to pin this exact workflow file on a tag-ref. The previous `^https://github.com/<repo>/` regex would have accepted a Sigstore bundle minted by any workflow on any branch in the same repo; the new pattern rejects sibling workflows. - SBOM coverage gate now walks every `Indirect != true` entry in go.mod and asserts a matching `pkg:golang/<path>@…` purl exists in the CycloneDX components[]. M3's "covers every module" rubric and M21's "≥1 component per direct module" rubric now have a falsifiable check; the previous `components ≥ 1` gate was a placeholder. - Recipe step 6 switched from `slsa-verifier verify-artifact` (legacy slsa-github-generator format) to `gh attestation verify` (the reference verifier for actions/attest-build-provenance's Sigstore bundle output). slsa-verifier ≥ 2.7.0 with `verify-github-attestation` is documented as the alternate path; earlier versions don't parse Bundle v0.3 and would have failed silently or noisily. - Recipe step 4 dropped `--exit-code` to match the CI fix; step 5 inherits the tightened cert-identity-regexp; the diffoscope-failure diagnostic row points at Go-toolchain drift (the actual common cause) rather than "compiler upgrade or -trimpath regression". - CHANGELOG entry added under [Unreleased] / Added; MILESTONES.md M3 flipped from ☐ to ⧗ with a flip-to-☑-on-merge note; top-level README.md routing table grew a row for auditors / supply-chain verifiers pointing at docs/reproducibility.md. - Dropped two unused job-level outputs (source_date_epoch, build_date) that no downstream job consumed; removed a vestigial `make clean` between builds (does nothing when artifacts live in mktemp dirs). Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
Pass-1 fix tightened the SBOM check to demand a component per direct go.mod require, but cyclonedx-gomod's `mod` BOM excludes test-only direct deps (testify, goleak) because they don't reach the main module's import graph. Intersect the expected set with `go list -deps ./cmd/tracecore` so the gate only demands SBOM entries for modules that actually ship in the linked binary. Verified locally: 10 direct go.mod requires → 8 runtime-reachable → all 8 present as `pkg:golang/<path>@…` purls in the m3test-4 SBOM. Assisted-by: Anthropic:claude-opus-4-7 [Claude Code] Signed-off-by: Tri Lam <trilamsr@gmail.com>
Five P1 items from the second-pass parallel review:
- Recipe step 6 now reads the bundle from disk (--bundle "$ATTEST")
instead of pulling it from GitHub's attestation API, and pins
--signer-workflow + --predicate-type. Two practical wins: the
verification works offline / air-gapped, and a sibling workflow
elsewhere in the repo cannot mint an attestation that passes the
documented command — cosign step 5 and gh-attest step 6 now anchor
to the same workflow-on-tag-ref identity.
- "If a step fails" row 6 label switched from `slsa-verifier` to
`gh attestation verify` so the diagnostic table matches the verb
the walkthrough uses.
- Recipe prerequisites paragraph dropped its dangling `slsa-verifier
≥ 2.7.0` alternate-path promise. The walkthrough never showed the
alternate command; adding it would have doubled the recipe surface
for marginal benefit. `gh attestation verify` is the single
documented verifier.
- SBOM job's checkout now pins to ${{ github.sha }} (the commit that
triggered the workflow) instead of the tag. A force-push to the tag
between the build and sbom jobs cannot produce an SBOM for a
different tree than was signed.
- MILESTONES.md M3 status line dropped the m3test-4 reference (stale
after subsequent test tags landed); replaced with "across the
v0.0.0-m3test-* series" so future test-tag iterations don't restale
the line. docs/FOLLOWUPS.md gains an M21 release-asset-shape
reconciliation bullet (raw binary vs tar.gz, .cosign.bundle vs .sig,
the .intoto.jsonl extension on Sigstore bundle JSON).
- Build #1 comment trimmed from an essay block to two sentences;
rationale lives in the commit history.
Deferred from Pass-2 (P2/L, not M3-blocking):
- diffoscope local exit-status wrapping (verifier copy-pasting one
block at a time can miss a non-zero exit; recipe polish, not gate
break)
- Repo tag-protection ruleset (org-policy decision, not PR scope)
- `go mod verify` in the build job (cheap hardening; defer to
separate supply-chain PR)
- Rekor log-index URL in release notes (post-fact audit polish)
- Caching `diffoscope-minimal` apt install (~10s, marginal on a
tag-triggered workflow)
Assisted-by: Anthropic:claude-opus-4-7 [Claude Code]
Signed-off-by: Tri Lam <trilamsr@gmail.com>
PR-body and three load-bearing source surfaces tightened to the documented conventions: - release.yml's provenance assertion drops a comment that paraphrased the `if` below it (subject-digest equality) — load-bearing logic unchanged, comment was pure narration. - MILESTONES.md §M21 picks up a Carry-forward from M3 sub-bullet spelling out the release-asset-shape decisions M3 deferred: raw-binary vs `.tar.gz`, `.cosign.bundle` vs detached `.sig`, the `.intoto.jsonl` extension misnomer on a Sigstore-bundle JSON. Same bullet points readers at the FOLLOWUPS section below for the rest of the hardening backlog so milestone-tracked and opportunistic follow-ups stay co-located per `feedback_deferral_tracking`. - docs/FOLLOWUPS.md gains "M3 release-pipeline hardening (post-PR #28)" — twelve trigger-clausal items: SLSA Build L3 via the reusable-workflow generator, nightly rebuild-and-diff cron, tag- protection ruleset on `v*`, `zizmor` + `actionlint` static-lint in `ci.yml`, github-actions Dependabot, `go mod verify`, build-env sanitization (LC_ALL/TZ/umask), cosign + `gh attestation` flag tightening, CycloneDX `mod`→`app` plus `specVersion` assertion, Rekor log-index URL in release notes, recipe polish (SDE echo, go env dump, `--html` artifact, local exit-status wrap), apt cache for `diffoscope-minimal`. Each entry cites its trigger. PR body separately rewritten to the problem → impact → solution → test plan shape, with the local `make ci` timing marked `(unverified — local measurement, not CI-cited)` per `feedback_no_invented_numbers`. Signed-off-by: Tri Lam <trilamsr@gmail.com>
…set shape Two honesty fixes to docs/FOLLOWUPS.md "M3 release-pipeline hardening (post-PR #28)": - The `github-actions` Dependabot row claimed the ecosystem still needed wiring. `.github/dependabot.yml:19-32` already enables it (package-ecosystem: github-actions on the .github root). Filed deferrals are supposed to be a backlog the project hasn't done yet; leaving an already-shipped item in the list misleads readers about real scope. Removed. - The "Release-asset shape reconciliation" row was duplicating the decision matrix that already lives in MILESTONES.md §M21 Carry-forward from M3. Collapsed to a single pointer line so the milestone-tracked deferral has one source of truth (per MEMORY.md `reference_deferral_tracking`: FOLLOWUPS is opportunistic, the Carry-forward sub-bullet on the milestone is milestone-tracked). Signed-off-by: Tri Lam <trilamsr@gmail.com>
Bring M5b (#29) into the M3 branch so PR #28 is mergeable. One real content conflict in CHANGELOG.md — both branches added an entry under [Unreleased] / Added; resolved by keeping both, M3 listed before M5b per the M3 → M5b → M6 → M21 minimum-viable v0.1.0 dependency chain documented in MILESTONES.md. docs/FOLLOWUPS.md auto-merged cleanly (M5b's additions land in disjoint sections from the M3 release-pipeline-hardening section added on this branch). No other files conflicted. `make ci` green (12.5 s wall, well under PRINCIPLES §10 60 s). doc-check passes; 214 markdown links resolve. Per MEMORY.md `feedback_no_history_rewrites` the resolution is a merge commit, not a rebase — origin/main is pushed history and cannot be rebased over. Signed-off-by: Tri Lam <trilamsr@gmail.com>
Catch the branch up to main: PR #30 (M4b failure-injection harness) and PR #31 (M11 NCCL FlightRecorder receiver + safe pickle parser) landed since the prior merge of #29. Both auto-merged cleanly into disjoint sections of CHANGELOG.md and docs/FOLLOWUPS.md — no manual conflict resolution required. `make ci` green (58.6 s wall, within PRINCIPLES §10 60 s; up from 12 s pre-merge because the M4b harness + chaos suite and M11 pickle parser added a substantial test surface). doc-check: 193 markdown links resolve. release.yml unchanged across the merge so the end-to-end attestation run on v0.0.0-m3test-7 still characterises the post-merge HEAD's release behavior. Signed-off-by: Tri Lam <trilamsr@gmail.com>
Catch up with PR #32 (M10 k8sevents receiver alpha). One content conflict in CHANGELOG.md — main's M5b entry was enriched with the values.schema.json + Artifact Hub annotation paragraph between PR #28's last merge and now. Resolved by keeping M3 (this PR's entry) above the post-enrichment M5b text. docs/FOLLOWUPS.md auto-merged cleanly. No other files conflicted; release.yml unchanged. doc-check green (193 markdown links resolve, 8 fenced bash/sh blocks shell-syntax-clean). Per MEMORY.md feedback_no_history_rewrites the resolution is a merge commit; origin/main is pushed history. Signed-off-by: Tri Lam <trilamsr@gmail.com>
The repo-level `Allow merge commits: false` setting already ensures main's history is linear by construction — squash and rebase are the only enabled merge methods, both produce a single linear commit on main. The separate `required_linear_history` rule added no additional guarantee for the main-branch outcome; its only distinct effect was to block PRs whose source branches absorbed merge commits via `git merge origin/main` conflict resolution. That collided with MEMORY.md `feedback_no_history_rewrites`, which forbids rebasing pushed history — leaving merge as the only honest way to bring main into a long-lived PR. Both surfaces updated: - `.github/branch-protection.yml:19` flipped to `false` with a rationale comment explaining the redundancy with allow_merge_commit. - `scripts/apply-branch-protection.sh:44` JSON payload flipped to match (the script's header explicitly requires both files to move together). Re-enable if a future workflow mixes regular merge-commits into the target branch (e.g. release branches, long-lived integration). The live `main` branch-protection setting will be flipped via the GitHub Settings UI in the same change window — this file edit makes the next apply-branch-protection.sh run idempotent against that. Signed-off-by: Tri Lam <trilamsr@gmail.com>
2 tasks
trilamsr
added a commit
that referenced
this pull request
May 15, 2026
…res (#38) ## Problem GitHub Settings → Branches → \`main\` was flipped in two ways during PR #28's merge window: \`required_linear_history\` re-enabled, \`required_signatures\` disabled. Both in-repo source-of-truth files (the YAML checklist + the apply-script JSON payload) absorbed the *opposite* values via PR #28's squash, so the docs no longer match live state. Next idempotent run of \`scripts/apply-branch-protection.sh\` would revert the live policy. ## Impact Re-aligns documented intent with live policy. No \`main\` behavior change — this is documentation catching up. ## Solution - \`.github/branch-protection.yml:15\`: \`require_linear_history: true\`, with a comment naming the known cost (blocks squash-merge for branches that absorbed merge commits) and the documented escape hatch (squash-collapse + force-push as a per-PR exception to \`feedback_no_history_rewrites\`). - \`.github/branch-protection.yml:38\`: \`require_signed_commits: false\`, with a comment naming the actual provenance chain on \`main\` today (PR-required gate + DCO \`Signed-off-by:\` + GitHub web-flow auto-sign on squash-merge) and the re-enable trigger (external contributors joining, or an explicit audit binding). - \`scripts/apply-branch-protection.sh:44,48\`: JSON payload flipped to match. ## Test plan - [x] \`gh api /repos/TraceCoreAI/tracecore/branches/main/protection\` → \`{required_linear_history: true, required_signatures: false}\` matches \`.github/branch-protection.yml\` lines 15 + 38 and \`scripts/apply-branch-protection.sh\` lines 44 + 48. - [x] \`bash -n scripts/apply-branch-protection.sh\` exits 0 (shell syntax clean). Signed-off-by: Tri Lam <trilamsr@gmail.com>
7 tasks
5 tasks
trilamsr
added a commit
that referenced
this pull request
Jun 2, 2026
…460) (#466) ## Summary Closes #460. The `exit 0` on `scripts/doc-check.sh` ran unconditionally whenever `docs/FAILURE-MODES.md` carried no `Test*`/`Fuzz*`/`Benchmark*` identifiers (its current state on `main` — `grep -c` = 0), silently bypassing every gate below it. Fix scopes the skip to the Go-test parity block only (if/else, not `exit`), then surfaces and fixes the dead refs the gates were supposed to be catching. ## Root cause Commit a57883f (#13) shipped `doc-check.sh` with one gate — the Go-test name parity check — so `[ -z "$referenced" ] && exit 0` was correct then. PRs #28, #56, #115, #131, #144, #149, #195, #234, #241, #443, #455, #459 (and others) appended gates **below** that line without recognising they'd become dead code whenever `FAILURE-MODES.md` lost its `Test*` references. PR #459 worked around the bug by placing its new YAML gate *above* line 99 and tracked the root cause separately as #460. ## What surfaced Once `exit 0` was removed, three real issues fired: 1. **Dead `.md` link**: `docs/FOLLOWUPS.md` → `followups/otlphttp.md`. The shard was never committed to `main`'s ancestry. Folded into the existing "Shards deleted post-v0.2.0 as fully resolved-via-pivot" prose block (sibling treatment to M9, M14, M16). 2. **Banned-phrase hits** (3x `production-grade`): reworded in `docs/cut-criteria.yaml.md` (2x) and `install/kubernetes/tracecore/README.md` (1x) to falsifiable language. 3. **`docs/getting-started.md` block cap**: 7 fenced bash/sh blocks. The M6 cap of 5 was set for the quickstart only — `## Install via Helm` and `## Air-gapped install` are alternate deployment paths that landed post-M6 and aren't part of the quickstart budget. Rescoped the gate to count blocks inside the `## Walkthrough` H2 section only (1 block, well under cap). ## Gate count Empirically verified via `grep -c '^doc-check: '` on `make doc-check` output on a clean tree: | State | Status lines emitted | Gates the early-exit was hiding | |---|---|---| | Pre-fix on `main` (post-#459) | 3 (trust-posture, YAML cross-link, parity-skip) | 14 | | Post-fix this PR (post-rebase) | 17 | 0 | The "14 gates hidden" number is invariant across the rebase: it counts gates placed below the early-exit line. The "3 → 17" total reflects post-#459 reality on `main`; pre-#459 baseline was "2 → 16" (the figure originally in this PR body), and #459 itself worked around the bug by placing its YAML gate above line 99. ## Mutation tests Each gate below the original early-exit was confirmed to fire post-fix: | Mutation | Gate expected to fire | Exit code post-mutation | Exit code post-restore | |---|---|---|---| | Inject `[bad](nonexistent-ghost.md)` into `docs/FOLLOWUPS.md` | markdown link-rot | 1 | 0 | | Append `blazing-fast` + `rock-solid` to `docs/getting-started.md` | banned-phrase lint | 1 | 0 | | Delete `<!-- tested-against: ... -->` from `docs/integrations/datadog.md` | M6 recipe markers | 1 | 0 | ## Test plan - [x] `make doc-check` exits 0 on clean tree (re-run post-rebase onto origin/main; 17 status lines) - [x] 3 mutation tests above each toggle exit 1 → 0 across mutate / restore - [x] Pre-push hooks green: golangci-lint (0 issues), `go vet ./...`, `go mod verify`, `attribute-namespace-check` (100 attrs, all documented), `register-lint`, `actionlint`, `zizmor`, `deprecation-check`, `no-autoupdate-check` - [x] Rebased onto current `origin/main` (includes #459, #461, #462, #456); no conflicts; gate count re-verified empirically post-rebase - [x] No changes to gates above line 99 (the trust-posture callout + YAML cross-link gate from #459 still run and emit unchanged status lines) ## Self-grade **A+** — root cause named in commit body (a57883f #13 with one gate; gates appended below without exit-path awareness); 3 mutation tests (success criteria required 1–2); rescoped the getting-started gate to match M6 intent rather than papering over the surfaced overflow; the `[ -z "$referenced" ]` legitimate skip is preserved via if/else (not `:` no-op, which would have left the `defined=` / `orphans=` block running on empty input); gate count corrected empirically post-rebase per reviewer B feedback. ```release-notes - fix(ci): `scripts/doc-check.sh` no longer exits 0 at the Go-test parity gate when `docs/FAILURE-MODES.md` carries no `Test*` references. 14 gates below that line (link-rot, banned-phrase, M6 recipe markers, etc.) are now actually enforced on every `make doc-check` invocation. Closes #460. ``` --------- Signed-off-by: Tri Lam <tree@lumalabs.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
v0.1.0 (M21) needs every release artifact to be byte-reproducible from the same SHA and verifiable end-to-end. M3 owns that pipeline.
Impact
Closes MILESTONES.md §M3. Unblocks M21 (release tag).
Solution
release.ymlbuildslinux/amd64twice at everyv*tag into isolatedmktemp -ddirs with freshGOCACHE(cold-vs-cold byte-equality is the assertion a third party reproduces — not warm-vs-warm);diffoscopefails closed on any byte diff.-trimpath+SOURCE_DATE_EPOCHhonored; embeddedBuildDatecross-checked againststringsof the binary.cyclonedx-gomod mod. Coverage gate intersects directgo.modrequires withgo list -deps ./cmd/tracecore, so the SBOM must enumerate every runtime-reachable direct module (test-only deps liketestify/goleakcorrectly excluded — they don't ship in the binary).sign-blob+ smokeverify-blob. Cert identity pinned to^https://github.com/<repo>/\.github/workflows/release\.yml@refs/tags/so a sibling workflow on another branch cannot mint a passing bundle. No long-lived secrets;id-token: writeis the only elevated permission outside thereleasejob.actions/attest-build-provenance. Bundle'spredicateType == https://slsa.dev/provenance/v1andsubject[0].digest.sha256are cross-checked against the build job's digest before the artifact is uploaded.${{ github.sha }}(not the mutable tag) so a force-push between build and SBOM cannot diverge the BOM from the binary.docs/reproducibility.mdis the third-party verification recipe (rebuild → diffoscope → cosign →gh attestation verify --bundle --signer-workflow --owner --predicate-type→ SBOM inspect). Works offline because step 6 reads the bundle from disk.scripts/doc-check.shgates its presence andbash -nsyntax of every fenced block; mutation-tested.M3 steps live in
release.yml, notmake ci. The base branch absorbed M5b (#29), M4b (#30), and M11 (#31) during this PR's life; merges with each are reflected in the branch andmake cistill runs under the PRINCIPLES §10 60 s budget on darwin/arm64 ((unverified — local measurement, not CI-cited)).Test plan
make cigreen on the latest commit; under 60 s — seeverifyon the PR's CI workflow run.if then fiinto a fenced block → gate fails with block-#; restore → gate passes.release.ymlruns end-to-end onv0.0.0-m3test-7(commitb6c745f) with diffoscope / cosign / SLSA / SBOM all green — run 25914673989.release.ymlis unchanged across the subsequent commits on this branch (the two merges ofmainplus two doc fixups touched no workflow file), so the run still characterizes the current HEAD's release behavior.v0.0.0-m3test-7:cosign verify-blob→Verified OK;gh attestation verifyexit 0 withpredicateType=https://slsa.dev/provenance/v1andbuildSignerURIpinned torelease.yml@refs/tags/v0.0.0-m3test-7.Out of scope (filed as follow-ups)
Above-the-floor hardening (SLSA L3 via the reusable-workflow generator,
zizmor/actionlint, repo tag-protection onv*, nightly rebuild-and-diff cron,go mod verify, build-env sanitization, cosign +gh attestationflag tightening, CycloneDXmod→app, Rekor log-index, recipe polish, apt cache) lives indocs/FOLLOWUPS.mdunder "M3 release-pipeline hardening (post-PR #28)" and as a Carry-forward bullet onMILESTONES.md§M21.