Publish hardening on top of #17: win-cuda binary hashes in the manifest, full-coverage publish guard#20
Merged
Conversation
…x_by, redist provenance
…ing_systems subset)
The arm64 slice was built on the macos-14 runner. On M5/A19 hardware that binary logs 'error compiling source' for the Metal library (it falls back to the standard path and still runs, but the error is alarming). Building on macos-26 (the newest Metal SDK, same runner upstream uses for arm64) compiles cleanly on M5 while the explicit deploy_target=14.0 keeps minos at 14.0, so the bundle still loads on macOS 14+. Verified on M5 Max, M3 Max, M2 Pro and the macos-14 CI runner: minos 14.0, no compile error, identical throughput.
The sha256 index only carried the cudart-llama-bin-win-cuda-*.zip
digests. The installer resolves an attempt's hash by exact asset name
first and falls back to a cudart alias, so every Windows CUDA binary
zip was assigned the cudart digest and failed download verification,
degrading those hosts to source builds. It also meant no win-cuda-13.3
entry existed at all, so 13.3-driver hosts could never take the
in-release build through the fork manifest path.
Record the llama-{tag}-bin-win-cuda-*.zip binaries in the index under
their own names (same windows-cuda-upstream kind as the paired cudart).
Verified against b9518: the 12.4 binary entry now digests 52e8dddb
(GitHub's published asset digest) instead of inheriting cudart's
8c79a9b2, and the 13.3 entry exists.
…DA x64 The publish guard only asserted cuda12/cuda13 x64 parity. macOS, ROCm, and the linux arm64 bundle were enforced indirectly (child if-no-files-found plus needs atomicity), so a download-artifact anomaly inside assemble - green children, files absent from dist/ - could still publish a release missing whole platform lines, exactly the failure shape that stranded cuda12 hosts at b9518. Assert presence of both macOS slices, the arm64 cuda13-portable bundle, and the default ROCm gfx set on both OSes. The resolve-time input guard already pins publish runs to the full default matrix, so the expected names are exact. Collect all missing pieces before failing so one run reports the full gap list.
This was referenced Jun 10, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #17 (oobabooga:unsloth-build): the first commits here are that branch verbatim, plus two fixes on top. Merge #17 first; this PR then reduces to the last two commits. Both address findings from the fleet validation recorded on #17 and unslothai/unsloth#5963.
1. Manifest: record upstream win-cuda binary hashes under their own names
The sha256 index only carried the
cudart-llama-bin-win-cuda-*.zipdigests. The installer resolves an attempt's hash by exact asset name first and only then falls back to a cudart alias, so every Windows CUDA binary zip was assigned the cudart digest and failed download verification, degrading those hosts to source builds. It also meant no win-cuda-13.3 entry existed at all, so a 13.3-driver host could never take the in-release build through the fork manifest path.assemble_metadata.pynow records thellama-{tag}-bin-win-cuda-*.zipbinaries under their own names (samewindows-cuda-upstreamkind as the paired cudart). No installer change is needed: exact-name lookup wins, and the alias stays as a backstop for older manifests.Verified by running the script against the live b9518 release: the 12.4 binary entry digests
52e8dddb(GitHub's published asset digest) instead of inheriting cudart's8c79a9b2, and the 13.3 entry now exists:2. CI: verify macOS, ROCm, and arm64 bundles before publish, not just CUDA x64
The publish guard only asserted cuda12/cuda13 x64 parity. macOS, ROCm, and the linux arm64 bundle were enforced indirectly (child
if-no-files-found: errorplusneedsatomicity), so a download-artifact anomaly inside assemble - green children, files absent from dist/ - could still publish a release missing whole platform lines, the same failure shape that stranded cuda12 hosts at b9518.The guard now also asserts both macOS slices, the arm64 cuda13-portable bundle, and the default ROCm gfx set on both OSes, and collects every missing piece before failing so one run reports the full gap list. The resolve-time input guard already pins publish runs to the full default matrix, so the expected names are exact and
publish=falsesubset test runs are unaffected.Tested by extracting the guard body and running it against a synthetic dist/: full set passes; removing the macOS x64 slice, a ROCm zip, or a cuda12 profile each fails with the specific missing asset named.