feat: add compressed SFS, shrink pipeline, multi-version Perl, and release workflow#32
Open
lbe wants to merge 1 commit into
Open
feat: add compressed SFS, shrink pipeline, multi-version Perl, and release workflow#32lbe wants to merge 1 commit into
lbe wants to merge 1 commit into
Conversation
…lease workflow Multi-version Perl support (5.16.x and 5.42.x lines) with WASI cross-compilation, deterministic shrink pipeline, LZ4-compressed SFS embedding, and a matrix release workflow for generating variant WASM artifacts. Pipeline & Docker - Extend Dockerfile and build scripts for native prefix, shrink regen, prefix prep, wasm link (NO_MATHOMS/XS init, SFS compression, embed-prefix=false paths). - Add build.sh orchestration script for local end-to-end builds. - Refresh hints-wasi, prepare-prefix, build-wasm, build-exiftool, build-native-perl, build-wasi-libs. - Add .dockerignore and root package.json for workspace tooling. Shrink pipeline - Add tools/regen-wasm-shrink.sh, tools/trace-zeroperl-deps.pl, tools/emit-wasm-xs-bundle.pl for traced dependency collection and deterministic shrink artifact generation. - Add tools/wasm-shrink.env for tracer configuration. - Replace glob.patch with more resilient patch_glob.pl; add patch_mg.pl, patch_iperlsys.pl, patch_sv_locale.pl for version-specific source fixes. - Add tools/unicore-strip.pl, tools/file-glob-shim.pm, tools/perltidy-skip.txt, tools/extract-missing-paths.pl. - Add tools/wasm-xs-kernel.txt, tools/check-wasm-shrink-determinism.sh. SFS compression - Rework tools/sfs.js for LZ4-framed entries with codec=0/1 per-file. - Add stubs/sfs_runtime.c, stubs/sfs_compression.c, stubs/sfs_lz4.h, stubs/sfs_runtime.h, stubs/sfs_compression.h for lazy decompression with LRU cache (20 MB cap, 4096 entries). - Wire ZEROPERL_SFS_COMPRESS through Dockerfile, prepare-prefix.sh, build-wasm.sh. Smoke tests and verification - Add tests/smoke/core-smoke.pl, tests/smoke/core-mod-smoke.pl, tests/smoke/lib/Core/TestMod.pm, and sample corpus (sample.jpg.b64, sample.tiff.b64, sample.xmp). - Add tools/wasm-smoke.mjs, tools/wasm-smoke.sh, tools/verify-embedded-inc.mjs for embedded prefix verification. - Add tests/sfs/test-generator.js, tests/sfs/test-sfs.c, tests/sfs/Makefile for native SFS unit tests. CI and release - Add zeroperl-release.yml: matrix builds 4 WASM variants in parallel (exiftool-5.16.3, exiftool-5.42.2, perl-only-5.16.3, perl-only-5.42.2) with full CI verification (shrink determinism, smoke tests, embedded @inc check) and publishes as a draft GitHub release. - Remove zeroperl.yml (replaced by zeroperl-release.yml). - Update nodefs workflow and Binaryen/WABT setup actions. - Update all GitHub Actions to Node.js 24 compatible versions (checkout@v7, upload-artifact@v7, download-artifact@v8, action-gh-release@v3). Submodules and repo hygiene - Add zeroperl-ts git submodule; remove go-exiftool-wasm submodule. - Remove test-matrix.sh, test-matrix-all.sh, run-failed-and-report.sh (depended on go-exiftool-wasm). - Remove perl-delta-spec.md, .vscode/settings.json, check-fs.mjs. - Update .gitignore; add gen/, output/, test-output/, tmp/ entries. Docs - Add comprehensive ARCHITECTURE.md covering build pipeline, runtime architecture, shrink modes, SFS compression, and CI verification. - Revise README.md with build instructions, build arg reference, and submodule setup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The changes in this pull request were driven by my need to have as small of a WASM file as I could create to support go-exiftool-wasm, a Go package that is based upon translating WASM to native Go. I will be the first to admit that it is very large for a pull request. I did my best to approach your original package from an extend, not rewrite approach. I think was able to follow this approach except for needing to move from patch files to perl patch scripts to support various versions of perl without having to have a patch file per version.
A few points on the perl versions:
I wll stop here, just short of War & Peace. I hope that this merge meets you acceptance. Please let me know if there is anything that I can do to help. LBE
Multi-version Perl support (5.16.x and 5.42.x lines) with WASI cross-compilation, deterministic shrink pipeline, LZ4-compressed SFS embedding, and a matrix release workflow for generating variant WASM artifacts.
Pipeline & Docker
Shrink pipeline
SFS compression
Smoke tests and verification
CI and release
Submodules and repo hygiene
Docs