feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers)

[polaz]

Summary

Phase 6.1 of #28: lay the C ABI foundation so external consumers can replace libzstd.so.1 with our build. Covers the verbatim-vendored upstream headers, the new c-api/ workspace member that produces cdylib + staticlib, and the extern "C" wrappers for the simple + context + error + frame-inspection slice of zstd.h.

Phase 6 design (#28 Phase 6) chose vendor upstream headers verbatim + hand-written extern "C" wrappers so macros / typedef names / doxygen comments / enum numeric values are byte-for-byte identical to what consumers parse today. Target upstream version: v1.5.7 (matches Alpine zstd-1.5.7-r2).

Deliverables

1. Vendored headers (upstream v1.5.7 verbatim)

• c-api/include/zstd.h
• c-api/include/zdict.h
• c-api/include/zstd_errors.h

Verbatim copy. No edits. Install path: /usr/include/. Tracking comment at top of each file records the upstream SHA + version we vendored from.

2. New workspace member c-api/

[package]
name = "structured-zstd-c"
# ...

[lib]
crate-type = ["cdylib", "staticlib"]

build.rs:

• Sets cargo:rustc-cdylib-link-arg=-Wl,-soname,libzstd.so.1 (Linux) — the linker emits the SONAME consumers expect.
• Generates c-api/libzstd.pc with the Version: 1.5.7 line so pkg-config --modversion libzstd matches upstream.
• Installs the three headers under OUT_DIR for downstream packaging.

3. Extern "C" wrappers covered by this issue

Scope: the synchronous slice of zstd.h. Advanced/streaming/dictionary moves to 6.2 (#62), CLI moves to 6.3 (#63).

Simple API:

• ZSTD_compress, ZSTD_decompress, ZSTD_compressBound
• ZSTD_isError, ZSTD_getErrorName
• ZSTD_getFrameContentSize, ZSTD_findFrameCompressedSize
• ZSTD_minCLevel, ZSTD_maxCLevel, ZSTD_defaultCLevel
• ZSTD_versionNumber, ZSTD_versionString

Context API (synchronous part):

• ZSTD_createCCtx, ZSTD_freeCCtx
• ZSTD_createDCtx, ZSTD_freeDCtx
• ZSTD_compressCCtx, ZSTD_decompressDCtx
• ZSTD_sizeof_CCtx, ZSTD_sizeof_DCtx

Error code mapping (zstd_errors.h):

• ZSTD_ErrorCode enum exposed to Rust as #[repr(u32)]
• Rust Err(...) variants → ZSTD_ErrorCode via a From impl
• ZSTD_getErrorCode, ZSTD_getErrorString, ZSTD_isError all bottomed on this mapping

Frame inspection:

• ZSTD_getFrameHeader, ZSTD_getFrameHeader_advanced
• ZSTD_findDecompressedSize
• ZSTD_decompressBound, ZSTD_frameHeaderSize

4. ABI invariants locked in tests

• c-api/tests/abi.rs: static asserts on opaque struct sizes/alignments via core::mem::size_of so a Rust-side refactor that grows CCtx/DCtx past the upstream sentinel size fails CI rather than ABI-breaking consumers.
• c-api/tests/c_consumer.c (compiled by build.rs via cc for the test cfg): a tiny program that #include <zstd.h>s the vendored header, calls ZSTD_compress + ZSTD_decompress, asserts roundtrip. Catches "compiles in Rust but a real C consumer can't link" regressions.
• c-api/tests/symbols.rs: parses nm -D output of the built .so and asserts every symbol declared in the vendored .h is exported with the right binding (T, not U).

Out of scope (covered by sibling Phase 6 issues)

• Streaming ZSTD_CStream/DStream API → 6.2 (perf(bench): add rust/ffi delta charts by corpus+compression params #62)
• Advanced parameter API (ZSTD_CCtx_setParameter family) → 6.2 (perf(bench): add rust/ffi delta charts by corpus+compression params #62)
• Dictionary API (ZSTD_CDict/ZSTD_DDict, ZDICT_trainFromBuffer) → 6.2 (perf(bench): add rust/ffi delta charts by corpus+compression params #62)
• zstd CLI binary → 6.3 (feat(encoding): numeric compression levels (1-22) API #63)
• Multi-threaded compression (ZSTD_c_nbWorkers) → 6.4 (chore: release v0.0.7 #64)
• Legacy frame format decoders → 6.5/6.6 (perf(bench): add rust/ffi delta benchmark artifacts #65/perf(decoding): SIMD HUF decode kernels with runtime CPU dispatch #66)
• Conformance + reverse-dep matrix → 6.7 (perf(encoding): implement row-based match finder for fast/dfast levels #67)

Acceptance criteria

• c-api/ workspace member builds both cdylib (libstructured_zstd.so) and staticlib (libstructured_zstd.a) without unsafe-feature opt-outs.
• Built .so has SONAME=libzstd.so.1 verifiable via readelf -d / otool -D.
• Vendored zstd.h/zdict.h/zstd_errors.h are byte-for-byte identical to upstream v1.5.7 release tarballs (CI diff against pinned SHA).
• c-api/libzstd.pc reports Version: 1.5.7, Libs: -lzstd, Cflags: -I${includedir}.
• All listed synchronous wrappers compile + link + round-trip a 1 MiB random buffer through ZSTD_compress → ZSTD_decompress (in c_consumer.c).
• Error code mapping is bidirectional: every Rust error variant has a deterministic ZSTD_ErrorCode, and ZSTD_getErrorCode(ZSTD_compress(...)) recovers it.
• symbols.rs snapshot test: every symbol declared in vendored zstd.h headers appears in nm -D output of the built .so.
• Existing Rust test suite still passes (cargo nextest run --features hash,std,dict_builder).

Estimate

~10-12 working days (vendoring + workspace plumbing + ~30 wrappers + ABI tests).

Blocked by

• feat: encoder architecture rewrite (split monolith, const-generic Strategy, arena allocator) #111 (encoder architecture rewrite) — internal API must stabilize first; otherwise wrappers chase a moving target.
• feat: configurable compression parameters API #27 (configurable parameters API) — for ZSTD_CCtx_setParameter in 6.2, but 6.1 only needs the simple + context API which doesn't depend on it. 6.1 is unblocked by feat: configurable compression parameters API #27 itself but the broader phase is.

References

• Upstream v1.5.7: https://github.com/facebook/zstd/releases/tag/v1.5.7
• Alpine zstd package: https://pkgs.alphalinux.org/package/edge/main/x86_64/zstd
• Phase 6 design discussion: Roadmap: structured-zstd feature parity with C zstd #28 (roadmap)

[coderabbitai]

🔗 Related PRs

#30 - feat: large literals block support (>262KB) [merged]
#38 - test(bench): expand zstd benchmark suite [merged]
#58 - perf(decoding): branchless bitstream reader with mask table and BMI2 support [merged]
#60 - feat(encoding): write frame content size in encoder output [merged]
#92 - perf(decoding): SIMD HUF kernels with runtime dispatch [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!