refactor(encoding): split match_generator monolith + separate tests from production#445
Merged
Conversation
Move the per-level tuning config out of match_generator.rs into a dedicated levels/config.rs: the HcConfig / RowConfig / DfastConfig / FastConfig knobs, LevelParams + LEVEL_TABLE, the public-parameter overrides, the source-size tiering, and the workspace estimators. Pure mechanical move with no behaviour change: the code is relocated verbatim (only the relative `super::` paths rewritten to absolute `crate::encoding::` for the deeper module, and the resolution API widened to pub(crate)), so the compressed output stays byte-identical. match_generator.rs drops 1119 lines (13125 -> 12006). Consumers (frame_compressor, dfast, row) and the public re-export of the workspace estimators now source the API from levels::config. Part of #441.
Move the optimal parser's per-CPU-tier price-set kernels
(priceset_range_nonabort_{scalar,avx2,neon,sse41,simd128} plus their
cached-price / improved-mask helpers) out of match_generator.rs into
hc/priceset.rs. Verbatim move (kernels unchanged; only the opt-parser type
imports pulled in and the nonabort entry points widened to pub(crate)), so the
output is byte-identical. match_generator.rs: 12006 -> 11306.
Part of #441.
Move the HashChain / binary-tree match generator out of match_generator.rs into hc/generator.rs: the HcMatchGenerator storage + methods, the btlazy2 cost helpers, the optimal-plan body macros, and the build_optimal_plan driver. The cross-module macros (bt_insert_*, hash3_candidate, for_each_repcode) now resolve through hc::generator; their consumers (match_table::storage, hc) are repointed. Verbatim move (only relative paths rewritten to absolute and the driver-facing surface widened to pub(crate)), so the output is byte-identical. The small HcBackend enum stays in match_generator (its impl moved). match_generator.rs: 11306 -> 7438. Part of #441.
Move the binary-tree optimal parser out of hc/generator.rs into hc/optimal.rs: the build_optimal_plan DP + its per-CPU-tier kernels, the candidate-collection / price-set body macros, and the btlazy2 cost helpers, as a second `impl HcMatchGenerator` block over the same matcher. Both files are hc/ submodules so no path rewrite was needed; output is byte-identical. The DP body macros reference opt types unqualified, which rustc's unused-import lint cannot see (so it false-flags every such import) -- handled by a documented module-level suppression in optimal.rs; generator.rs's now-redundant (qualified) imports were removed by hand. hc/generator.rs: 3918 -> 1691. Part of #441.
Convert match_generator.rs into a directory module and move the seven dictionary entry points (resident check + reapply, prime, snapshot restore/capture, invalidate, entropy seed) out of the driver's main `Matcher` impl into a `dict_prime` child module. The snapshot path reads/writes the driver's most invariant-critical private state (`primed`, `reset_shape`, the `PrimedKey` snapshot identity that guards against restoring a mismatched matcher shape and emitting an undecodable match). A child module reaches those private fields directly, so the concern relocates with zero field widening -- encapsulation stays intact. The trait surface keeps thin `#[inline]` forwarders to the child's inherent `*_impl` helpers, so output is byte-identical. match_generator/mod.rs: 7438 -> 7000. Part of #441.
Relocate the ~4.9k lines of unit / round-trip tests (parse x search matrix, per-level parameter resolution, greedy/lazy/optimal round-trips, dictionary priming) out of the driver module body into a dedicated `tests` submodule file. Production code stays in mod.rs; the test file is `#[cfg(test)] mod tests;`. match_generator/mod.rs: 7000 -> 2063 (production only). Part of #441.
Relocate every inline `#[cfg(test)] mod tests { ... }` block across the
crate into a sibling `tests.rs` submodule, leaving each source file with
production code only plus a `#[cfg(test)] mod tests;` declaration. Test
bodies are moved verbatim (rustfmt re-indents; string literals untouched).
Relative `include_bytes!` paths in the moved dictionary tests gain one
`../` level to keep resolving to `zstd/dict_tests` from the deeper file
location.
No production logic changes; the full suite still runs the same 824 cases.
Separate the remaining inline test code that was not wrapped in a
`mod tests { ... }` block (so the earlier sweep did not catch it) into
sibling `tests.rs` submodules:
- fse, huff0 (mod + encoder): bare module-scope `#[test]` tail blocks.
- bit_reader_reverse: the `mod test` block plus the two `#[cfg(test)]`
helper fns that only its tests used.
- sequence_section_decoder: module-scope tests plus the two nested
`#[cfg(test)] mod` blocks; `super::` paths gain one level for the
deeper nesting.
- hc/generator, hc/priceset: test fns left behind by the match-generator
split. The `#[cfg(test)]` `start_matching` driver method stays with the
struct (it reaches private fields); only the test cases move.
Two production helpers (`distribute_weights` / `redistribute_weights`,
called from `legacy_distributed_weights`) that sat among the encoder
tests are kept in production. Relative `include_bytes!` paths adjusted
for the deeper file locations. Suite unchanged at 824 cases; x86_64 +
BMI2 build cross-checked for the gated bit-reader helper.
Extract the `#[cfg(test)] mod <name>_tests { ... }` blocks (which the
first sweep skipped because they are not named `tests`) into sibling
files declared `#[cfg(test)] mod <name>_tests;`. A named module relocated
to its own same-named file keeps its parent, so `super::` paths are
unchanged and the bodies move verbatim.
Covers compress_bound / hc / tag_mask / neon_tag_mask / ldm_helper /
extend_with_repcode / bt_pair_index_wrap / stream_abs_headroom / storage
/ zerocopy_robustness / burst_gate / frame_inspection / inline_helper /
portable_helper tests across the encoding and decoding trees. Suite
unchanged at 824 cases; x86_64 build cross-checked for the arch-gated
tag-mask tests.
Relocate the inline `#[cfg(test)] mod tests` blocks in `progress` and the binary root into sibling `tests.rs` files (the crate root's submodule lands at `src/tests.rs`). Production code only in the source files; 22 cli tests unchanged.
|
Too many files changed for review. ( |
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes. 💤 Files selected but had no reviewable changes (6)
⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR moves inline tests into separate modules across CLI, decoding, dictionary, and encoding code, and adds the HC generator module plus related matcher tests and wiring. ChangesCLI and codec test extraction
HC generator extraction
Estimated code review effort🎯 5 (Critical) | ⏱️ ~100 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (10)
cli/src/progress/tests.rs (1)
1-31: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFix rustfmt violations to unblock CI
Line 1 onward in this file is currently failing
cargo fmt --check. Please runcargo fmt --all(or format this file) before merge.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@cli/src/progress/tests.rs` around lines 1 - 31, The test file cli/src/progress/tests.rs is failing cargo fmt --check due to formatting violations. Run cargo fmt --all or cargo fmt on this specific file to automatically apply the correct Rust formatting standards to the human_readable_filesize and human_readable_duration test functions and any other formatting issues in the file.Source: Pipeline failures
zstd/src/encoding/compress_bound_tests.rs (1)
1-32: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winApply rustfmt to this test file.
cargo fmt --checkreports a diff here, so CI will stay failing until this file is formatted.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/encoding/compress_bound_tests.rs` around lines 1 - 32, The test file compress_bound_tests.rs does not conform to Rust's standard formatting rules as reported by cargo fmt. Run the Rust formatter on this file to automatically fix any formatting issues. This applies to all test functions in the file including matches_upstream_formula_below_threshold, drops_margin_at_and_above_threshold, saturates_instead_of_wrapping, and always_fits_real_compressed_output, as well as the imports and overall code structure. Use cargo fmt to apply the standard formatting rules and resolve the CI failure.Source: Pipeline failures
zstd/src/encoding/dfast/extend_with_repcode_tests.rs (1)
1-414: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFormat this extracted dfast test module with rustfmt.
The lint job reports a
cargo fmt --checkdiff in this file, so this is currently blocking a clean CI pass.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/encoding/dfast/extend_with_repcode_tests.rs` around lines 1 - 414, The test file extend_with_repcode_tests.rs is not formatted according to Rust's standard formatting rules, which is causing the cargo fmt --check to fail. Run cargo fmt on this file to automatically apply the standard Rust formatting conventions. This will fix all formatting issues and allow the CI to pass without any manual code changes needed, only the formatter output.Source: Pipeline failures
zstd/src/encoding/bt/ldm_helper_tests.rs (1)
1-333: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRun
cargo fmton this extracted test module before merge.CI is failing
cargo fmt --checkfor this file, so the lint gate is currently red.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/encoding/bt/ldm_helper_tests.rs` around lines 1 - 333, The test file ldm_helper_tests.rs is not formatted according to Rust's standard code style and is failing the cargo fmt --check lint gate in CI. Run cargo fmt on this file to automatically fix all formatting issues and ensure the code complies with the formatting standards required for merge.Source: Pipeline failures
zstd/src/encoding/hc/hc_tests.rs (1)
1-187: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winApply rustfmt for this test module to unblock CI.
cargo fmt --checkis failing on this file in CI; please runcargo fmt --alland commit the formatting-only update.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/encoding/hc/hc_tests.rs` around lines 1 - 187, The hc_tests.rs file has formatting inconsistencies that are failing the rustfmt checks in CI. Run cargo fmt --all from the repository root to automatically reformat all files including the test functions chain_candidates_returns_sentinels_when_suffix_too_short, chain_candidates_terminates_on_self_loop_with_in_range_pick, hash_chain_candidate_picks_longest_forward_over_shorter_with_backward_room, and hash_chain_candidate_forward_ties_keep_first_visited, then commit the formatting-only changes.Source: Pipeline failures
zstd/src/decoding/literals_section_decoder/burst_gate_tests.rs (1)
1-256: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winRun rustfmt on this file to unblock CI.
cargo fmt --checkis failing for this file, so this PR cannot merge until formatting is applied.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/decoding/literals_section_decoder/burst_gate_tests.rs` around lines 1 - 256, The file burst_gate_tests.rs contains formatting violations that are preventing the CI checks from passing. Run rustfmt on this file to automatically apply the correct Rust formatting standards. You can do this by executing the command to format the file, which will fix all whitespace, indentation, and other style issues throughout the test module.Source: Pipeline failures
zstd/src/decoding/literals_section_decoder/zerocopy_robustness_tests.rs (1)
1-132: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winFormat this test file with rustfmt.
The lint pipeline reports a formatting diff here.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/decoding/literals_section_decoder/zerocopy_robustness_tests.rs` around lines 1 - 132, The test file zerocopy_robustness_tests.rs has formatting issues detected by the lint pipeline. Run rustfmt on this file to apply standard Rust formatting rules and resolve all formatting inconsistencies throughout the entire module, including the helper functions like raw_section, rle_section, and fresh_scratch as well as all test functions like raw_truncated_source_returns_error_no_panic, rle_empty_source_returns_error_no_panic, compressed_truncated_source_returns_error_no_panic, and rle_view_excludes_pre_existing_target_bytes.Source: Pipeline failures
zstd/src/decoding/exec_sequence_inline/inline_helper_tests.rs (1)
1-87: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRun rustfmt on this new test module.
CI shows
cargo fmt --checkfailing for this file, so this PR stays red until formatting is applied.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/decoding/exec_sequence_inline/inline_helper_tests.rs` around lines 1 - 87, The test module in inline_helper_tests.rs is not properly formatted according to Rust's standard formatting rules, causing cargo fmt --check to fail. Run rustfmt on the inline_helper_tests.rs file to automatically apply the correct formatting to all test functions (copy16_copies_exactly_16_bytes, wildcopy_no_overlap_short_length_overshoots, wildcopy_no_overlap_length_above_16_uses_multiple_iters, wildcopy_overlap_8byte_stride_rle_expansion_offset_8, overlap_copy8_offset_ge_8_does_plain_copy, and overlap_copy8_offset_lt_8_spreads_source) and ensure CI passes.Source: Pipeline failures
zstd/src/decoding/exec_sequence_inline/portable_helper_tests.rs (1)
1-90: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winApply rustfmt to this file before merge.
cargo fmt --checkis failing on this module in CI.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/decoding/exec_sequence_inline/portable_helper_tests.rs` around lines 1 - 90, The file portable_helper_tests.rs does not comply with Rust formatting standards as indicated by the failing cargo fmt check in CI. Run cargo fmt on this file to automatically apply the correct formatting conventions and ensure all code style issues are resolved before merging.Source: Pipeline failures
zstd/src/decoding/frame_inspection_tests.rs (1)
1-202: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRun rustfmt for this file to unblock CI.
CI reports this file would be reformatted (
cargo fmt --checkfailure). Please format it before merge.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@zstd/src/decoding/frame_inspection_tests.rs` around lines 1 - 202, The file has formatting issues that cause CI to fail the `cargo fmt --check` validation. Run `cargo fmt` on the frame_inspection_tests.rs file to automatically reformat it according to Rust style standards, which will resolve the CI formatting failure.Source: Pipeline failures
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cli/src/tests.rs`:
- Around line 1-219: The cli/src/tests.rs file has formatting issues detected by
CI. Run cargo fmt --all from the project root to automatically apply the Rust
formatting standards to all files, including this test file. This will ensure
all formatting violations are corrected to pass the linting checks.
In `@zstd/src/decoding/frame_decoder/tests.rs`:
- Around line 2150-2152: The test at line 2150 uses a runtime-relative path
"./dict_tests/dictionary" which depends on the current working directory when
the test runs, making it fragile and test harness-dependent. Instead of using
std::fs::read with a relative path to load the dictionary fixture, identify the
compile-time fixture loading strategy used elsewhere in this test file (likely
using include_bytes! or similar compile-time macros) and apply the same approach
to obtain the raw dictionary data for the Dictionary::decode_dict call. This
ensures the fixture is embedded at compile-time and independent of runtime
working directory.
---
Outside diff comments:
In `@cli/src/progress/tests.rs`:
- Around line 1-31: The test file cli/src/progress/tests.rs is failing cargo fmt
--check due to formatting violations. Run cargo fmt --all or cargo fmt on this
specific file to automatically apply the correct Rust formatting standards to
the human_readable_filesize and human_readable_duration test functions and any
other formatting issues in the file.
In `@zstd/src/decoding/exec_sequence_inline/inline_helper_tests.rs`:
- Around line 1-87: The test module in inline_helper_tests.rs is not properly
formatted according to Rust's standard formatting rules, causing cargo fmt
--check to fail. Run rustfmt on the inline_helper_tests.rs file to automatically
apply the correct formatting to all test functions
(copy16_copies_exactly_16_bytes, wildcopy_no_overlap_short_length_overshoots,
wildcopy_no_overlap_length_above_16_uses_multiple_iters,
wildcopy_overlap_8byte_stride_rle_expansion_offset_8,
overlap_copy8_offset_ge_8_does_plain_copy, and
overlap_copy8_offset_lt_8_spreads_source) and ensure CI passes.
In `@zstd/src/decoding/exec_sequence_inline/portable_helper_tests.rs`:
- Around line 1-90: The file portable_helper_tests.rs does not comply with Rust
formatting standards as indicated by the failing cargo fmt check in CI. Run
cargo fmt on this file to automatically apply the correct formatting conventions
and ensure all code style issues are resolved before merging.
In `@zstd/src/decoding/frame_inspection_tests.rs`:
- Around line 1-202: The file has formatting issues that cause CI to fail the
`cargo fmt --check` validation. Run `cargo fmt` on the frame_inspection_tests.rs
file to automatically reformat it according to Rust style standards, which will
resolve the CI formatting failure.
In `@zstd/src/decoding/literals_section_decoder/burst_gate_tests.rs`:
- Around line 1-256: The file burst_gate_tests.rs contains formatting violations
that are preventing the CI checks from passing. Run rustfmt on this file to
automatically apply the correct Rust formatting standards. You can do this by
executing the command to format the file, which will fix all whitespace,
indentation, and other style issues throughout the test module.
In `@zstd/src/decoding/literals_section_decoder/zerocopy_robustness_tests.rs`:
- Around line 1-132: The test file zerocopy_robustness_tests.rs has formatting
issues detected by the lint pipeline. Run rustfmt on this file to apply standard
Rust formatting rules and resolve all formatting inconsistencies throughout the
entire module, including the helper functions like raw_section, rle_section, and
fresh_scratch as well as all test functions like
raw_truncated_source_returns_error_no_panic,
rle_empty_source_returns_error_no_panic,
compressed_truncated_source_returns_error_no_panic, and
rle_view_excludes_pre_existing_target_bytes.
In `@zstd/src/encoding/bt/ldm_helper_tests.rs`:
- Around line 1-333: The test file ldm_helper_tests.rs is not formatted
according to Rust's standard code style and is failing the cargo fmt --check
lint gate in CI. Run cargo fmt on this file to automatically fix all formatting
issues and ensure the code complies with the formatting standards required for
merge.
In `@zstd/src/encoding/compress_bound_tests.rs`:
- Around line 1-32: The test file compress_bound_tests.rs does not conform to
Rust's standard formatting rules as reported by cargo fmt. Run the Rust
formatter on this file to automatically fix any formatting issues. This applies
to all test functions in the file including
matches_upstream_formula_below_threshold, drops_margin_at_and_above_threshold,
saturates_instead_of_wrapping, and always_fits_real_compressed_output, as well
as the imports and overall code structure. Use cargo fmt to apply the standard
formatting rules and resolve the CI failure.
In `@zstd/src/encoding/dfast/extend_with_repcode_tests.rs`:
- Around line 1-414: The test file extend_with_repcode_tests.rs is not formatted
according to Rust's standard formatting rules, which is causing the cargo fmt
--check to fail. Run cargo fmt on this file to automatically apply the standard
Rust formatting conventions. This will fix all formatting issues and allow the
CI to pass without any manual code changes needed, only the formatter output.
In `@zstd/src/encoding/hc/hc_tests.rs`:
- Around line 1-187: The hc_tests.rs file has formatting inconsistencies that
are failing the rustfmt checks in CI. Run cargo fmt --all from the repository
root to automatically reformat all files including the test functions
chain_candidates_returns_sentinels_when_suffix_too_short,
chain_candidates_terminates_on_self_loop_with_in_range_pick,
hash_chain_candidate_picks_longest_forward_over_shorter_with_backward_room, and
hash_chain_candidate_forward_ties_keep_first_visited, then commit the
formatting-only changes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 9dc1eba6-bf05-42a6-b382-4c899adb3c91
📒 Files selected for processing (148)
cli/src/main.rscli/src/progress.rscli/src/progress/tests.rscli/src/tests.rszstd/src/bit_io/bit_reader_reverse.rszstd/src/bit_io/bit_reader_reverse/tests.rszstd/src/bit_io/bit_writer.rszstd/src/bit_io/bit_writer/tests.rszstd/src/cpu_kernel.rszstd/src/cpu_kernel/tests.rszstd/src/decoding/block_decoder.rszstd/src/decoding/block_decoder/tests.rszstd/src/decoding/buffer_backend.rszstd/src/decoding/buffer_backend/tests.rszstd/src/decoding/decode_buffer.rszstd/src/decoding/decode_buffer/tests.rszstd/src/decoding/dictionary.rszstd/src/decoding/dictionary/tests.rszstd/src/decoding/errors.rszstd/src/decoding/errors/tests.rszstd/src/decoding/exec_sequence_inline.rszstd/src/decoding/exec_sequence_inline/inline_helper_tests.rszstd/src/decoding/exec_sequence_inline/portable_helper_tests.rszstd/src/decoding/flat_buf.rszstd/src/decoding/flat_buf/tests.rszstd/src/decoding/frame_decoder.rszstd/src/decoding/frame_decoder/tests.rszstd/src/decoding/frame_inspection_tests.rszstd/src/decoding/literals_section_decoder.rszstd/src/decoding/literals_section_decoder/burst_gate_tests.rszstd/src/decoding/literals_section_decoder/zerocopy_robustness_tests.rszstd/src/decoding/mod.rszstd/src/decoding/ringbuffer.rszstd/src/decoding/ringbuffer/tests.rszstd/src/decoding/scratch.rszstd/src/decoding/scratch/tests.rszstd/src/decoding/sequence_execution.rszstd/src/decoding/sequence_execution/tests.rszstd/src/decoding/sequence_section_decoder.rszstd/src/decoding/sequence_section_decoder/tests.rszstd/src/decoding/simd_copy.rszstd/src/decoding/simd_copy/tests.rszstd/src/decoding/streaming_decoder.rszstd/src/decoding/streaming_decoder/tests.rszstd/src/decoding/user_slice_buf.rszstd/src/decoding/user_slice_buf/tests.rszstd/src/dictionary/fastcover.rszstd/src/dictionary/fastcover/tests.rszstd/src/dictionary/frequency.rszstd/src/dictionary/frequency/tests.rszstd/src/dictionary/mod.rszstd/src/dictionary/reservoir.rszstd/src/dictionary/reservoir/tests.rszstd/src/dictionary/tests.rszstd/src/encoding/block_header.rszstd/src/encoding/block_header/tests.rszstd/src/encoding/blocks/compressed.rszstd/src/encoding/blocks/compressed/tests.rszstd/src/encoding/bt/ldm_helper_tests.rszstd/src/encoding/bt/mod.rszstd/src/encoding/compress_bound_tests.rszstd/src/encoding/cparams.rszstd/src/encoding/cparams/tests.rszstd/src/encoding/dfast/extend_with_repcode_tests.rszstd/src/encoding/dfast/mod.rszstd/src/encoding/dict_attach.rszstd/src/encoding/dict_attach/tests.rszstd/src/encoding/fastpath/mod.rszstd/src/encoding/fastpath/neon.rszstd/src/encoding/fastpath/neon/tests.rszstd/src/encoding/fastpath/scalar.rszstd/src/encoding/fastpath/scalar/tests.rszstd/src/encoding/fastpath/simd128.rszstd/src/encoding/fastpath/simd128/tests.rszstd/src/encoding/fastpath/tests.rszstd/src/encoding/frame_compressor.rszstd/src/encoding/frame_compressor/tests.rszstd/src/encoding/frame_header.rszstd/src/encoding/frame_header/tests.rszstd/src/encoding/hc/generator.rszstd/src/encoding/hc/generator/tests.rszstd/src/encoding/hc/hc_tests.rszstd/src/encoding/hc/mod.rszstd/src/encoding/hc/optimal.rszstd/src/encoding/hc/priceset.rszstd/src/encoding/hc/priceset/tests.rszstd/src/encoding/incompressible.rszstd/src/encoding/incompressible/tests.rszstd/src/encoding/ldm/gear_hash.rszstd/src/encoding/ldm/gear_hash/tests.rszstd/src/encoding/ldm/mod.rszstd/src/encoding/ldm/params.rszstd/src/encoding/ldm/params/tests.rszstd/src/encoding/ldm/search.rszstd/src/encoding/ldm/search/tests.rszstd/src/encoding/ldm/table.rszstd/src/encoding/ldm/table/tests.rszstd/src/encoding/ldm/tests.rszstd/src/encoding/levels/config.rszstd/src/encoding/levels/fastest.rszstd/src/encoding/levels/fastest/tests.rszstd/src/encoding/levels/mod.rszstd/src/encoding/match_generator.rszstd/src/encoding/match_generator/dict_prime.rszstd/src/encoding/match_generator/mod.rszstd/src/encoding/match_generator/tests.rszstd/src/encoding/match_table/storage.rszstd/src/encoding/match_table/storage/bt_pair_index_wrap_tests.rszstd/src/encoding/match_table/storage/storage_tests.rszstd/src/encoding/match_table/storage/stream_abs_headroom_tests.rszstd/src/encoding/mod.rszstd/src/encoding/parameters.rszstd/src/encoding/parameters/tests.rszstd/src/encoding/row/mod.rszstd/src/encoding/row/neon_tag_mask_tests.rszstd/src/encoding/row/tag_mask_tests.rszstd/src/encoding/sequence_capture.rszstd/src/encoding/sequence_capture/tests.rszstd/src/encoding/simple/fast_kernel/count.rszstd/src/encoding/simple/fast_kernel/count/tests.rszstd/src/encoding/simple/fast_kernel/hash_table.rszstd/src/encoding/simple/fast_kernel/hash_table/tests.rszstd/src/encoding/simple/fast_kernel/kernel.rszstd/src/encoding/simple/fast_kernel/kernel/tests.rszstd/src/encoding/simple/fast_matcher.rszstd/src/encoding/simple/fast_matcher/tests.rszstd/src/encoding/strategy.rszstd/src/encoding/strategy/tests.rszstd/src/encoding/streaming_encoder.rszstd/src/encoding/streaming_encoder/tests.rszstd/src/encoding/util.rszstd/src/encoding/util/tests.rszstd/src/fse/fse_decoder.rszstd/src/fse/fse_decoder/tests.rszstd/src/fse/mod.rszstd/src/fse/tests.rszstd/src/histogram.rszstd/src/histogram/tests.rszstd/src/huff0/huf_cstream.rszstd/src/huff0/huf_cstream/tests.rszstd/src/huff0/huff0_decoder.rszstd/src/huff0/huff0_decoder/tests.rszstd/src/huff0/huff0_encoder.rszstd/src/huff0/huff0_encoder/tests.rszstd/src/huff0/mod.rszstd/src/huff0/tests.rszstd/src/skippable.rszstd/src/skippable/tests.rs
Replace the lone CWD-relative `std::fs::read("./dict_tests/dictionary")`
with the `include_bytes!("../../../dict_tests/dictionary")` the rest of
the file already uses, so the test no longer depends on the harness
working directory.
The `#[cfg(feature = "bench_internals")]` white-box capture helpers (`huf_weight_description_for_test`, `huf_encode4x_for_test`, `level22_block_ranges` / `merge_block_delimiters` / `collect_level22_sequences[_with_delimiters]`) are bench facade reached from the crate-root `bench_internals` surface, not test cases. The test sweep had swept them into the `tests` submodules, so the `bench_internals` clippy build could no longer resolve them. Move them back to `huff0_encoder` / `match_generator` production (restoring the `super::cost_model` paths the deeper nesting had rewritten).
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
`fse::round_trip` and its `check_tables` helper are gated `#[cfg(any(test, feature = "fuzz_exports"))]` and consumed by the `fse` fuzz target through the crate's public surface, not test cases. The test sweep moved them into `fse/tests.rs` (compiled only under `test`), so the `fuzz_exports` build could no longer resolve `round_trip` and the encoder's `into_table` / `encode` went dead. Move both back to `fse/mod.rs`.
This was referenced Jun 23, 2026
Closed
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related refactors, no production logic changes (compress/decompress output is byte-identical; the suite runs the same 824 cases throughout).
1. Split the
match_generator.rsmonolith (closes #441)The ~13k-line driver is broken into an orchestrator plus per-concern modules:
match_generator/mod.rsencoding/levels/config.rsencoding/hc/priceset.rsencoding/hc/generator.rsHcMatchGenerator+ chain matcherencoding/hc/optimal.rsmatch_generator/dict_prime.rsdict_primeis a child module so it reaches the driver's private, invariant-bearing snapshot state (primed/reset_shape/PrimedKey) directly, with zero field-visibility widening; theMatchertrait surface forwards to it via thin#[inline]delegations.2. Separate test code from production across the crate
Every inline test module and module-scope
#[test]is relocated into a sibling test file, leaving production sources with code only plus a#[cfg(test)] mod ...;declaration. Covers ~75 files acrosszstdandcli. Test-support items that are methods/fields on production types (test-only accessors over private state) intentionally stay with their type.Testing
cargo nextest run -p structured-zstd --features dict_builder— 824 passedcargo nextest run -p structured-zstd-cli— 22 passedcargo clippy --tests+cargo fmtcleancargo check --workspace --testscleanx86_64-apple-darwincross-checked for the arch-gated tests (row tag-mask, reverse-bit-reader BMI2 helper) that the local aarch64 build compiles outCloses #441
Summary by CodeRabbit
Tests
Refactor