Problem
After the small-frame FSE-table-selection fix (PR #433), Fast/dfast dictionary frames are at or below libzstd parity on both arches. The remaining dictionary-path gap is at the Row / lazy backends (levels ~6-9), which stay 2-3x slower than libzstd and also lose a little ratio.
Authoritative compare numbers (ours vs libzstd compress_usingCDict, same host, dict_matrix example, raw-content 8 KiB dict, dict-id flag off):
| fixture |
lvl |
i9 (x86) sp |
M1 (aarch64) sp |
ratio |
| logs-1k |
6 |
2.75x |
2.10x |
1.44x (26 vs 18 B) |
| logs-1k |
9 |
2.02x |
2.00x |
1.00x |
| logs-4k |
9 |
2.05x |
1.94x |
1.00x |
| logs-16k |
9 |
1.80x |
1.86x |
1.00x |
(sp = ours_us / c_us; >1 = slower. ratio = our_bytes / c_bytes.)
Scope
- Profile the L6 (Row/greedy-lazy) and L9 (HashChain/lazy2) dictionary path on both arches (x86 BMI2 on the i9, NEON on the M1 — a same-hardware ours-vs-libzstd delta is authoritative per arch).
- L6 also shows a ratio gap (we emit 26 vs libzstd's 18 bytes on logs-1k): the lazy matcher finds fewer/shorter dictionary matches on the dict-primed chains. Ratio is the first priority.
- Likely targets: the Row tag-mask SIMD coverage, dict-primed chain traversal depth, and the lazy carry-forward decision on dict matches.
Acceptance
- L6/L9 dictionary throughput within ~1.2x of libzstd on the bench fixtures (both arches), ratio >= libzstd.
- No regression on the Fast/dfast/BT dictionary paths or non-dict levels.
Follow-up to PR #433 (which fixed the 5.5x copy-mode Fast dict ratio bug, the resident-dict retained-budget over-window bug, and the small-frame FSE-selection hotspot).
Problem
After the small-frame FSE-table-selection fix (PR #433), Fast/dfast dictionary frames are at or below libzstd parity on both arches. The remaining dictionary-path gap is at the Row / lazy backends (levels ~6-9), which stay 2-3x slower than libzstd and also lose a little ratio.
Authoritative
comparenumbers (ours vs libzstdcompress_usingCDict, same host,dict_matrixexample, raw-content 8 KiB dict, dict-id flag off):(
sp= ours_us / c_us; >1 = slower.ratio= our_bytes / c_bytes.)Scope
Acceptance
Follow-up to PR #433 (which fixed the 5.5x copy-mode Fast dict ratio bug, the resident-dict retained-budget over-window bug, and the small-frame FSE-selection hotspot).