Skip to content

Fix ROLZ inverse overread and transform test crashes#38

Merged
flanglet merged 1 commit into
flanglet:masterfrom
BICHENG:master
Jun 12, 2026
Merged

Fix ROLZ inverse overread and transform test crashes#38
flanglet merged 1 commit into
flanglet:masterfrom
BICHENG:master

Conversation

@BICHENG

@BICHENG BICHENG commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Fixes #37.

This fixes the crash from #37. I hit it through 7zFM + 7-Zip-zstd, then reduced it to kanzi-cpp.

Changes

  • ROLZCodec1::inverse() now passes only the encoded bytes left after srcIdx to the bitstream reader. Small ROLZ blocks no longer get a 64 KiB input window when fewer bytes are available.
  • TestTransforms.cpp moves the 1 MiB temporary buffer from the stack to heap storage.
  • Transform tests release the current transform object on early returns.

Checks

On upstream commit f5839572, an MSVC x64 build can reproduce the crash with:

testTransforms -type=ROLZ -noperf
testTransforms -type=SRT -noperf
testTransforms -type=RANK -noperf
testTransforms -type=all -noperf

On this branch, the same commands pass.

The CI check does this in two steps:

  • first it builds f5839572 with MSVC x64 and expects at least one of those transform commands to fail;
  • then it builds this branch and expects all of them to pass.

That ties the CI check directly to #37.

It also compresses and decompresses generated random, structured, and mixed 4 MiB files with Kanzi CLI levels 1 to 9. SHA-256 hashes match after decompression.

Covered platforms: Ubuntu gcc, Ubuntu clang, macOS clang, MSVC x64, MSVC x86.

CI run:
https://github.com/BICHENG/kanzi-cpp/actions/runs/27264743969

Fix ROLZCodec1::inverse() so ifixedbuf only receives the encoded input bytes that remain after srcIdx. Small ROLZ chunks no longer expose a synthetic 64 KiB read window to the decoder.

Move the transform test scratch buffer from the Windows stack to heap storage, and release FileTransformSequence on early exits so the harness does not leak state while running failure paths.

Extend Kanzi quality CI into a regression contrast: build a known-bad MSVC x64 baseline and require the transform suite to fail there, build the fixed head and require it to pass, then run transform and CLI stream round-trip checks across gcc, clang, macOS, MSVC x64, and MSVC x86. The stream round trips now cover compression levels 1 through 9.

Keep native-transform-quality.md out of the repository so it remains a local working note.
@BICHENG BICHENG force-pushed the master branch 3 times, most recently from 7bc4c95 to c982054 Compare June 11, 2026 09:47
@BICHENG BICHENG changed the title Fix ROLZ inverse out-of-bounds read and test stack overflow Fix SliceArray handling in Kanzi transforms Jun 11, 2026
@BICHENG

BICHENG commented Jun 11, 2026

Copy link
Copy Markdown
Contributor Author

Cleaned up the PR a bit
The ROLZ crash led to the SliceArray fix here. The SRT/RANK/all crashes look separate after another check, so I am leaving the test stack issue out of this PR.

@flanglet

Copy link
Copy Markdown
Owner

Hey, thanks for the changes.
I would have merged the previous PR but I will not merge this one. It is way too big and touches too many files (Lines changed: 583 additions & 191 deletions). There is too much code churn even for cosmetic reasons. I want to fix the actual transform issues where they happen.
What are the SRT/RANK/all crashes? Do they happen in the test code only?

@sonarqubecloud

Copy link
Copy Markdown

@BICHENG

BICHENG commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Yeah... I made this PR too broad.
Pushed it back to the smaller version.

The SRT/RANK/all crashes look like the test program using too much stack on Windows, so I kept that as a test-side fix.

@BICHENG BICHENG changed the title Fix SliceArray handling in Kanzi transforms Fix ROLZ inverse overread and transform test crashes Jun 12, 2026
@flanglet flanglet merged commit 7d4f6d3 into flanglet:master Jun 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ROLZ inverse out-of-bounds read and stack overflow

2 participants