cksum: failing GNU tests

[jtracey]

The latest version of GNU's tests add tests/misc/cksum, tests/misc/cksum-a, tests/misc/cksum-c, and tests/misc/sm3sum, all of which currently fail. This will track the respective problems from each.

[str4d]

uu_cksum correctly implements cksum 8.30, which only has --help and --version. However, the GNU Coreutils 9.0 docs for cksum list the following options:

• -a/--algorithm
• --debug
• --untagged
• All options supported by b2sum:
  • -l/--length
  • All options supported by md5sum:
    • -b/--binary (unsupported for cksum, always on)
    • -c/--check
      • cksum auto-detects the algorithm if given data in the --tag format.
      • cksum does not support --check with -a sysv, -a bsd, or -a crc.
    • --ignore-missing
    • --quiet
    • --status
    • --tag
      • cksum uses this as its default output format (use --untagged for old format).
    • -t/--text (unsupported for cksum, always off)
    • -w/--warn
    • --strict
    • -z/--zero

So uu_cksum is correct for Coreutils 8.30 compatibility, but for Coreutils 9.0 compatibility, we need to flesh it out.

[str4d]

Investigating the codebase a bit further, I see that both md5sum and b2sum are implemented via hashsum. This suggests we should make cksum also be implemented by hashsum. I've checked off above all the options currently implemented by hashsum.

[str4d]

Looks like test/misc/b2sum is also failing, both because hashsum doesn't support -l/--length, and cksum doesn't support -a blake2b.

I'm going to try implementing this.

[str4d]

I made a bunch of progress on this and then ran out of time to finish the PRs. Picking this back up now; I'll rebase my current local work and get some initial PRs up.

[str4d]

Ugh, this is annoying: b3sum was added in #3108, and its corresponding upstream binary defines -l/--length in bytes, whereas GNU Coreutils b2sum defines -l/--length in bits.

There are three options here:

• Define -l/--length as bytes to match upstream b3sum. Given that compatibility with GNU Coreutils is being prioritised (per this issue), I think that would not be considered acceptable as b2sum would then never pass GNU tests.
• Define -l/--length as bits to match GNU Coreutils. I really dislike this option, for two reasons:
  • --length being bits means it needs to be a multiple of 8, so 7/8 of values that upstream b3sum accepts will be rejected here. That's not so bad, it makes the difference discoverable.
  • In the cases where an upstream length is a multiple of 8, b3sum here would output 1/8 of the length, e.g. instead of a 16-byte hash the user gets a 16-bit hash (2 bytes). This is a TERRIBLE silent security issue.
• Define -l/--length as bits when used with b2sum or cksum, and bytes when used with b3sum. This is awful for UX; in particular, it is undefined how hashsum should treat it. The least-bad option might be to just special-case the byte interpretation to b3sum, and not allow it to be used with -a blake3 on hashsum (otherwise hashsum would almost certainly end up conflicting with cksum if the latter adds BLAKE3 support).

We have a somewhat-similar issue in that hashsum itself defines a separate --bits flag for its SHA3 and SHAKE support, which is effectively duplicative of GNU Coreutils -l/--length. I'm not going to try solve that here.

[str4d]

cc @oconnor663 who wrote b3sum