Sorting check sums created

[orbittwz]

When creating a check sum, the files aren't in order...
This has been discussed before, but hasn't been fixed yet.

[redactedscribe]

The trick is to first make sure that your file manager is sorted A-Z, and secondly, after selecting the files you want to hash, right-click the first selected file and then choose Create Checksum File. This will always produce a checksum with A-Z ordering.

It does make sense however that HashCheck should sort the items to be hashed before hashing them so that the checksum would always be A-Z ordered no matter the user's order of actions. In fact, I think it may already do this, it's just that right-clicking any other selected item but the first one causes this bug (@idrassi).

[idrassi]

HashCheck hashes files based on the order they are received from Windows Explorer. But when operating on an SSD, HashCheck utilizes multithreading to enhance the hashing speed by processing multiple files concurrently. This, however, means that the output order can vary depending on factors like file size and thread completion times.

Concerning the suggested workaround of sorting in Explorer, it will work only on traditional HD and it will have no effect when multithreading is used.

Based on this, the potential solutions are:

• Disable Multithreading: This would slow down the computation, which might not be preferable for users, especially with large datasets.
• Post-Processing Sorting: After all checksums are calculated, we could sort the entries in the checksum file. This ensures a consistent order with minimal performance overhead since the sorting process is much quicker than hashing.

Given the trade-offs, the second option seems to be more efficient and user-friendly. This will ensure that the checksum file entries are sorted without compromising the hashing speed.

I will further evaluate the feasibility of this solution and I will let you know.

[orbittwz]

Hey!
Thanks for answering so thoroughly but the question, does the program actually does it?
Being said, I already went back to OpenHashTab since the sorting problem has been solved also there...
Maybe you can get some inspiration or contact the author for help with your program?
https://github.com/namazso/OpenHashTab

[redactedscribe]

Given the trade-offs, the second option seems to be more efficient and user-friendly. This will ensure that the checksum file entries are sorted without compromising the hashing speed.

This would be ideal. A sort before write shouldn't be any noticeably slower.

HashCheck hashes files based on the order they are received from Windows Explorer. But when operating on an SSD, HashCheck utilizes multithreading to enhance the hashing speed by processing multiple files concurrently.

Also, it doesn't look like there's any indication when HashCheck is hashing in parallel on an SSD, but it may be useful to communicate this. Either in the README or on the UI. I personally haven't noticed HashCheck changing its hashing behaviour between HDD/SSD, but now knowing this I'll be using my SSDs.

Additionally, an indication of how many files have been hashed out of the total to be processed would be good, e.g. "(5/16)" or "(1/1)" for a single file in the title bar.

[ZPNRG]

@idrassi Thank you for your comment above explaining the SSD/multithreading issue at work. I've used HashCheck since 2009 (the original Kai Liu version) and then eventually switched to Gurnec's fork in 2016 before switching to your fork in 2021. The checksum files being created with a very inconsistent order has been quite irritating, but I wasn't sure what was the cause of it and why sometimes it happened and sometimes it didn't. Knowing that the drive being used when they are created (SSD vs. spinning HD) is at play fully explains the results I see all the time.

I definitely vote for the "Post-Processing" option so that the order of files in the checksum file created is the same as the order of the files on disk that the checksum file is created from.

[Ronald-G-Andrade]

Post-Processing Sorting: After all checksums are calculated, we could sort the entries in the checksum file. This ensures a consistent order with minimal performance overhead since the sorting process is much quicker than hashing.

Given the trade-offs, the second option seems to be more efficient and user-friendly. This will ensure that the checksum file entries are sorted without compromising the hashing speed.

I also vote for the "Post-Processing Sorting", it's definetly more efficient. I really hope to have good news about this implementation. Thank you dear @idrassi for your great ideas.

Additionally, an indication of how many files have been hashed out of the total to be processed would be good, e.g. "(5/16)" or "(1/1)" for a single file in the title bar.

Oh, and as @redactedscribe said: it would be good if HashCheck Shell Extension have an indicator of how many files have been hashed out of the total to be processed, like "(108/2700)".

[ZPNRG]

@Ronald-G-Andrade I'm not sure that @idrassi will ever update this again and I certainly wouldn't hold my breath.

[Ronald-G-Andrade]

@Ronald-G-Andrade I'm not sure that @idrassi will ever update this again and I certainly wouldn't hold my breath.

Well, we should never lose hope. Because the app is very useful, there are a few improvements that if they are implemented, it would be perfect.

[ZPNRG]

@Ronald-G-Andrade, I agree that the app only needs some improvements, but the trend is concerning. Kai Liu's original work has been forked and used by many, but all seem to do the same thing: get in, make a few changes, and then abandon the projects relatively quickly. It isn't like any of the developers formally abandoned either. They just demonstrate the projects are abandoned by silence and lack of development.

[Ronald-G-Andrade]

@ZPNRG It's true dear mate. Yesterday I saw the forks about the original developer (Kai Liu), and only @gurnec and @idrassi made really good forks. I really don't know why they simply abandoned the project without archiving or making an announcement.

• Kai Liu (original developer): https://code.kliu.org/hashcheck/
  Latest version: 2.1.11 (2009-07-01)

• Christopher Gurnee (@gurnec): https://github.com/gurnec/HashCheck/releases
  Latest version: 2.4.0 (2016-09-07)

• Mounir Idrassi (@idrassi): https://github.com/idrassi/HashCheck/releases
  Latest version: 2.5.0 (2021-01-27)

Another detail is that @idrassi continue his development in DirHash app (https://github.com/idrassi/DirHash), but that is a CLI (needs CMD, PowerShell or Terminal), it doesn't have GUI or Shell Integration like HashCheck Shell Extension. The latest version is 1.26.1 (from 2024-07-20).

[NintendoManiac64]

My only request if that the sorting process be able to be done in a timely fashion even if/when hashing a million files (common with Linux file systems), i.e. presumably sorted and/or post-processed while the actual hashing process is occurring so as to avoid the issues that the program "ExactFile" has.

I previously benchmarked SHA1 on a 2core/2thread Pentium G3258 @ ~4.5GHz (basically an overclockable Haswell i3 without SMT, no AVX, and a cut-down iGPU) with 264 thousand files taking 400GB which took 5 hours to fully finish in "ExactFile", much of it being the post-processing sorting process that happens after it hashes everything - meanwhile HashCheck took a bit over 30 minutes (and, yes, ExactFile is also multi-threaded).