Repository navigation
feature: flag to preserve original input order #7
Description
Activity
For now, using flag -t 1 (single thread) will maintain the original 1:1 order.
Well, yeah ... but it also drastically reduces performance (of course). 😁
But ... combining
-t 1with GNU parallel's--keep-orderflag,parallelwill do the work of reassembling the outputs in the correct order. So that's a workaround that preserves performance.This is something I previously looked at, but performance was not what I wanted it to be, so a keep order option was scrapped. I'll put this back on the todo list.
That's generous, but maybe the workaround is good enough, and your time would be better spent on something with higher payoff. 😅
About to push a PR which adds indexing to keep the output order the same as input. Negligible performance hit.
$ sha1sum rockyou.txt 768abc17bafbce37bf3cf1e946f6d9970428b7b3 rockyou.txt# v1.1.5-dev; 2025-09-09.1600 (with indexing) $ go run hashgen.go -m plaintext -w rockyou.txt -o tmp.txt 2025/09/09 15:50:49 Starting... 2025/09/09 15:50:49 Processing file: rockyou.txt 2025/09/09 15:50:49 Hash function: plaintext 2025/09/09 15:50:49 CPU Threads: 16 2025/09/09 15:50:49 Finished processing 14344391 lines in 0.378 sec (37.978 M lines/sec) $ sha1sum tmp.txt 768abc17bafbce37bf3cf1e946f6d9970428b7b3 tmp.txt
# v1.1.4 (single threaded to maintain input/output order) $ hashgen -t 1 -m md5 -w rockyou.txt -o tmp.txt 2025/09/09 15:59:20 Starting... 2025/09/09 15:59:20 Processing file: rockyou.txt 2025/09/09 15:59:20 Hash function: md5 2025/09/09 15:59:20 CPU Threads: 1 2025/09/09 15:59:23 Finished processing 14344391 lines in 3.564 sec (4.025 M lines/sec) $ sha1sum tmp.txt 1e6844b0063da081abd394a46926b55e5113d5a1 tmp.txt# v1.1.5-dev; 2025-09-09.1600 (with indexing) $ go run hashgen.go -m md5 -w rockyou.txt -o tmp.txt 2025/09/09 16:02:59 Starting... 2025/09/09 16:02:59 Processing file: rockyou.txt 2025/09/09 16:02:59 Hash function: md5 2025/09/09 16:02:59 CPU Threads: 16 2025/09/09 16:03:00 Finished processing 14344391 lines in 0.471 sec (30.448 M lines/sec) $ sha1sum tmp.txt 1e6844b0063da081abd394a46926b55e5113d5a1 tmp.txt- added a commit that references this issue
on Sep 9, 2025 Slick! Thanks very much!
Reacted by cycloneYou're welcome!
For some use cases, it would be beneficial to retain the original input order.
(It's understood that some speed may need to be sacrified. It's also understood that the increased code complexity may be more than you want to maintain!)