feature: flag to preserve original input order

For some use cases, it would be beneficial to retain the original input order.

(It's understood that some speed may need to be sacrified. It's also understood that the increased code complexity may be more than you want to maintain!)

For now, using flag -t 1 (single thread) will maintain the original 1:1 order.

Well, yeah ... but it also drastically reduces performance (of course). 😁

But ... combining -t 1 with GNU parallel's --keep-order flag, parallel will do the work of reassembling the outputs in the correct order. So that's a workaround that preserves performance.

This is something I previously looked at, but performance was not what I wanted it to be, so a keep order option was scrapped. I'll put this back on the todo list.

That's generous, but maybe the workaround is good enough, and your time would be better spent on something with higher payoff. 😅

About to push a PR which adds indexing to keep the output order the same as input. Negligible performance hit.

$ sha1sum rockyou.txt 
768abc17bafbce37bf3cf1e946f6d9970428b7b3  rockyou.txt

# v1.1.5-dev; 2025-09-09.1600 (with indexing)
$ go run hashgen.go -m plaintext -w rockyou.txt -o tmp.txt
2025/09/09 15:50:49 Starting...
2025/09/09 15:50:49 Processing file: rockyou.txt
2025/09/09 15:50:49 Hash function: plaintext
2025/09/09 15:50:49 CPU Threads: 16
2025/09/09 15:50:49 Finished processing 14344391 lines in 0.378 sec (37.978 M lines/sec)

$ sha1sum tmp.txt
768abc17bafbce37bf3cf1e946f6d9970428b7b3  tmp.txt

# v1.1.4 (single threaded to maintain input/output order)
$ hashgen -t 1 -m md5 -w rockyou.txt -o tmp.txt 
2025/09/09 15:59:20 Starting...
2025/09/09 15:59:20 Processing file: rockyou.txt
2025/09/09 15:59:20 Hash function: md5
2025/09/09 15:59:20 CPU Threads: 1
2025/09/09 15:59:23 Finished processing 14344391 lines in 3.564 sec (4.025 M lines/sec)

$ sha1sum tmp.txt 
1e6844b0063da081abd394a46926b55e5113d5a1  tmp.txt

# v1.1.5-dev; 2025-09-09.1600 (with indexing)
$ go run hashgen.go -m md5 -w rockyou.txt -o tmp.txt
2025/09/09 16:02:59 Starting...
2025/09/09 16:02:59 Processing file: rockyou.txt
2025/09/09 16:02:59 Hash function: md5
2025/09/09 16:02:59 CPU Threads: 16
2025/09/09 16:03:00 Finished processing 14344391 lines in 0.471 sec (30.448 M lines/sec)

$ sha1sum tmp.txt
1e6844b0063da081abd394a46926b55e5113d5a1  tmp.txt

added feature: "keep-order" from #7

Slick! Thanks very much!

You're welcome!

Navigation Menu

Repository navigation

feature: flag to preserve original input order #7

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

feature: flag to preserve original input order #7

Description

Activity

cyclone-github commented on Sep 9, 2025

roycewilliams commented on Sep 9, 2025

cyclone-github commented on Sep 9, 2025

roycewilliams commented on Sep 9, 2025

cyclone-github commented on Sep 9, 2025

roycewilliams commented on Sep 11, 2025

cyclone-github commented on Sep 11, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions