dedup: Shell utility for deduplicating lines
Per the title, I hope I'm not the only who occasionally misses something like this? Well, after tackling a larger project, I ended up with lots of reusable C code I had written and I thought it'd be great to try it out on something completely different and learn a few new things on the way. Try it out if you want! (POSIX-only for now)
Backstory: I was inspecting the output of gcc -print-search-dirs and, after you parse and resolve all those paths, you end up with some duplicates (atleast on Debian 13). Traditional tools, such as uniq(1) and sort -u aren't well-suited for this. The former can only deduplicate adjacent lines and the latter changes the order, but preserving the order was important in this case!
Experienced AWK users are probably screaming at their monitors right now, since that would be a great tool for this, and indeed it is, but where's the fun in writing one line of code? :) In all seriousness, I feel like a standalone tool for such task fits well within the UNIX tradition of using a shell to compose small programs. GNU AWK on my system is a 850K binary (within 500K of bash)!
Okay, that's it. Cheers!