Get the lines unique on the first field(s)

uniq is a great tool, since it returns the unique (adjacent) lines of the given input. But it has a limitation: it can't check for uniqueness only the first N fields (while it allows to ignore them, weird).

So, what to do if you have a long file, and lines with several fields, but you're only interested in getting the line with the different first 2 field (but all the rest of the line content? awk to the rescue!

$ awk '!x[$1]++' file

will print the (complete) lines of file that has the first field unique. You can set $1$2 to have lines unique on the first 2 fields, and so on. Thanks to this forum post, but there's some other interesting articles.

Trick of today: find -daystart

What to get the files older than today? Run

    find /path/ -type f -daystart -mtime +0

it will return only the files older than today, no matter the time the command is executed (by default, -mtime counts multiples of 24 hours from now). Kinda nice when you want to archive yesterday log files.

Print a NUL-terminated string with awk

I thought it would have been easier to print a NUL-terminated string in awk (mawk as it's the default in Debian), but after some trial-and-fail I was able to come up with this kinda ugly solution:


$ echo -e "123\n456" | awk 'BEGIN { ORS="" } { print $0 ; printf("%c", "") }'  | xargs -n1 -0 echo 
123
456

That:

• set the Output Records Separator (ORS) to the empty string (default is the \n, new line)
• print the input line (do your elaboration there, if you need)
• print the null character, as explained in the mawk manpage: "mawk cannot handle ascii NUL \0 in the source or data files.  You can output NUL using printf with %c, and any other 8 bit character is acceptable input."
• show that it's actually emitting NUL-terminated strings

There's really no better way to do that?