Spent about an hour and I see some useful commands that extend past what “cut” can do. But really when dealing with printf() format statements is anyone using awk scripts for this?
Or is everyone just using their familiar scripting language. I’d reach for Python for the problems being presented as useful for awk.
awk is pretty damn solid. When I was completely rewriting the gravity.sh script from Pi-hole about six years back, it was easily the fastest for parsing and uniquely sorting content from files with a couple million lines. It made things much more usable on Raspberry Pi Zero hardware, since changing to another language like Python was out of the question.
awk is awesome! I love it, and I do not regret learning how to use it.
That said, my workflow invariably always shifts from starting with awk to do something simply with a tiny one-liner, to then doing that with perl or python, and sometimes even creating a file to make the by-now multi-line scripts more easily readable.
I do not recommend starting with awk, if you do not know other languages already such as Python.
I think it’s pretty niche but is a great tool for parsing / converting data into a format that is more easily digested by another program.
Think for example a report from an 80’s system that spits out many tab separated values in a different format based on some code. Then these tables are all separated by two blank lines and order of them is randomised. To top that off you need to then pipe it all to a different program that only accepts a specific format.
You could do it in Python by doing a parse, process, stringify code but if you know awk you can do all those steps at the same time with less code.
Sure, in the age of REST the Python approach is better but awk is a very powerful tool for the “I have a specific output but need a specific input” problem.
Every day. piping srdout to a combination of awk and sed makes shell operations a lot easier. A lot of my earlier perl hacks have now been replaced by a combination of awk, sed, and xargs
awk predates perl as well as python by a pretty large margin (1978); it’s useful, of course, for processing things in a pipeline, but as it became obsolete as a general-purpose scripting language, users have had less and less of a reason to learn its syntax in detail – so nowadays it shows up in one-liners where it could be replaced by a tiny bit of cut.
I had worked through a good bit of the O’Reilly ‘sed & awk’ book – the first programming book I got, after being enticed by shell scripting in general. Once I learned a bit of Python, & got better at vim scripting, though, I started using it less and less; today I barely remember its syntax.
awk will always have a soft spot for me, but I can see why not many take the time to learn it. It tends to be needed right there at the border of problem complexity where you are probably better using a full-fledged scripting tool.
But learning awk is great for that “now you’re thinking in pipes” ah-hah moment.
Yes! Awk is great, I use it all the time for text processing problems that are beyond the scope of normal filters but aren’t worth writing a whole program for. It’s pretty versatile, and you can split expressions up and chain them together when they get too complicated. Try piping the output into sh sometime. It can be messy though and my awk programs tend to be write-only
I use awk all the time. a very common and probably simplest reason I use it is it’s ability to handle variable column locations for data.
if you know you always want the last field you can do something like
awk ‘{print $NF}’
but usually using it as for performing more advanced operations all in one go without having to pipe something three times.
sure you can do grep cut grep printf, but you can instead do the pattern matching, the field, the formatting, whatever you need to all in one place.
it’s also got a bunch of more advanced usage of course since it’s its own language. one of my favorite advanced one liners is one that will recognize if it is going to print a duplicate line anywhere in your output and prevent it. in cases where you don’t want to sort your output but you also want to remove duplicates it is extremely helpful and quick rather than running post-processing on the output in another way.
all that said main reason I use it is because I know it and it’s fast, there’s nothing you can do in awk that you can’t do in Python or whatever else you’re more comfortable with. The best tool for the job is the one that gets it done quickly and accurately. unless your environment is limited and it prevents the installation of tools you’re more familiar with then there’s no real reason to use this over Python.
I use awk a good bit. Not as much as when I was doing data work, though. It’s better than cut at splitting on whitespace in many contexts. Lots of other nice uses like variable scope, regex, and summing
Add comment