Benchmark: cut characters in bash • 25 Mar 2022
Post #1 in this bash benchmark series,
measuring the speed of common bash text manipulations.
Calculate hit rate from a log file • 21 Oct 2004
You have a huge file that contains one line per request/transaction. Some lines are of one type (e.g. ‘HIT’), some of another (e.g. MISS). Let’s say you want to calculate the hit-rate, but as fast as possible.
We take a Squid log file of about 140MB. How long does it take to count how many lines it has?
$ time wc -l /var/log/squid/access.log
845212 /var/log/squid/access.log
real 0m6.523s
(about 21.4 MB/s or 130.000 lines/s)