Monthly Archive for November, 2004

Page 2 of 2

Squid cachemgr.cgi UI hack

Published on November 8, 2004 in Linux. 1 Comment

Squid has a little system statistics viewer built-in:

The cache manager (cachemgr.cgi) is a CGI utility for displaying statistics about the squid process as it runs. The cache manager is a convenient way to manage the cache and view statistics without logging into the server.
(from Squid FAQ)

The only thing is … it’s so ugly! It uses plain HTML and cannot be customized, the FAQ says. However, there is a way to do it:

  1. copy cachemgr.cgi to cachemgr2.cgi so if you do something wrong, the original is not lost.
  2. open the CGI file in a text-editor. I used vi, but if you’re not used to working with it, use something else (emacs?).
  3. in the binary file, look for some text portions that look like HTML code
  4. while keeping in mind that the # of characters should remain the same, change the <title> and <style> to something that suits you. You will have to do this at 2 locations in the file: one for the homepage template and one for the other pages’ template.
  5. suggestion: just let the CGI use a style.css file that you drop into the same folder.
    <link rel="stylesheet" type="text/css" href="style.css" mce_href="style.css" /> and fill up with spaces to keep the same # characters
  6. verify that the cachemgr and the cachemgr2 have the same # bytes
  7. now use cachemgr2 to display your statistics.

    8. I did something a bit different (I wanted to use the CSS of my own website), so I ‘ll show you the difference between the two versions.
    In order to get to the following comparison, I did a strings cachemgr.cgi > cachemgr.txt to extract only the text parts, and I did a diff cachemgr.txt cachemgr2.txt to compare both files. You cannot do a file comparison of 2 binary files.

    <em>173,174c173,174</em>
    < <HTML><HEAD><TITLE>Cache Manager Interface</TITLE>
    < <STYLE type="text/css"><!-- BODY{background-color:#ffffff;font-family:verdana,sans-serif} --></STYLE></HEAD>
    ---
    > <HTML><HEAD><TITLE>Cache Manager (pforret)</TITLE>
    > <link rel="stylesheet" type="text/css" href="http://www.forret.com/forret/forret.css" mce_href="http://www.forret.com/forret/forret.css" /> </HEAD>
    <em>199c199</em>
    < <STYLE type="text/css"><!-- BODY{background-color:#ffffff;font-family:verdana,sans-serif} TABLE{background-color:#333333;border:0pt;padding:0pt}TH,TD{background-color:#ffffff}--></STYLE>
    ---
    > <link rel="stylesheet" type=text/css href="http://www.forret.com/forret/forret.css" mce_href="http://www.forret.com/forret/forret.css"><!-- TABLE{background-color:#333333;border:0pt;padding:0pt} TH,TD{background-color:#ffffff}--></STYLE>

    Probe disk performance (MRTG)

    Published on November 3, 2004 in Linux. 0 Comments

    The hdparam can be used to monitor the throughput speed of a hard disk:
    # <strong>hdparm -tT /dev/hda</strong>
    /dev/hda:
    Timing buffer-cache reads: 888 MB in 2.00 seconds = 444.00 MB/sec
    Timing buffered disk reads: 20 MB in 3.30 seconds = 6.06 MB/sec

    This would be an interesting performance metric to see plotted against time. So let’s convert it to a format ready for MRTG.

    • The only numbers we need are the last ones: resulting speed. This can be parsed from the output as follows:
      #hdparm -tT /dev/hda | gawk -F = "/seconds/ { print $2}" 

      440.00 MB/sec   3.30 MB/sec
    • if we could suppose that the results will always be in “MB/sec”, we could parse out the numbers with
      (...) | gawk "{print $1}"
      and then add a line to our MRTG config files to adjust the units:
      kMG[_]: M,G,T,P,X
      But let’s say that KB/sec or GB/sec speeds are possible.
    • One gawk can do the conversion trick:
      #(...) | gawk "/GB/ {print $1*1000000000} /MB/ {print $1*1000000} /KB/ {print $1*1000}" 

      440000000 3300000
    • To have a complete MRTG-ready output, we also add the boot time on line 3 and the name of the MRTG output on line 4
    • Q: Do we need 2 gawks one after the other? Can’t one do it?
      A: You could do it in 1, I guess, but the parsing would be more complex. I use 2 because the FS (field separator) changes: the first gawk uses the ‘=’ character, the second uses the normal whitespace.

    Date formatting in GAWK: boot time

    Published on November 2, 2004 in Linux. 0 Comments

    I have one server with apparently an exceptional stability:
    # uptime

    3:45pm  up 524 days,  1:22,  1 user,  load average: 0.44, 0.16, 0.13

    Unfortunately I know this is not correct (I remember rebooting it some weeks ago). So what are other ways to get the date/time of the last boot?

    Looking at the RedHat manuals, the following thing should work too:
    # cat /proc/stat
    cpu 33813143 210619911 30093342 59435750
    cpu0 33813143 210619911 30093342 59435749
    (...)
    btime 1096157569
    (...)

    The btime gives us the last boot time in seconds since 1 Jan 1970. I can find and convert it with gawk:
    # gawk "/btime/{ print (`date +%s` - $2) / (3600 * 24.0) ,"days -",strftime("%a %b %d %H:%M:%S %Z %Y",$2)}" /proc/stat
    38.6473 days - Sun Sep 26 02:12:49 CEST 2004
    Which gives us an uptime of 38,6 days – that looks more like it!

    Another way of calculating the uptime:
    # gawk "/cpu/ {print $1,($2 + $3 + $4 + $5)/(3600 * 24 * 100)}" /proc/stat
    cpu 38.6515
    cpu0 38.6515
    Confirmation of the previous measurement!

    # cat /proc/uptime
    45282758.17 663091.26
    The first number is the # of seconds since last boot. The other one (idle time) we don’t need. What is that in days?
    # gawk "{print $1/(3600 * 24.0)}" /proc/uptime
    524.106

    This is where the wrong data is coming from! So I’ll ignore this data.

    Remark: This server is one of my oldest ones and is still running Redhat 7.2 (Enigma). Looks like this bug was fixed in later versions of RedHat, since none of my other servers have it.

    It’s the latency, stupid!

    Published on November 1, 2004 in bandwidth. 0 Comments

    While working on some bandwidth-related stuff (my bandwidth calculator), I came across an excellent article on “latency vs. bandwidth” by Stuart Cheshire. It was originally written in 1996, so focuses a lot on modems, but Fact 1, 2 and 4 are still valid.

    His points:

    Fact One: Making more bandwidth is easy

    You can just put enough slow connections in parallel to get a fast one.

    Fact Two: Once you have bad latency you’re stuck with it

    Parallel devices, compression, … nothing helps!

    Fact Three: Current consumer devices have appallingly bad latency

    Modems are evil (but now, with cable and ADSL, this is less of an issue)

    Fact Four: Making limited bandwidth go further is easy

    Compression and caching help a lot. (This article was written about the time MP3 was invented, but long before it became hugely popular. DivX came later, in 1999)

    The following calculation is eye-opening:

    # The distance from Stanford to Boston is 4320km.
    # The speed of light in vacuum is 300 x 10^6 m/s.
    # The speed of light in fibre is roughly 66% of the speed of light in vacuum.
    # The speed of light in fibre is 300 x 10^6 m/s * 0.66 = 200 x 10^6 m/s.
    # The one-way delay to Boston is 4320 km / 200 x 10^6 m/s = 21.6ms.
    # The round-trip time to Boston and back is 43.2ms.
    # The current ping time from Stanford to Boston over today’s Internet is about 85ms:
    [cheshire@nitro]$ ping -c 1 lcs.mit.edu
    PING lcs.mit.edu (18.26.0.36): 56 data bytes
    64 bytes from 18.26.0.36: icmp_seq=0 ttl=238 time=84.5 ms
    # So: the hardware of the Internet can currently achieve within a factor of two of the speed of light.

    Definitions of latency:

    Latency, a synonym for delay, is an expression of how much time it takes for a packet of data to get from one designated point to another

    techtarget.com

    Latency is the time a message takes to traverse a system

    wikipedia.org