Feed-based automatic download/caching

Interesting idea on HubLog (actually it’s an idea of Les Orchard).

I’d love to pay a monthly fee to have shows by Joss Whedon stream on down to my file-server with BitTorrent. I’d love to subscribe to favorite indie bands’ releases and have them show up in the music folder. I’m cheap, so I’d like the price to be low, but I’ll still pay for what I like. And I’d love to do all this, still being able to tinker, still seeing that people producing things I like get paid, without going to jail or letting them empty my wallet with a wet/dry vac.

Sounds a bit like a RSS+BitTorrent version of MMS, which basically is a combination of an SMS notification and a GPRS download. Do RSS and/or Atom have extensions that could support this? Looking at Orchard I would guess that with RSS, you would have to use a hack like ‘automatically follow all links if they point to a .mpg file‘. Atom seems to provide for a Link:Type attribute, which allows neater integration.

So the system could be as follows:

  • a watcher application periodically checks certain RSS/Atom links (basically this is an aggregator)
  • it triggers on certain conditions, like the occurence of a type="video/mpeg" mime-type link, or a link that ends in .mpg/.mpeg
  • when it triggers it starts downloading in one of several ways, (S)FTP, BitTorrent, … – presumably you would need proper authentication (this is where payment comes in)
  • you are alerted when new content is available on your local system (this could be a 2nd ‘delayed’ feed)

It’s basically caching of content before it is consumed. Pre-caching?

[Listening to: “That Night” – Jazzanova – Jazzanova Remixed – Disc 1]

Metafilter on Steve Gadd

Great post on Steve Gadd on MetaFilter. He’s one of the reasons I wanted to start drumming, after hearing him on Simon & Garfunkel’s “Concert in Central Park”. When I started browsing through the Steve Gadd Grooves and Fills, I discovered that he was also responsible for “Chuck E’s In Love” (Rickie Lee Jones) and “Stuff Like That” (Quincy Jones). Tunes that got stuck in my head. He’s probably on more of my CDs than I can imagine. Janis Ian, Michael Franks, Al Jarreau, Michael McDonald, Steely Dan, …

I wonder who his worthy counterpart would be as a bass player (in my collection, then). Tony Levin? Pino Palladino? Anthony Jackson? My vote would go to the latter.

Nice follow-up at mrfeinberg.com.

[Listening to: “Harlequin” – Weather Report – Heavy Weather]

Engineering for the ego

I impressed my daughter today! The CD-ROM drive in her computer was stuck halfway and wouldn’t move anymore.

I opened up the case, took out the drive, removed the casing and peeked into the interior with a completely fake ‘I know what I’m doing’ look on my face. I saw a piece of plastic that was twisted, I removed it (silently praying), I put everything back together (actually I did that twice, I forgot some parts the first time) and voila: it worked! Elementary, my dear Watson. And I probably made the same impression on Clemence as my father used to make on me: my dad can fix anything!

Actually, for some reason this technique works quite often: you reverse-engineer a gizmo, optionally you actually fix something, you put everything back together and it works. The trick is then to put an expensive label on it: “Presumably a bad contact”, “Undoubtedly a rogue device driver”. In this case: an acute mechanical encumbrance.

[Listening to: “So What” – Miles Davis – Kind of Blue]

A Pentium 4 is not necessarily a Pentium 4

I was throwing my DAW system together, first time I actually ‘built’ my own PC, and I thought it went kind of smooth. But my PC did not want to boot every other time. It just started beeping ee-oo-ee-oo, which indicated a CPU problem. I upgraded to the newest BIOS posted on the Aopen site, and then I got the real culprit:

This motherboard does not support Prescott CPU, turn off power to prevent damage.

As if I should know the difference between a Prescott and a Northwood processor… I didn’t even know I bought a Prescott.
Anyway, this Anandtech article made it clear: both use Socket 478, but my i865G motherboard only accepts the older ones. Well, I’m learning and that was the purpose. I’ll change my Pentium4 for a Pentium 4.

[Listening to: “Finally [the Sting Reprise]” – Julie McKnight – Yoshiesque, Vol. 2 Disc 2]

No life without CURLs

If you’re a web server administrator – as I am – every now and then, someone yells at you “The site is down, fix it!”.
A tool you should always keep handy is CURL. It’s a command-line web client (multi-platform – I use it on Win2K), that allows you to see the conversation between a web server and a client (like e.g. your browser).
Let’s say if you check out a page like this: curl -I http://someserver.com/whatever, you could get

  • curl: (6) Couldn't resolve host someserver.com'
       can you CURL www.google.com? => you’re using a wrong domain name (DNS record does not exist)
       you can’t CURL anything => your DNS settings are wrong, or your DNS server does not work as expected
  • curl: (7) Connect failed
       there is no web server answering on that port
       => is the server down? does site run on other port? does a restart help?
  • HTTP/1.1 401 Access Denied [WWW-Authenticate: Basic realm="someserver.com"]
       you have to log in to this page – your browser would pop up a username/password screen
  • HTTP/1.1 403 Access Forbidden
       there is no ‘default’ document in the folder you are requesting
       => option 1: there should be a document index.html, default.htm, … in each folder
       => option 2: you want the folder to be browsable, so you have to configure this on the server
  • HTTP/1.1 404 Object Not Found
       the document you ask for does not exist.
       => are you asking for the right document (typo)? on the right server?
  • HTTP/1.1 500 Server Error
       typically only for dynamic pages (asp/php/cgi), where the program code contains an error and there is no HTML result that can be shown.
       => go look into the web server or program logs in order to find the bug.
  • HTTP/1.1 302 Found [Location: http://someotherserver.com/somepath]
       the site is redirecting you to some other URL, which itself might give an error 404/403/connect failed/…
       302 codes are not errors, it’s a common way of sending browser to the right location.
       (for example: if this 302 redirects you to a wrong port http://someserver.com:88/whatever and nothing responds on that port, in your IE browser it looks like someserver.com is failing, whereas it really is someserver.com:88)
  • HTTP/1.1 200 OK
       This is the ‘normal’ result of asking a web page.

For SSL/HTTPS problems, you better use curl -Ivk, because it gives you “* Connected to someserver.com port 443 * SSL connection using RC4-MD5 * Server certificate: (certificate content) …” information.

Have CURL installed on all your machines, Linux or Windows. Actually, add unxutils.sourceforge.net to your Windows machine, so you can work with gawk and wget too.

CURL is not the same as a real browser (especially when working with cookies/sessions), but it can help you solve 90% of typical web server problems. If the problem is browser-related, and you want to see the actual conversation between your browser and the web server, use the Ethereal network protocol analyzer.


This post from Anil Dash brings back memories:

Current number of days that Michael McDonald’s “I Keep Forgettin'” has been stuck in my head: Four.

I remember back in 1991 I was listening to that song for hours (Yep, she dumped me). I can safely say I must have had it in my head for over a month. Well, four days is a start …

StereoSearch – a new Hizmo

I’ve got a habit of every now and then making tiny web applications to solve some issue/challenge I encounter. I call these Hizmo’s, for lack of a better term. (You have to be Belgian to get this typo).

The last one I made is called StereoSearch, an interface for searching search engines at the same time with the same keywords. They are in 2 separate windows side-by-side, so you can easily compare the top 5. I use it to compare Yahoo and Google results, and to test the new GigaBlast search engine. Again, no rocket science, just handy.

Older Hizmo’s are:

  • LoginForm generator: for creating a minimal login screen for HotMail, Google, …
  • DisciplineBrowser: OK, this one was for fun. It allows you to browse the web for defined # of minutes and then alerts you to go back to work.
  • HTTPHeader Analyzer: show HTTP headers (I made this one while working with caching servers)
  • WorldSpeed tester: tests your connection speed to different parts of the world.

UPDATE: these tools are now on web.forret.com!

Portable audio & copy protection

Dear Record Company,

I just received my latest shipment from Amazon.co.uk. Previewing songs, referral to related artists, only 2 days between ordering and delivery, I love shopping the Amazon way. Which is probably why I buy around 50 CDs a year.

Amongst the CDs I ordered(BuzzinFly, Kinobe, Deep Dish) was the new Zero 7 CD: “When it falls”. I saw the group live in Brussels, they were great!

Now the issue is the following: their CD has an invention from hell called “copy-protection”. I don’t know what you use, Macrovision or something, but it stinks!
The backcover mentions “can also be played on most CD-ROM drives, by means of the included compressed file”. So I tried it on my PC.

Result: my Windows Media Player keeps crashing on it – mostly taking my PC with it, because the CD-ROM becomes inaccesible. My MusicMatch won’t play it, and I haven’t even tried RealPlayer, because I don’t want to reboot more than 3 times in a day, life’s too short. So I can only look at the cover and wonder what it sounds like. Oh, it will probably play in my DVD player back home (it’s an old Sony, it doesn’t support MP3). But not here, now, in the 4 devices I have at my disposal that I use for ‘normal’ CDs.

Not only do you ‘make it hard’ to copy the CD, you also ‘make it hard’ to listen to it. All that to prevent me from copying my CD. Why in heaven’s name would I want to copy/rip the CD?

  • Recently my car got vandalized, all my CDs were gone. Luckily they were copies. I keep the originals back home.
  • There is no CD insurance that gives me a new CD when the old one is scratched.
  • I make MP3 compilations for listening in my car. 10 CDs give me 60 albums.
  • My Nokia 5510 does not accept CDs

But I know, you guys are very anal about your content. And every young kid is copying and exchanging hundreds of albums1, right?

I can only say, you’re now using a flaky ‘copy protection’ system that prohibits me from listening to a CD I just purchased. It will not play on over 80% of the audio equipment I’ve got.
What’s next, CDs only playable on your own ‘certified’ CD players?

My request to record companies:
Inform the on-line record stores on what CDs are using ‘copy procection’.
Make sure I can see on Amazon if you’ve messed with a CD I want to buy.
Just like DVD zones 1, 2 and 3: I don’t want to buy Zone 1 DVDs, because I can’t play ’em.
I don’t want to buy ‘Copy protected’ CDs, because I can’t play ’em!

Oh wait, here’s a thought: offer your music on iTunes, tell ’em to speed up their Europe operation and I’ll happily download it at 1$/song and enjoy it whenever and wherever I choose. Stop swimming against the stream, use it.

1Just out of curiosity: did the average kid buy 100 albums/year 20 years ago? Are you sure it’s all ‘missed business’? I know I couldn’t afford that.

[Listening to: “Home” – Zero 7 – When It Falls] — (sigh) I wish …

Fancy redirection on IIS 5

The task seemed easy enough:

set up a new web site sub.domain.com such that every URL http://sub.domain.com/some_path is automatically redirected to process.aspx/?path=some_path (on the same domain or on www.domain.com – both have .Net running)

Because we run our sites on Win2K/IIS the following options were immediately discarded:

A. Install & configure Apache

I like Apache, let there be no mistake about that, but it’s just too much hassle managing 2 different web servers on 1 machine. I like having one management interface. Mind you, it would have been so easy:
RedirectMatch /(.*) http://www.domain.com/process.aspx?path=$1

B. Develop a Python daemon

What’s wrong with being a geek 🙂 ? This is after all a simple single-purpose web site, you have an HTTP GET coming in and a 302 Redirect going out with super-basic parsing (check out the regular expression above). But again: having several web servers on 1 machine is asking for chaos. Plus, how robust is the Python SimpleHTTPServer? Has it been tested with attacks from Blaster/Mydoom and the likes?

Ok, this leaves us with the IIS server. So I create a new IIS site, that takes all the sub.domain.com requests (using a host-header). And now just add redirection, right? Let’s see …

C. /default.asp page

This might have worked if all the incoming URLs were like http://sub.domain.com/?some_path (see the question mark?). But that’s not the case.

BTW: Why .asp instead of .aspx? Because it’s basically a hack, and hacking is way easier in ASP (if you’re me).

D. Custom 404 script

Configure the IIS to redirect all not found pages (‘404’ errors) to a /404.asp script. The script basically has to do:
Response.Redirect "/process.aspx?path=" & Request.ServerVariables("URL")
Only that I get as URL “/404.asp” instead of the original URL. I can’t seem to get the some_path. So I try this for 10 minutes and then move on, basically because I ‘m thinking the last method will be the fastest. If I would have looked further, the QueryString contains “404;http://sub.domain.com/some_path“, and I could have parsed it out.

Additional remark here: if your home directory is empty, and you call http://sub.domain.com/ (home page), you get an error 403 Access Forbidden error, instead of 404 Page not found. So you would have to either remap that error 403 too, or create a /default.asp that catches this case.

E. Use IIS redirection

Sure, IIS supports automatic redirection out of the box! Regular expressions and everything! Let’s elaborate on this method.
IIS redirection

  • Trial E.1: Redirect all URLs to /process/?path=$V ($V would translate here in “/some_path”). Only, when you redirect everything to to a subfolder, this policy is inherited by this subfolder, and you also redirect calls to that subfolder. So you get a nice endless redirection loop that creates links like /process/?path=process/?path=process/?path=process/?path=…
  • Trial E.2: oh, but that is what the checkbox ‘A directory below this one’ is for. Only, it doesn’t work: I get 404 errors. I’m probably doing something wrong.
  • Trial E.3: oh, but you can use a more sophisticated redirection, and turn off redirection for subfolders with “!”. Ok, I try *;/process/*;!/*;/process/?path=$0. Nope. Try setting “!” as redirection for /process/ folder. Syntax error. Try “*;*;!” as redirection for /process folder. Nope, I get get redirection to a non-existent page “!”. Ok, drop this one.
  • Trial E.4: wait, what if I don’t redirect to a script one folder lower, but just to /process.aspx?path=$V? Ta-daaa! Here’s the endless loop again!
  • Final try E.5: This is what worked at last: install the process.aspx on the destination server and redirect to it (in other words, exactly what I would have done if I had used Apache)

Trivial, right? Did I ‘solve’ the redirection spaghetti in IIS? Not really. Did I get a final result that worked? Yep. Who’s the man?

Moodlex & Ableton Live

I saw a gig some weeks ago by Moodlex: just one guy with a portable PC, mixing his music live.
It was awesome, terribly funky. I just had to know what he was using for software, that made it so easy and intuitive to do live mixing/composing.
I think there ‘s a fair chance he was using Ableton Live. I’ve downloaded the trial version and hope to test it out soon on my new DAW system.

Oh yeah, I also saw Briskey there, a group similar to Sven van Hees, but not that good. Only, their female jazz drummer, Isolde Lasoen, she’s great – in more than one aspect. I hope to see her again with the band Skeemz on the Brussels Jazz Marathon. She’s the girl on the sofa in the back on their first video, but you have to see her drumming to fully appreciate.