Some drivers love fancy chrome wheel rims, some add a huge wing spoiler on their car, or fuzzy dice hanging from their rear-view mirror. Me, when I bought my new car, I decided that I wanted a Wifi network in my car. So that any passenger with a laptop/iPod could read his email. And I imagined driving to distant holiday locations while my passenger where watching movies streamed from a NAS disk built into the car.
This weekend I finished episode one: Internet in the car. I used the following components:
The important details here are:
- The Huawei stick is compatible with the TP-Link router.
- The Huawei stick can be configured with ‘Save my PIN’, so that when it starts up, it connects to the 3G network without any manual intervention.
- The TP-Link router runs on 12V DC (which is what a car has)
For the rest, the exercise was quite straight-forward: I configured the Huawei stick on my laptop with the right PIN code, popped it in myTP-Link router, configured the right 3G settings for Telenet (see here). Then I took an old 12V power transformer, chopped off the connector and linked it up to an old car cigarette lighter type of plug (sometimes it’s good to have an archive of obsolete cables and power supplies). I then hid the router under the base plate of my trunk, where there is also the battery (the BMW X1 stores the battery in the back, where you would normally have the spare tyre). I switch on the car and 20 seconds later, I have a Wifi network “OnTheRoadAgain” that is connected to the Internet. Proof of concept is OK!
The next step now is to add a (Synology) NAS, which also runs on 12V, and hook up my iPad to the car Wifi to view my collection of backed-up DVDs from that disk. And maybe run some extra programs (e.g. MRTG for monitoring) on that NAS. To be continued!
I did a bandwidth test the other day with the iPhone SpeedTest tool. I wanted to compare the speed using (standard) GPRS, using 3G and my own Wifi. The results were all a power of ten apart:
- iPhone on Proximus GPRS: 35 kbps (download & upload)
- iPhone on Proximus 3G: 350 kbps (download & upload)
- iPhone via Wifi: 3500 kbps (download – upload is +- 300 kbps)
The real reason is that I wanted to see how fast I would wear out my Proximus data plan (200MB per month). The answer: with GPRS I would need more than 12 hours of continuous downloading, with 3G I could do it in less than 2 hours. So GPRS is pretty safe, it’s also easier on your battery, but you have to live with slow, pre-1996 modem-like performance. The latency – the time it takes to get your first byte after requesting a URL – is easily 10 to 50 seconds. Not milliseconds, seconds!
As a side note: do not take a time-based data subscription, certainly not with the iPhone. My first post-iPhone Proximus invoice was 800,- euro, which is more than the price of my iPhone! When I contacted them about that, they immediately offered to reimburse it and advised me to switch to a size-based plan. I guess I was not the first one …
Seth Godin came up with a visualisation of ‘means of communication’: bandwidth vs sync(chronicity). He took a number of ‘old’ (postal mail, radio) and ‘new’ (blogs, Youtube and -of course- Twitter) technologies and ranked them on a 2D graph according to ‘quality’ (density or bandwidth) and ‘sync’ (speed of reaction).
Although it is an interesting way of visualizing things, and I consider Seth a very bright and creative guy, I am bothered by the fact that the graph is neither clear, correct nor complete.
Continue reading Seth’s bandwidth vs synchronicity graph: it’s a start
I know, there are so many ‘funny’ videos you just have to share with your friends. So you send them an email. But for god’s sake, not with a 5MB movie in attachment! For all you know, he/she might not even be able to play that MOV/WMV/XVid movie anyway. Don’t send a movie, send a link!
WHY EMAILING VIDEOS IS BAD
- Email makes big files bigger
Binary files (like videos) are encoded, or rather exploded, by your email program (Outlook/Hotmail/Gmail/…) as text-only Base64 MIME attachments. Your 5MB file is transformed into a 6.85MB text file before is sent. Email is a very inefficient way to share videos with several other people.
- You hurt the recipients
Your email will have to be downloaded before the recipient can see it. If he is on a slow connection, this might mean 15 minutes of obnoxious delay before he can continue working, start receiving the emails that arrived after your ‘cute puppy’ movie. The movie, if it is not deleted, will add 5 MB of storage to the Inbox. If his Outlook/Exchange quota is 100MB (not uncommon on corporate email systems), you just ate 5% of all the place he has to store contracts, meeting reports and office gossip.
- You hurt yourself
By sending a 5MB video, you force your email program to upload a 6.85 MB file to your mail server. If you’re on a basic DSL line, this will easily take up to 10 minutes, during which all your other Internet activity will go very slow. You also add a big chunk to your “Sent Items” folder, bringing you closer to your quota limit.
- You hurt the Internet
All those forwarded videos make for a huge amount of unnecessary traffic that eats up bandwidth at ISPs and inspire them to keep prices high. Not that they needed the extra inspiration.
- It’s force-feeding-video, not video on demand
You are forcing people to download the whole file before they can decide whether they want to see it now, or ever at all. Youtube and the other sites have a very easy-to-use ‘Send video link’ form that will give the receipient the link, with a screenshot and the description text. Then he/she can decide when, where, how and *IF* to watch the video.
(Yes, this is less a problem with web-based mail like Gmail or Hotmail)
HOW TO FORWARD A VIDEO LINK
- public, popular movie
Don’t think you’re the first one to have seen this movie. Chances are it’s featured on Youtube, Google Video, DailyMotion, Vimeo, in multiple versions (FR subtitles if that’s what you like), in a format everyone can view, available to send as just a link “
http://www.youtube.com/watch?v=_RgL2MKfWTo“. Less that 50 characters for a full 1:14 of hilarious time loss.
- private, ‘secret’ movie
Even if you have a movie you recorded/made yourself and want to show only to a limited number of people (“OMG, Britney, you were, like, *so* drunk!!“), then upload it yourself to Youtube, Flickr or Vimeo, put a password on it and send link+ password to those recipients. It will be so much easier for everyone to forward that secret video that no one was supposed to see (“788 views just yesterday? How’s that possible?“).
We thank you.
Met deze actie willen wij een statement maken: stilstand is achteruitgang. De toestand op de Belgische markt dient te veranderen. De gebruikers willen van de datalimieten af, meer concurrentie en betere tarieven voor hun breedband-internet.
Dag van de download
Dit zijn de beste prijzen per GB in Belgie:
En dit zijn de meest frappante gevallen van belachelijke data limieten:
Ine mentions that Dreamhost has become a more reliable hoster. I am actually tracking Dreamhost performance, and I can only agree.
This is the current response time of a WordPress blog on Dreamhost:
Wordpress is a database-powered PHP application, so this response time includes the MySQL queries and PHP overhead.
This is the very similar performance of a second blog on another Dreamhost server:
Continue reading Dreamhost has better performance now
Just how many pictures does Flickr receive every day? I found a way to estimate the # of images that they add to their database, and another way to get average (original) file sizes for those images. The result? Their storage growth, i.e. their upload bandwidth, and the growth rate of their storage system (how many days to reach a terabyte?)
Number of photos per hour
You see here that weekends, Sundays specifically, are the most busy days for uploads. You can see peaks of almost 68.000 pictures an hour (almost 20 pictures a second). Peaks are around 22h CET (or 1 PM PDT – in California). The lowest rates (still around 20.000 photos/hour) are 12h apart: 10h CET (1 AM PDT).
The average inflow of pictures is: 38.400 photos/hour. That is around 10 photos/second, 920.000 photos/day.
Average photo size
And how big are those pictures? I have found a way to estimate average filesize (and maximum, while I’m at it). It’s not perfect, but quite accurate. How? That’s classified. I could tell you, but then I’d have to … Anyway: these are the numbers:
On average, a picture uploaded to Flickr is 555.2 KB big. They receive files up to 7.3 MB (what number of megapixels would that be?) and quite a lot of 3MB images. My Canon 350D makes 8 megapixel images (3456 x 2304 pixels) that are between 2 and 4 MB large. But the ones I send to Flickr (after Picasa processing) are typically smaller: 1200 x 800 (300 – 600KB) or 1024 x 683 (200 – 400KB).
What happens if we multiply both numbers?
38.400 pictures/hour x 555,2 KB/pic = 21,3 GB/hour = 5,9MB/sec or 47,3 Mbps. Storagewise, this is 15,3 terabyte/month of new pictures. Thank God storage prices are dropping.
Five years ago, a server with a few hundred gigabytes of storage – one of many needed to handle uploads of member photographs – would have cost Flickr about $250,000. Today, Mr Butterfield says, “you can get a terabyte of storage for about $5,000”. (via ft.com)
Peak bandwidth usage: let’s take 60.000 pictures/hour x 3MB/pic: 180GB/hour = 50 MB/sec or 400 Mbps. This is probably still peanuts compared to their outgoing bandwidth.
After a conversation with Ine on HD formats (1080i vs. 1080p), I researched the topic a bit further. Let me resume some of the things I have learned up till now:
Real HD and HD-ready
HD or ‘high definition’ as defined for screens, projectors and TV, defines 2 resolutions. The smaller one has 720 lines of each 1280 pixels, the bigger one 1080 lines of each 1920 pixels. They can be used with different frame rates: refreshed at 24 fps (a common movie standard) up to 50/60fps (often used for TV). To limit the necessary bandwidth in some cases ‘interlaced scanning’ is used: 1 frame contains all the odd lines, the next only the even lines. This effectively halves the throughput, at the cost of image quality (rapid moving lines appear jagged).
The two most common formats are:
- 720p60: 1280×720, 60 fps progressive scanning, used e.g. in USA-based HDTV broadcasts
- 1080i50 or 1080i60: 1920×1080, 50 or 60 fps interlaced scanning. The higher resolution makes it better for larger screens and movies, but the interlacing has a bad influence on fast moving images (like e.g. sports).
What kind of resolution do we have now? Regular digital TV (SD or ‘Standard Definition’) consists of 480 lines of 720 pixels each. DVD, for instance, allows for 480i and 480p. So, HD delivers at least 3x that resolution.
“HD Ready“, a label that a lot of TVs/screens carry now, just indicates that:
- The minimum native resolution of the display (e.g. LCD, PDP) or display engine (e.g. DLP) is 720 physical lines in wide aspect ratio.
- The display device accepts HD input via Analogue YPbPr1, DVI or HDMI
- HD capable inputs accept the following HD video formats: 1280×720 @ 50 and 60Hz progressive (“720p”), and 1920×1080 @ 50 and 60Hz interlaced (“1080i”)
- The DVI or HDMI input supports content protection (HDCP)
from eicta.org (PDF)
Even if the display can only show 720p, and so must ‘downsample’ an incoming 1080i signal to that lower resolution, it can be called “HD Ready”.
Continue reading HD – 720p, 1080i and 1080p
I’ve been following the Database War Stories of O’Reilly Radar: how companies use text-based alternatives to classic relational database systems in order to cope with huge volumes. Check out the stories of Findory/Amazon, Google File System, Flickr and Second Life. Anyway, this seemed like a good moment to share some of my database war stories. Let me take you back to the early nineties.
1993 @ Ukkel
I arrive at Sopres, one of the larger direct marketing / database management companies in Belgium. Fresh from university (and 1 year of military service), I expect to see RDBMS everywhere and dive into SQL. Imagine my surprise when I see that, yes, there are a lot of Sybase SQLServer databases around, but the bulk of the work is done with something they call ‘square files’ (see below). They have built a whole set of tools to work with those and by using them myself, I learn to appreciate the advantanges of the system (speed, mainly) and grow a fairly accurate intuition for things like queries, indexes and outer joins.
Continue reading Database war stories: DB vs ‘square’ files
Youtube seems to be losing some of its early adopters: Coolz0r quits the service, while Nathan even embarks on a grassroots activism mission to ruin the company (by getting its most popular uploaders banned – I have mixed feelings about that one). The issue is: to protect themselves from lawsuits, Youtube is taking the approach of deleting videos and even users upon first suspicion of (copyright) problems. They already received an ultimatum from NBC in Feb, then a proof to Jason Calacanis that it was ‘not a real business‘.
What I found interesting in the whole controversy are the astounding numbers that popped up: # videos shown per day, bandwidth usage, bandwidth costs. Get ready for some big numbers:
Continue reading Youtube bandwidth: terabytes per day