Archive for the 'bandwidth' Category

Dag van de download

Dag Van De Download

Met deze actie willen wij een statement maken: stilstand is achteruitgang. De toestand op de Belgische markt dient te veranderen. De gebruikers willen van de datalimieten af, meer concurrentie en betere tarieven voor hun breedband-internet.
Dag van de download

Dit zijn de beste prijzen per GB in Belgie:
Breedband in Belgie: prijs per GB

En dit zijn de meest frappante gevallen van belachelijke data limieten:
Breedband in Belgie: volume rapst op

If you're new here, you may want to subscribe to my RSS feed or receive updates via email. Thanks for visiting!

Dreamhost has better performance now

Ine mentions that Dreamhost has become a more reliable hoster. I am actually tracking Dreamhost performance, and I can only agree.

This is the current response time of a Wordpress blog on Dreamhost:
Wordpress on Dreamhost: #1
Wordpress is a database-powered PHP application, so this response time includes the MySQL queries and PHP overhead.

This is the very similar performance of a second blog on another Dreamhost server:
Wordpress on Dreamhost: #2
Continue reading ‘Dreamhost has better performance now’

A picture a day: Flickr’s storage growth

Just how many pictures does Flickr receive every day? I found a way to estimate the # of images that they add to their database, and another way to get average (original) file sizes for those images. The result? Their storage growth, i.e. their upload bandwidth, and the growth rate of their storage system (how many days to reach a terabyte?)

Number of photos per hour

Flickr: #photos per hour
You see here that weekends, Sundays specifically, are the most busy days for uploads. You can see peaks of almost 68.000 pictures an hour (almost 20 pictures a second). Peaks are around 22h CET (or 1 PM PDT - in California). The lowest rates (still around 20.000 photos/hour) are 12h apart: 10h CET (1 AM PDT).
The average inflow of pictures is: 38.400 photos/hour. That is around 10 photos/second, 920.000 photos/day.

Average photo size

Flickr: average and max photo size
And how big are those pictures? I have found a way to estimate average filesize (and maximum, while I’m at it). It’s not perfect, but quite accurate. How? That’s classified. I could tell you, but then I’d have to … Anyway: these are the numbers:
On average, a picture uploaded to Flickr is 555.2 KB big. They receive files up to 7.3 MB (what number of megapixels would that be?) and quite a lot of 3MB images. My Canon 350D makes 8 megapixel images (3456 x 2304 pixels) that are between 2 and 4 MB large. But the ones I send to Flickr (after Picasa processing) are typically smaller: 1200 x 800 (300 - 600KB) or 1024 x 683 (200 - 400KB).

Upload bandwidth

What happens if we multiply both numbers?
38.400 pictures/hour x 555,2 KB/pic = 21,3 GB/hour = 5,9MB/sec or 47,3 Mbps. Storagewise, this is 15,3 terabyte/month of new pictures. Thank God storage prices are dropping.

Five years ago, a server with a few hundred gigabytes of storage – one of many needed to handle uploads of member photographs – would have cost Flickr about $250,000. Today, Mr Butterfield says, “you can get a terabyte of storage for about $5,000”. (via ft.com)

Peak bandwidth usage: let’s take 60.000 pictures/hour x 3MB/pic: 180GB/hour = 50 MB/sec or 400 Mbps. This is probably still peanuts compared to their outgoing bandwidth.

HD - 720p, 1080i and 1080p

After a conversation with Ine on HD formats (1080i vs. 1080p), I researched the topic a bit further. Let me resume some of the things I have learned up till now:
HD quality: 720p and 1080i

Real HD and HD-ready

HD or ‘high definition’ as defined for screens, projectors and TV, defines 2 resolutions. The smaller one has 720 lines of each 1280 pixels, the bigger one 1080 lines of each 1920 pixels. They can be used with different frame rates: refreshed at 24 fps (a common movie standard) up to 50/60fps (often used for TV). To limit the necessary bandwidth in some cases ‘interlaced scanning’ is used: 1 frame contains all the odd lines, the next only the even lines. This effectively halves the throughput, at the cost of image quality (rapid moving lines appear jagged).
The two most common formats are:

  • 720p60: 1280×720, 60 fps progressive scanning, used e.g. in USA-based HDTV broadcasts
  • 1080i50 or 1080i60: 1920×1080, 50 or 60 fps interlaced scanning. The higher resolution makes it better for larger screens and movies, but the interlacing has a bad influence on fast moving images (like e.g. sports).

What kind of resolution do we have now? Regular digital TV (SD or ‘Standard Definition’) consists of 480 lines of 720 pixels each. DVD, for instance, allows for 480i and 480p. So, HD delivers at least 3x that resolution.

HD Ready“, a label that a lot of TVs/screens carry now, just indicates that:

  • The minimum native resolution of the display (e.g. LCD, PDP) or display engine (e.g. DLP) is 720 physical lines in wide aspect ratio.
  • The display device accepts HD input via Analogue YPbPr1, DVI or HDMI
  • HD capable inputs accept the following HD video formats: 1280×720 @ 50 and 60Hz progressive (“720p”), and 1920×1080 @ 50 and 60Hz interlaced (“1080i”)
  • The DVI or HDMI input supports content protection (HDCP)

from eicta.org (PDF)

Even if the display can only show 720p, and so must ‘downsample’ an incoming 1080i signal to that lower resolution, it can be called “HD Ready”.
Continue reading ‘HD - 720p, 1080i and 1080p’

Database war stories: DB vs ’square’ files

Plug and PlayI’ve been following the Database War Stories of O’Reilly Radar: how companies use text-based alternatives to classic relational database systems in order to cope with huge volumes. Check out the stories of Findory/Amazon, Google File System, Flickr and Second Life. Anyway, this seemed like a good moment to share some of my database war stories. Let me take you back to the early nineties.

1993 @ Ukkel
I arrive at Sopres, one of the larger direct marketing / database management companies in Belgium. Fresh from university (and 1 year of military service), I expect to see RDBMS everywhere and dive into SQL. Imagine my surprise when I see that, yes, there are a lot of Sybase SQLServer databases around, but the bulk of the work is done with something they call ’square files’ (see below). They have built a whole set of tools to work with those and by using them myself, I learn to appreciate the advantanges of the system (speed, mainly) and grow a fairly accurate intuition for things like queries, indexes and outer joins.
Continue reading ‘Database war stories: DB vs ’square’ files’

Youtube bandwidth: terabytes per day

Youtube (Blogpulse)
Youtube seems to be losing some of its early adopters: Coolz0r quits the service, while Nathan even embarks on a grassroots activism mission to ruin the company (by getting its most popular uploaders banned - I have mixed feelings about that one). The issue is: to protect themselves from lawsuits, Youtube is taking the approach of deleting videos and even users upon first suspicion of (copyright) problems. They already received an ultimatum from NBC in Feb, then a proof to Jason Calacanis that it was ‘not a real business‘.

Youtube traffic
What I found interesting in the whole controversy are the astounding numbers that popped up: # videos shown per day, bandwidth usage, bandwidth costs. Get ready for some big numbers:
Continue reading ‘Youtube bandwidth: terabytes per day’

Bluehost vs Dreamhost

As you might have read in my Migrating to Wordpress article, I am now the proud owner of both a Bluehost and Dreamhost account. These two shared hosting providers have similar strong offerings for a similar low price, but they’re nevertheless different. Let’s compare both:

The raw numbers

BLUEHOST.COM
Bluehost
DREAMHOST.COM
Dreamhost
PRICE
$6.95/mon (2 years prepaid) $7.95/mon (2 years prepaid)
FEATURES
  • 10 GB storage
  • 250GB/mon bandwidth
  • 20 GB storage
  • 1000GB/mon bandwidth
  • 6 domains, 20 subdomains
  • 50 MySQL, 50 Postgres DB
  • 2500 email addresses
  • PHP, Perl, Python, Ruby on Rails
  • unlimited domains
  • unlimited MySQL DBs
  • unlimited email addresses
  • PHP, Ruby on Rails
ONE-CLICK Install
CPanel/Fantastico: Wordpress, pMachine, Nucleus, Drupal, Joomla, PhpNuke, Typo3, phpBB2, OS Commerce, Coppermine, Gallery, PHPList, Advanced Poll, PHProject, SohoLaunch, PhpWiki, PhpAdsNew, WebCalendar, Moodle, … Home-made: Wordpress, phpBB, Advanced Poll, osCommerce, MediaWiki, Joomla, Gallery, WebCalendar

Continue reading ‘Bluehost vs Dreamhost’

Double Wifi: municipal wifi with protection

I have written about FON before (they provide a business model for sharing one’s bandwidth through Wifi). They use a custom firmware for the Linksys WRT54G routers. I have the feeling that current Wifi routers (or access points) cannot offer a good balance of security/flexibility. Opening your own network for everyone is currently too dangerous. There’s Wifi trolls that gobble up your bandwidth and there’s hackers that scan your ports for vulnerabilities. My idea is that now you would need 2 Wifi zones, one behind the other, each having different security and different policies. With access points costing as little as 25 euro, that is not a big investment.

I see 2 scenario’s:
Scenario 1: first the public
Double Wifi: first the public

Description
The first router is connected to your broadband and serves the PUBLIC zone (e.g. SSID “FREEWIFI”). On one of the wired Ethernet connections (the Linksys has 4 of those) the other router is connected, that serves the PRIVATE zone (e.g. SSID “PROTECTED”). Both are in a different IP range. The PUBLIC one requires no login, the PRIVATE one requires WPA + maybe MAC address checking.
PRO
* both the Internet and the PUBLIC zone are outside your PRIVATE network, so you can have the same firewall settings for both, and ‘dangerous’ traffic never passes over your INTERNAL network.
* the first router can be configured to prioritize traffic from the fixed ports i.e. the PRIVATE network.
CONTRA
* If the PUBLIC router does not support QoS (Quality of Service) or bandwidth shaping, then a wifi troll can consume all the available bandwidth, and the PRIVATE network is left without anything.
* if the PUBLIC router is broken (or switched off) no one has Internet connection.

Continue reading ‘Double Wifi: municipal wifi with protection’