• anyone can edit from any place, even at the same time
• the servers are more reliable than a server at the office
• I can use the info (with CSV/Excel export) in other programs through a web link

But there is a problem popping up at random moments with that last export or ‘publish’ functionality. Sometimes when you download the published link of a CSV export (through curl), you get an error ‘Moved Temporarily - The document has moved‘ with a redirect to a www.google.com address. And if you don’t follow HTTP 302 redirects, you can’t get to the actual content. In the past I’ve always worked around it or waited until the error went away, but today I searched a bit further. So for those who have the same question: read and learn!

The redirect is actually for authentication. Although I publish without requiring signing in, so one would expect no authentication process, there actually is one. See what it does (I used wget in verbose mode to get the HTTP headers):

>>>:~$ wget -v “https://spreadsheets.google.com/(…)&output=csv”
-- https://spreadsheets.google.com/(...)&output=csv

(...)

Location: https://www.google.com/... (first redirect)

-- https://www.google.com/(...)/ServiceLogin?=...

(…)

Location: https://spreadsheets.google.com/... (second redirect)

-- https://spreadsheets.google.com/(...)&output=csv&ndplr=1

(...)

Saving to: ...

So what is the solution: just add “&ndplr=1” to your URL and you will skip the authentication redirect. I’m not sure what the NDPLR parameter name stands for, let’s just call it: “Never Do Published Link Redirection“.

Google Calendar’s web interface doesn’t really allow you to bulk delete. There is no way to select several dozens of appointments and delete them in one go. But I found a way that works (suggested here):

• Install Mozilla Thunderbird (desktop email client)
• Install Mozilla Lightning (calendar plugin for Thunderbird)
• Install Provider for Google Calendar (Gcal plugin for Lightning)
• Look up the Google Calendar Private iCal URL of your calendar (something like http://www.google.com/calendar/ical/...%40group.calendar.google.com/private-.../basic.ics)
• Add it to Thunderbird with FILE/NEW/CALENDAR/NETWORK/GOOGLE CALENDAR
• You now have a read/write connection to your Google Calendar!

Select the appointments you want to delete, hit the ‘Del’ button and see them disappear one by one.

When Rupert Murdoch announced that he would remove his sites from Google (in order to make a deal with Microsoft, so that only Bing would have the NewsCorp pages, as we now assume), he apparently wasn’t kidding. Although all Google web sites still indicate that e.g. MySpace has 179 million pages in the index, the Google API is currently returning another number for that: only 7 million. The total number of NewsCorp pages (a sum of MySpace, IGN, RottenTomatoes, …) has dropped from 192 million to 12 million.

(trend via http://trend.visualizor.com/g/1011 )

Which sites are Newscorp?

Let me give you some of his ‘big’ sites and how their # indexed pages have dropped:

• Myspace: from 179 mio to 7 mio
• RottenTomatoes: from 4 mio to 100.000
• IGN: from 4 mio to 300.000
• Stats.com: from 2.4 mio to 50.000
• News.com.au: from 1.2 mio to 70.000
• Sky.com: from 1.4 mio to 85.000

I suspect the Fox, National Geographic, Daily Telegraph, and other sites will soon follow.

Did he send in the robots?

I checked to see if NewsCorp finally started using the robots.txt file, because that’s the way you’re supposed to remove content from Google, not with press conferences.

Myspace:

User-agent: *
Disallow:

RottenTomatoes:

User-agent: Mediapartners-Google
Disallow:

And the answer there is “no”. So I’m not sure how they tell the Google crawler to stay out.

— UPDATE —

Source of the data:

The numbers come from http://tools.forret.com/newscorp/, which uses the Google Search API. I double-checked the replies from the API: for MySpace.com I get "estimatedResultCount": "6950000" so 7 million, not 179 million. If there’s an error, it’s in the Googleplex.

So I decided to let Facebook check my Gmail contact list to see if I had missed some contacts (people using aliases, etc …). After carefully selecting a couple of FB friends to invite (a buddy from the army, …), I clicked ‘Select’ and then ‘OK’ on the next screen that I supposed was a ‘Confirm’ window. I didn’t even read what was written on it. Some minutes later I saw emails starting to come in on different email aliases I had created in all my years of Internet activity. Apparently I allowed Facebook to send email messages to all Gmail contacts with email addresses that were not yet ‘known’ in Facebook. I have about 1500 addresses in my Gmail, let’s say some 500 already have a FB profile: so I just allowed Facebook to send out 1000 ‘unsollicited commercial emails’ or *spam* on my behalf. There is no way for me to know how many emails went out, nor to whom. I feel strongly embarrased, since I have been a strong opponent of spam for years, and since I have no idea who I have bothered with this bulk mail.

A company like Facebook probably has a whole team concentrated on user experience and workflow streamlining, so I can only assume that this strategy is by design. They probably have to keep the monthly exponential growth numbers so they use every opportunity to collect new email addresses. This is plain wrong. The default should be ‘opt in‘, not ‘opt out‘ (that is, select those you want to invite instead of unselect those you don’t wanto to invite).

So dear Christopher Cox and/or Chamath Palihapitiya at Facebook, while you will probably say that ‘but it is clearly written on the page that they’re about to send an invitation to (in my case, 1000??) contacts‘, you know that you are wrong on this one. You’re spamming. Big time, like real jerks. Since you’re probably not going to do anything about it, Google: any ideas?

http://www.google.com/support/forum/p/gmail/thread?tid=46004a5733eee4f0&hl=en

http://blogs.zdnet.com/social/?p=266

http://www.smartmobs.com/2007/09/02/facebook-friending-spam/

It’s not a bad widget, but it’s too limited. You can only display “Day/Week/Month/Agenda” style, the colors and fonts are fixed and it does funny stuff for events that continue after 12:00AM (which tango events regularly do, believe me).

I’ve already talked about the fact that iCal is a sissy format and that Gcal needs some more features. I was just thinking that it would be nice if some company would jump on that and provide the whistles and bells for iCal/vCal feeds (like those of Google Calendar), just like Feedburner did with RSS/podcast feeds (and they got bought by Google, so maybe their idea wasn’t half bad). So I introduce the following concept: CalendarBurner (since the Calburner/iCalburner domains are taken).

CalendarBurner

• takes any iCal/vCal feed as input -  can also interpret hCalendar and whatever other calendar format might exist (there must be at least 1 XML based format, right?)
• it knows about Gcal, Upcoming, Apple iCal and other hosted calendar applications
• it can ‘explode’ calendars: convert “every Thursday from X Jan to Y July, but not in May” to N individual events. Calendars where each event is non-repeating are easier to process. (E.g. in Google Calendar, a weekly event that goes on until Dec 14th is interpreted as “inclusive” that Dec 14th date. If you feed that iCal feed into PHPiCalendar, it interprets this repeating event as “before” Dec 14th, so it stops at Dec 7th. With an ‘exploded’ calendar, there is no ambiguity)
• It can visualize a calendar as an IFRAME, with a JS widget, through RSS feeds … and every time fully customisable in colors, fonts, links, order …
• Google Calendar has an RSS feed but this shows the events with the date they were created, not when the actual event takes place. It is one way of showing things, but you also need ‘most recent past events’, ‘next 10 events’ …
• It caches calendar info, so that a particular popular calendar does not crash when consulted by too many people.
• it uses easy URLs: /cal/ATS56GE78SH/rss20 and /cal/ATS56GE78SH/js
• it knows about client applications: when fed into Outlook, iCal, Sunbird, … it uses the features of those programs
• it allows for filtering, tagging, merging … (the “Remix” part of my Get/Remix/Deliver proposal)
• it gives you stats on subscribers, clickthrough, popular events …
• it links with Google Maps
• … (any ideas?)

It would really be nice if some innovative company would jump in this void. With an appropriate freemium model I’m sure it would take off and make calendaring an easier task to do.

Google Transit Feed

Metadata/Folksonomy

Currently an event in the agenda has the fields Title, Date/time (with recurrency, if any) , Location and Description. What I really miss is Tags (or categories, keywords, whatever you want to call them). Tags would allow me to attribute events to categories so that I can easily slice and dice them: only display the “milonga’s”, the events in Antwerp, the events in a specific place. Now I had to write a modified ‘filtered printable view’ for PHPiCalendar so that I can search on specific words in the event title, but that is really a hack. E.g. I now ask every editor to create the event titles as

“[TYPE]: [name of the event] @ [LOCATION]“

so that I can filter on “CONCERT:” or “@ Gent”. With the tags “concert, gent, polariteit, openair” it would be so much easier.

The iCalendar specification even mentions a ‘Categories’ field, although Google Agenda currently does not use it.

Filter/Merge/Expand

Currently you can only get the URL of the full ICS feed, nor do the ‘Calendar’ or ‘Agenda’ view allow you to filter the output on a certain date, keyword in the title or location. And sometimes I only need to have the feed or data for events in the future, to be exact, 1 month in the future. While we’re talking about the Remix part (see above): how about ‘Merge’? I could take the ICS feed for tango festivals (which is maybe not hosted on Google), merge it with my feed, and show both in 1 view, with -say- different colours. I also suggest an ‘Expand’ option: instead of listing a recurring event in 1 record with the recurrence data attached, expand it as N separate events, so that it’s easier to work with. An example: if I want the events for the next two weeks, I don’t want an entry ‘Every Thursday at 8PM’, I want two entries: one for the next Thursday, listed with all the other events for that Thursday, and one for the week after.

An example of URL syntax:
http://www.google.com/…/basic.ics?scope=2weeks&summary=milonga:&expand=true

RSS/REST API

RSS feeds are made for things that happened (in the past) and a calendar is mostly used for things that WILL happen (in the future). This means that an RSS feed with dates in the future looks rather weird. The way Google solved this for their RSS feeds (which they indicate with the XML icon – bad practice!) is to use as the date, the moment the event was added to the agenda, regardless of when the actual event takes place. So it is not necessarily a feed of ‘upcoming events’ but rather of ‘recently added events’. I would allow the user to select a ‘future RSS’ format for the next 10 upcoming events, or the next 7 days. There are so many tools that can do neat stuff with RSS feeds, it’s a pity you can’t use them in the logicl, intuitive way now.

The agenda also needs a public REST API  for reading and writing to the service. There is a Google Agenda REST API. i should have looked better. Maybe I should dive into the PHP client library. But an API is made for a different audience than the filter features I requested above.

Let’s see what happens when Google Agenda goes out of Beta…

• I use Google Maps‘ “My Maps” to create a collection of pointers on a map. I called this map “milonga.be tango venues”. It’s not complete yet, but I have about 25 locations in it for the moment. I can easily link to this page so anyone can see it: Belgian tango venues.
• But let’s say I want to embed it into a page. I could do it in-line (which would add a lot of JavaScript to the HTML) or use an inline frame (IFRAME). I decided to use the frame approach and build a generic KML visualizor.

So how can you use it to show any KML/GeoRSS feed on your website?

1. Go to the forret.com Google Maps visualizor tool
2. Copy/paste the KML feed URL. Example 1: the KML link from Google Maps:
   
   Example 2: Flickr feeds also have a Flickr GeoRSS format which is compatible (now also KML).
3. Choose the appropriate center point. Currently you have to copy/paste it from Google Maps or another application, I still have to add some interface magic to do it in the page.
4. Press “Show!” and copy/paste the resulting IFRAME HTML code. Voila!

An example: Peter‘s KML feed of tango locations in Brussels.

So I am using Google’s “My maps” feature as my online map editor and KML generator, and the regular Google Maps as the visualisation tool. I’m a happy man!

And this is what YouTube is developing as a new ‘beta’ version:

• they got rid of the large banner (” Use Quicklists!”) which was basically a waste of space.
• all clip info is now under the video instead of to the right. All info on and action on a video are now in one place. I like it.
• Youtube likes you to check out other videos. Most newly freed space is dedicated to thumbnails of other (‘related’/'promoted’) clips: you now see 13 of them on the frist page instead of 5.

When I started dancing argentine tango, there were two sites that gave you an update of where and when you could dance tango in Belgium. The first one was tango.be, with a frame-based layout that I don’t find the most user-friendly nor visually pleasing, and the second www.milonga.be, with a Flash-based agenda that was quite easy to use. Unfortunately the editor of the latter had to stop the site due to lack of time. Two weeks ago I noticed that he had even let the domain name expire and it was free again. Five minutes later I was the new owner of milonga.be. My goal: to make it again into a comprehensive overview of where to take tango courses and dance tango in Belgium.

WordPress again

Oh, what can I say, I know WordPress so well now, I use it wherever I can. So yes, it’s a WordPress site, with the K2 template, but with (currently) only static pages and no posts. I’ve divided the site into 2 parts: where to follow classes, and where to go dancing (practicas, milongas, salons, workshop). I’m obviously going to sprinkle some Web2.0 gold dust on the project. One example of this: Google Calendar.

Tango Calendar 2.0

One of the cornerstones of the site is the agenda/calendar. Most tango instructors keep some kind of agenda on their site (sometimes last updated in 2005), Marisa and Oliver keep an up-to-date agenda for most of Brussels and tango.be has an agenda for Flanders and Brussels. All these initiatives are separate, not connected and not always up-to-date. I considered some better alternatives:

SCENARIO 1: every tango teacher/organiser keeps a calendar that supports syndication, let’s say, Google Calendar. They add their own events and I find a way to merge all this data together. This is when I found out there is no library available for merging vCalendar/iCalendar feeds. By the way: have you ever seen that format? It’s like someone explained XML in English to a Frenchman and he asked a German subcontractor to develop it. (Actually, it’s a standard from 1998, so we can forgive the authors for not using XML)

BEGIN:VEVENT
DTSTART:19970714T170000Z
DTEND:19970715T035959Z
SUMMARY:Bastille Day Party
END:VEVENT

Some of the tango places do more than just tango and would probably put all these events onto 1 calendar, where I would have to filter out the tango-related stuff. And I would probably need access to all the agendas to correct errors or help them out with tricky issues like ‘how do I make a recurring event’.

SCENARIO 2: Because I wanted the option to only show tango events for Brussels or Antwerp, I though of several regional Google Calendars. I would then still have to figure out a way to merge all events into 1 calendar. But why manage 10 calendars when maybe one single one could do the job?

SCENARIO 3:
Let’s say I create one big Milonga.be calendar, where would I put it? Make or buy? ‘Make’ means hosting the calendar myself with a tool like PHPiCalendar, ‘buy’ means hosting it elsewhere (Google Calendar would be the preferred solution, since it’s free and hosted by, well, Google). Google manages the user accounts/passwords for me and already has syndication tools, so that’s what I chose for.

One big calendar where every organiser gets an editor account:

Embedding the full calendar into the milonga.be site is easy, since Google has <IFRAME>-based syndication tools built-in. But let’s make it a bit more difficult: filter out the Brussels events, or only the salons (the more upscale/expensive events).

I found a WordPress plugin for iCal feeds: iCal events. It parses the .ICS feed into an XML format, and allows you to print out a list of events (no calendar visualisation). I hacked it a bit to display monthly events better and to allow filtering and now it can take the Google feed, do a search on all events that contain ‘SALON:’ and show only those. But it still shows recurring events as ‘Every Wednesday until June 2007′ instead of expanding that to ‘Wednesday April 25′, ‘Wednesday May 2nd’ …

So I installed PHPiCalendar and found out it can use Google Calendar feeds quite easily. It has the text-based event-list (‘printer -friendly’) format as well as the calendar format, and it expands the recurring events. I can also use it to make customized <IFRAME>-based – or maybe even Javascript-based – syndication tools. So I’ll be diving into that PHP code.

Next step Google Maps!

Thanks to its huge geek-appeal (over 145.000 backlinks in Technorati), Twitter is well on its way towards a respectable PageRank 8. Twitter also uses pretty URLs, (twitter.com/[user]/statuses/[messageid] ), which Google likes a lot. Twitter also generously links from one account to the other (Twitter Friends). And Twitter has a LOT of (small bites of) content. As a result of that, whatever you say in Twitter may come back to haunt you through Google.

Exhibit 1: Pietel

When you do a Google search for “Pietel”, his Twitter account shows up as result #4 of 226.000. Being the good boy that he is, he just wrote that he finished his work assignment at home. But if his last remark would have been: “stupid job, silly colleagues, moron boss“, would he like that to show up on Google?
(depending on what Google server you fall, your results might be different, but the Twitter result has a good chance of ending up on the first page of results).

Exhibit 2: Clopin

Even worse here: the Twitter account of Clopin shows up as site #2 for a search on his alias. And it includes one of his last remarks on Roos Van Acker’s early oral exercises with a carrot. Which, if he would end up working for SBS/VT4 at some point, or if his boss turns out to be Roos’ jealous boyfriend without a sense of humour, might get him into trouble.

So I’m just saying: Twitter projects a false impression of a conversation amongst friends, and you might be tempted to say stuff a bit more harsh than you mean it. Which is why I use the “private” setting now on my Twitter account:

Now that Google changed the logo in my webmail client, I think we can say that Gmail is close to disappearing from Google’s (European) pages. They’re replacing it everywhere with the even less sexy “Google Mail”.

This might be due to the following legal hassle Google has in Europe:

The latest legal developments arrive just weeks after a European Union trademark office denied Google the rights to register the Gmail name across all of its member countries. Company representatives maintain that the EU ruling has no effect on its use of the trademark Gmail in countries other than Germany and the U.K. and that user experiences will be the same regardless of the service’s name.
news.com.com

They use the URL http://mail.google.com (no longer gmail.com or gmail.google.com) and on the welcome page there is no mentioning anymore of any “Gmail”. On the Google product overview page it’s still Gmail. Same thing for Google Talk: I see hardly any mention on “Gtalk” anymore.

If they’re gonna use common names for all their products, then maybe Froogle will become “Google Shopping”, Adsense/Adwords maybe “Google Ads” and Picasa “Google Photos”.