## Convert Black/White footage to thermographic-like video

I am doing some really cool research lately concerning video conversion and one of the issues I ran into concerns infrared imaging. In short: all the ‘cool’ thermographic (colour is dependent on radiated temperature) images are in color, and the images you get from cheap IR cameras is black and white. How do you convert the B/W into colour?

First of all: we’re talking about two different imaging technologies: the ‘real’ thermographic cameras measure only infra-red (not visible) light, are very precise, expensive and are created by companies like FLIR. They make the cameras that you might have seen in CSI.

On the other hand, you have the cheaper webcam/IP-cam devices that have ‘night-vision’. This means that, in addition to all visible light, the cameras are also sensitive to a part of the IR spectrum, and with some additional IR-lighting, one is capable of seeing in very dark places. They produce B/W images or colour, but then sometimes they show green grass as a bit purple/pink.

What I was trying to do is convert a ‘cheap’ B/W video into the coloured thermographic equivalent, but without expecting that e.g. orange will always mean 38°-39°. Since we start from a mixed visible/IR light video, brightness does not correlate perfectly with temperature. Still, the end result might be easier to interpret.

So this is my source material: a Foscam FI8918W video from Youtube: two cats playing in a kitchen.

I find some inspiration from a guy who did something similar with ImageMagick. The dark/cold areas are supposed to be black/blue, the medium areas more green, then going to yellow, orange, red and white. This seems like a job for the FFMPEG video filter.

I use the ‘curves’ filter and try first with really strong contrast colours:

The color scheme is kind of OK, but we lose a lot of detail (the cats become hard to distinguish). So then I try a more gradual approach:

And this is rather OK. I like it. The cats come out nice.

For the record: the curves used are approximately this:

`curves=r='0.4/0 0.6/1':g='.25/1 .75/.5 .9/0 1/1':b='0/1 .25/0 .75/0 1/1'`

All scripts and parameters are on my GitHub account: https://github.com/pforret/pfor_ffmpeg/tree/master/thermography

## Idea: Experimenting with movie hashing/fingerprinting

MEDIA FINGERPRINTING
Neal Krawetz wrote an interesting article on image fingerprinting, or how to search for images that are similar. He proposes an algorithm to do image fingerprinting and reproduce the functionality of TinEye, a service that allows to give one image and get back all the web pages where that picture, or a slightly modified version of it, is included. By resizing the image to 8×8 pixels, creating a B/W version, and then a binary (only black or white pixels, no grey levels), he reduces a picture to a hash that is 8 x 8 = 64 bits. This can then be compared to a database of hashes of millions of other pictures found on the web (by calculating the “Hamming distance” – read the article for details).
On the other hand, something similar can be done on segments of audio. Youtube has been doing it for years (using technology from Audible Magic) and recently the Echo Next has released Echo Print, a music fingerprint and identification service that does the same thing for free.

VIDEO FINGERPRINTING
Since I work (and play) a lot with video, I was thinking about how to extend the ideas Neal proposes to video. Video material consists of audio (which I will ignore for now) and a sequence of images (typically between 24 and 30 per second). I’ve taken a video clip from Youtube as inspiration, but I won’t tell you yet which one, let’s discover it while I create the fingerprint of 1 of the frames, using the method described by Neal:

 first we reduce the frame to a small format, e.g. 16×8 pixels then we desaturate: make the picture grayscale then we normalize it: we maximize the contrast, the darkest pixel become the new black, the lightest becomes the new white we calculate the average darkness and any pixel that is darker we make black, and the others become white -this is what is proposed in the article. Total size of fingerprint: 16 pix * 8 pix * 1 bit = 96 bits = 16 bytes I found the details in the above fingerprint too coarse, so I used 4 color levels instead of 2. Total size of fingerprint = 16 pix * 8 pix * 2 bits = 32 bytes

## My quest for “Let’s get lost” – Chet Baker by Bruce Weber

Let’s Get Lost (1988) is a American documentary film about the turbulent life and career of jazz trumpeter Chet Baker written and directed by Bruce Weber.

I saw Let’s Get Lost in Leuven, I guess around 1990. Chet Baker has been a weak spot for me ever since, because I now knew how much suffering was hiding behind that vulnerable voice. I tried to find a DVD of it but apart from a VHS tape (I don’t have a player) I couldn’t find anything. Then the other day I found the whole movie, split in 13 parts, on Youtube:

## Don’t send me a video, send me a link

I know, there are so many ‘funny’ videos you just have to share with your friends. So you send them an email. But for god’s sake, not with a 5MB movie in attachment! For all you know, he/she might not even be able to play that MOV/WMV/XVid movie anyway. Don’t send a movie, send a link!

1. Email makes big files bigger
Binary files (like videos) are encoded, or rather exploded, by your email program (Outlook/Hotmail/Gmail/…) as text-only Base64 MIME attachments. Your 5MB file is transformed into a 6.85MB text file before is sent. Email is a very inefficient way to share videos with several other people.
2. You hurt the recipients
Your email will have to be downloaded before the recipient can see it. If he is on a slow connection, this might mean 15 minutes of obnoxious delay before he can continue working, start receiving the emails that arrived after your ‘cute puppy’ movie. The movie, if it is not deleted, will add 5 MB of storage to the Inbox. If his Outlook/Exchange quota is 100MB (not uncommon on corporate email systems), you just ate 5% of all the place he has to store contracts, meeting reports and office gossip.
3. You hurt yourself
By sending a 5MB video, you force your email program to upload a 6.85 MB file to your mail server. If you’re on a basic DSL line, this will easily take up to 10 minutes, during which all your other Internet activity will go very slow. You also add a big chunk to your “Sent Items” folder, bringing you closer to your quota limit.
4. You hurt the Internet
All those forwarded videos make for a huge amount of unnecessary traffic that eats up bandwidth at ISPs and inspire them to keep prices high. Not that they needed the extra inspiration.
5. It’s force-feeding-video, not video on demand
You are forcing people to download the whole file before they can decide whether they want to see it now, or ever at all. Youtube and the other sites have a very easy-to-use ‘Send video link’ form that will give the receipient the link, with a screenshot and the description text. Then he/she can decide when, where, how and *IF* to watch the video.
(Yes, this is less a problem with web-based mail like Gmail or Hotmail)

HOW TO FORWARD A VIDEO LINK

1. public, popular movie
Don’t think you’re the first one to have seen this movie. Chances are it’s featured on Youtube, Google Video, DailyMotion, Vimeo, in multiple versions (FR subtitles if that’s what you like), in a format everyone can view, available to send as just a link “`http://www.youtube.com/watch?v=_RgL2MKfWTo`“. Less that 50 characters for a full 1:14 of hilarious time loss.
2. private, ‘secret’ movie
Even if you have a movie you recorded/made yourself and want to show only to a limited number of people (“OMG, Britney, you were, like, *so* drunk!!“), then upload it yourself to Youtube, Flickr or Vimeo, put a password on it and send link+ password to those recipients. It will be so much easier for everyone to forward that secret video that no one was supposed to see (“788 views just yesterday? How’s that possible?“).

We thank you.