In my day job I am a database programmer. I love data. The reason I love data is twofold. Firstly, within data there resides stories and I adore stories. Secondly, data offers a challenge in its presentation. How do you present data in a manner that is not only meaningful but also engages and inspires?

For work I have been asked to find out more about using Machine Learning to analyse data (I think almost everyone in my field across all workplaces is being prodded by their bosses to do the same). It is absolutely astonishing once you start getting into it.

But I am not writing this post to wax lyrical about Machine Learning from a data scientist perspective. No. If you have ever watched the excellent film Alien, then you will remember the chest-burster scene where the baby alien erupts out of John Hurt’s chest. Well, I had the metaphorical equivalent when I discovered that Machine Learning can colourise (in American “colorize”) old photos and film. The chest-burster exploded from inside of me yelling “what artistic things can I do with such a tool?”

My answer: a lot.

Here is a link to one such tool called DeOldify. The picture at the top of this post is an old photo of my great grandfather, William Clark Curling, in his Irish Lancer’s uniform. I have run it through DeOldify. In all honesty DeOldify has done a pretty average job compared to a human artist trying the same. (NOTE: take a look at their web site. There are some spectacular transformations there. I’m just being mean by showing this example.) Below is an artistic attempt I did on that photo using Photoshop a few years back.

DeOldify, as is, could not have done this. Why? The same reason that I could not have done this with just the original photo. That is, I did not know the colours involved (the uniform). Yes, I reckon that a Machine Learning app could clear up the spots and other visual corruptions. It could also identify the background and assign a uniform colour. But the uniform? Well, maybe if we had a comprehensive visual (color) listing of uniforms that was sufficiently good enough to identify the 5th Irish lancers uniform of 1890. In my case, I happened to know his regiment and simply looked up the colours.

What about another situation. A photo taken in the middle of the day and a photo taken around the so-called golden hours at either end of the day. The colour of the light is different. Even in different parts of the world the colour of the light can be subtly different. The colour of grass and foliage too. When I first took my daughter to England she commented on how green the grass looked compared to grass in Australia.

In an old black and white photo is there sufficient information to inform any Machine Learning app of location, time of day, etc? Maybe in some cases (perhaps angle of shadows on an old picture of the Eiffel Tower). But not in a lot.

Ok, so where am I heading?

Color palettes. From what I understand, Machine Learning is about teaching the neural net to apply different colours to different objects. Can we teach it to discern time of day from shadows and apply an appropriate palette? Can we teach it to request further information where necessary (i.e. “I have identified a uniform in the photo. What colour scheme does the uniform use?” or “In what country was this photo taken?” or even “Use default or custom palette for photos?”).

Recently I watched a great steam-punkish series called Carnival Row. The night-time city palette used in the cinematography was mostly teals and blacks with light-sources being a glorious yellow/orange. I could just imagine applying such a palette to an old photo of a city. Perhaps not the real colours of the time, but how striking would it be!

I really need to learn how to do this.

Thanks for reading my ravings.

Greg