Smart?

We keep hearing that computers are smart.  We hear about smart tablets, smart phones, i-pads, laptops.  Certainly, these devices are truly amazing.  Not so much because they are clever – which I want to argue they are not – but because the people who designed and built them are brilliant geniuses, and yet they sell them to us for amazingly little money.  Computers are able to do calculations very quickly.  It’s getting to be the case that the speed of light is the limiting factor.  But that’s what they do.  Compute, period.  This fact tends to get hidden behind a flood of Gigabytes, Cycles, Megabits per second, Megapixels and the like.

In the early days of so-called artificial intelligence, there were high hopes that programs such as neural networks would not only be able to duplicate the functions of the brain, but would also provide models of how our brains worked.  In other words, we could better understand ourselves through the behavior of our artificial creations.  We’re about twenty years older and a lot wiser now.

There’s no area where the difference between intelligence and computational speed is as dramatic as in the area of images. It's people who are smart.  We look at pictures and, without giving it a thought, we discover shapes, motion, assess our environment, obtain information.  We have a pretty good idea about how our eyes take the reflected light from what they see and code that for the brain.  We have much less idea about how that coded data ends up as information – in color and 3D no less – in our minds. This is a massively difficult problem and the best computer codes are not impressive at doing it.  Yet we do it at a casual glance.

Computers can’t even do this stuff with letters – by far the easiest situation since what we see is already a conveniently set up code.  That’s what the Captchas are all about.  Sure the letters are a bit misaligned, maybe fancied up, different scale, different backgrounds, but so what?  We see what they are immediately.  Computers battle and usually fail.  The whole point of a Captcha is to separate computers from humans!

Most of practical image processing is about making pictures more attractive or useful to humans.  Fortunately this is a much simpler problem than computer vision.  We do know quite a bit about what makes things easier for us.  We pick out shapes and objects in pictures and classify them, and we do this by identifying edges in the picture.  When we complain that an image is blurred or “soft”, we're actually uncomfortable because the edges are less distinct. 

Raw Saturn

Saturn edge enhanced

Here is an example.  The picture on the left is a raw image of Saturn from a space probe.  The one on the right has been focused.  Right?  Wrong.  We’ve just marked in the edges more clearly - something a program can do easily – and nothing else has changed.

We are much more tolerant of color variation; obviously it's much less important to us in doing that classification stuff.  Of course we want our pictures and movies and TVs in color, but some of us are old enough to remember when they were in black and white (well TV anyway!)  We never had any trouble identifying what was going on in the picture.  Color is a nice-to-have.

So I see my corgi, Megan, wondering around the garden, looking and sniffing, and I really wonder what she's seeing (in black and white) and smelling and how she interprets it all in her mind.  How could one ever tell?  All I know for sure is that what she’s doing is one hell of a lot smarter than any computer.

Michael – Thursday.