When AI learns the wrong thing

At the end of 3.2 we asked: with enough layers and enough weights, can it solve anything?

It doesn't work like that. This chapter is about two ways of learning the wrong thing.

Two sets of photos

Below are two sets of photos, A and B. Pick one and teach the machine with it.

Look through the photos before you press anything. That's the whole lab, really.

How many did it get? Now look at only the misses.

The ones with pink borders. What do they have in common?

The ones with the backgrounds swapped

Got it? Every miss is a cat on grass or a dog in snow.

Look at set A again.

So what the machine learned wasn't "cat vs dog." It was "white background vs green background."

Both worked equally well. Inside the photos we gave it, being in snow simply meant being a cat.

Tap a test photo and press See where the AI looked. It covers the photo one patch at a time and lights up wherever the answer wobbles most. The background lights up, not the animal.

The fix isn't in the code

Now teach it again with Photo set B.

In B, half the cats are in snow and half on grass — and the same for the dogs. Background can't tell them apart any more. So the machine has no choice but to look at the animal.

Side by side:

Same machine. Same arithmetic. All we changed was the photos.

You met this once already in 3.1. Always shooting from the same spot taught it "the thing on the left side of a desk." Same thing happening here.

The second way — memorising

There's another way to learn the wrong thing. This one isn't about photos. It happens when there are too many weights.

The more weights you add, the more the curve wriggles between the red dots. It's trying to get every practice question right.

But watch both scores.

• The practice score goes up

• The test score goes down

The test score wobbles up and down on the way. Try it one notch at a time, then push it all the way.

The curve has bent itself to fit only the practice questions, so when a new one turns up, the weights are in the wrong place.

That's called overfitting.

Memorising versus understanding

People often say overfitting means "the AI memorised it." Handy, but be careful with it.

The AI didn't write the answers down anywhere. There's no drawer of stored answers.

Here's what actually happened. The weights got turned to fit only the practice questions. So when a new question comes along, those weights are pointing somewhere useless.

Now take weights away again. The test score comes back. Stop it memorising and it does better.

That's every way of teaching so far

Up to now there's been one way to teach an AI: show it the answers.

But how do you teach something nobody knows the answer to? Riding a bike, say.

Next chapter, no answers at all — just rewards and penalties.