In lesson 7 your MLP reached 97.1 % training and 86.7 % validation accuracy. That 10-point gap was overfitting: the network memorised pictures instead of learning what a garment looks like.

This time the question is different. Not "how do we stop the memorising" but: does an MLP look at the picture at all?

What an MLP sees

An MLP receives 784 numbers — and nothing else. The 28 × 28 picture is unrolled into one row (x.reshape(len(x), -1) in fashion.py), and pixel 1 is just as much a "neighbour" of pixel 2 as of pixel 784. It does not know that pixel 29 is directly below pixel 1.

Two things follow:

1. Every pattern has to be learned again at every position. The corner of a shoelace at the top left and the same corner at the bottom right are two unrelated things to an MLP.
2. The weight count explodes. One dense layer turning 784 pixels into 16 × 26 × 26 numbers has 8,490,560 parameters. A convolutional layer producing the same-sized output has 160. You will see how below.

The idea

A convolutional network (CNN) rests on two observations about pictures:

• Patterns are local. You recognise a corner, an edge or a blob by looking at a small window, not at the whole picture.
• The same pattern can show up anywhere. So one small filter can be carried across the whole picture, using the same weights at every position (weight sharing).

Everything else — pooling, channels, the final dense layer — follows from these two.

What this lesson actually promises

The lesson climbs in steps:

1. Watch. One 3 × 3 filter on a tiny picture, every number written out. You write nothing.
2. Fill in the holes. A convolution with the loops given — two lines missing.
3. Write it yourself. Pooling and the CNN itself — empty skeletons.
4. Run it and measure.

And honestly about the result. On this course's 10,000-image subset a CNN does not produce a miracle jump. In our runs it beat the MLP by about 1–2 points, not ten. Its real advantage is different: the same number of parameters buys a better network, and it reasons about the picture's structure. Why, and how to check it, is in steps 5 and 6.

Guess before you read on

Write down:

1. Two networks, both around 20,000 parameters: an MLP and a CNN. Which wins on validation, and by how many points?
2. If you shuffle the pixels of all 10,000 training pictures with one and the same random order (the picture becomes unrecognisable), which network gets worse?