Lesson 5's model is ten neurons, each computing x @ w + b. Each of them is one straight line (in higher dimensions — a plane): "yes" on one side, "no" on the other. A single-layer network can do nothing else.

Is that a problem? Sometimes not: you can separate T-shirts from boots with a line. But here is data you cannot separate with a line:

        . . . . .
      .  o o o o  .
     .  o       o  .        o = inner ring (class 0)
     .  o   +   o  .        . = outer ring (class 1)
     .  o       o  .
      .  o o o o  .
        . . . . .

The inner ring is surrounded by the outer one. Draw any straight line — both sides will hold points of both classes. In this lesson that is not an illustration but a real number: one layer reaches about 54 % on this data, where guessing gives 50 %.

What this lesson brings

What hurt	What arrives
one layer = one line	a hidden layer between input and output
two layers with nothing between them = still one line	the ReLU activation: max(0, z)
"how many numbers does the model have?"	count_parameters
SGD with the same lr stopped working	the Adam optimizer

ReLU is one simple thing: it turns negative numbers into zero and leaves positive ones alone. It looks laughably simple. But that "bend" is exactly what stops a stack of layers from being one line.

The lesson climbs in steps

1. Watch. The rings, two Linear layers that equal one, and the ReLU that breaks that — with real numbers.
2. Fill the holes. The MLP class arrives almost complete. The ReLU and the parameter count are missing.
3. Write it yourself. The optimizer swap and build_model.
4. Run it. First the rings — there the leap is real. Then FashionMNIST — there the leap is modest, and you will see why.

Honest in advance

On FashionMNIST the gain in this lesson is small: roughly 4 points of training accuracy (85.5 → 89.5 in one pair of runs, 84.3 → 87.9 in another). The big leap is on the rings: 54.3 % → 100.0 %. Learning is really about how much your data's structure demands non-linearity; images of boots and T-shirts demand little.

And again: we are measuring training accuracy. In lesson 7 you will see the real one.