Three failures: one loud, two silent
Three failures. The first is loud, the other two silent. All printed values are real; you can run the code in a console (from neuron import *, from fashion import load_fashion).
1. Loud: the picture is not flattened
x, y = load_fashion("train", n=10_000)
img = x[0].reshape(28, 28)
neuron(img, W, torch.zeros(10))
RuntimeError: mat1 and mat2 shapes cannot be multiplied (28x28 and 784x10)
It is like lesson 2's first error: the inner numbers (28 and 784) do not match. The neuron expects [784] (or [n, 784]) and got a table.
2. Silent: softmax over the wrong axis
Softmax has to divide by the row sum (dim=1). Forget that — and write dim=0:
z = neuron(x[:4], W, torch.zeros(10))
bad = torch.exp(z) / torch.exp(z).sum(dim=0, keepdim=True)
row sums (should be 1.0): [2.6498520374298096, 2.4395833015441895, 2.4337193965911865, 2.4768457412719727]
column sums: [1.0, 1.0, 1.0] ...
No error. The same shapes, numbers between 0 and 1. But every picture's "probabilities" add up to 2.4–2.6, not to 1. Now each column adds up to 1: that is "which of the 4 pictures is most like a T-shirt?", not "which class is this picture?".
Would accuracy show it? With all 10,000 pictures and dim=0, the same untrained model gets 6.8 % instead of 10.2 %, and its answer agrees with the correct softmax's answer on only 25.9 % of pictures. But 6.8 % is inside the same 3 % – 15 % spread you saw in the last step — so the number says "well, chance". What catches it is not accuracy but one check: do the rows add up to 1. That is what your test_rows_sum_to_one does.
3. The sneakiest: accuracy lies
Let us ask differently: not "which of 10 classes?" but "is this a sandal?". A neuron that always says "no" computes nothing — and its accuracy is:
binary accuracy of 'always no': 90.1%
fraction of sandals in the data: 9.9%
90.1 %. With no training, with no idea what a sandal is.
The reason is simple: sandals are 9.9 % of the data, so the answer "no" is right in 90.1 % of cases. A number like "90 %" sounds wonderful and means nothing until you know what number a fool would get.
That is why we talked about a baseline in the previous step. It always comes together with accuracy:
| task | the foolish answer | accuracy |
|---|---|---|
| 10 classes, one of them | random | ~10 % |
| "is this a sandal?" | always "no" | 90.1 % |
On the first row, 40 % would be a great result. On the second, 90 % is empty.
When in later lessons you see "model accuracy X %", the first question is always: and how much would a fool get?
What to take away
- Loud failures are a gift. They tell you where to look.
- Silent ones are caught only by a test or a check, not by the result itself. Row sums, shapes, means — trust them more than "looks good".
- Accuracy without a baseline says nothing.