Drills
Four tasks, from easy to sly. Write them in the console or in a separate drills.py — they do not go into the main project. After each one you are told what you should see, so you know whether you got it.
1. Shape arithmetic
You have t = torch.zeros(5, 28, 28). What shape will each of these give before you run it, and then check:
t.reshape(5, -1)t.reshape(-1)t.sum(dim=0)t.mean(dim=(1, 2))t[2]t[:, 14]
Check: [5, 784], [3920], [28, 28], [5], [28, 28], [5, 28]. If you got sum(dim=0) wrong, think about which axis disappears: the one you sum over.
2. Which category is brightest
Load load_fashion("train", n=1000) and compute, for each of the 10 classes, the mean pixel value. Use boolean indexing: x[y == 3] picks only the rows whose label is 3.
Which class is brightest, which darkest? Did you guess?
Check: the brightest is Pullover — 0.394, the darkest Sandal — 0.132. After them: Coat 0.385, Bag 0.339, Shirt 0.329, T-shirt 0.325, Ankle boot 0.296, Dress 0.274, Trouser 0.226, Sneaker 0.166. Class sizes are uneven (86–115), but the mean takes care of that.
(This is the first time numbers tell you something about the data. A likely reason is that sandals are a thin construction with a lot of empty background — but that is a guess, not something measured.)
3. A gradient for a vector
Before running, write down your answers, then check with .grad:
w = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
(w ** 2).sum().backward() # what w.grad do you expect?
and
w = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
x = torch.tensor([10.0, 20.0, 30.0])
(w * x).sum().backward() # and now?
Check: tensor([2., 4., 6.]) and tensor([10., 20., 30.]). The second matters: the gradient of a weighted sum with respect to a weight is the input itself. That is exactly why lesson 3's neuron weights "look at" pixels.
4. The silent broadcast
You have x = torch.rand(10, 784) — ten flattened pictures. You want to subtract each one's own mean from it:
x - x.mean(dim=1)
Run it. What happens?
Now change to x = torch.rand(784, 784) and run the same line. And then with keepdim=True. After each, check whether each row's mean became zero: result.mean(dim=1).
Check: with [10, 784] — an error (The size of tensor a (784) must match the size of tensor b (10) at non-singleton dimension 1). With [784, 784] — no error at all, but the row means are about 0.01, not zero, because every row subtracted not its own mean but a column's. With keepdim=True — about 5e-08 (a true zero, apart from floating-point noise).
The takeaway: when shapes "happen" to match, PyTorch will not warn you. What breaks one test can silently break your whole code.