PyTorch error messages look menacing, but they speak very precisely. In this step you break code on purpose, so that the next time you see the same messages you know what they mean.

Nothing to declare — open a Python console (python, then import torch) and type line by line. All the output below is real.

1. Shapes do not match

torch.ones(2, 3) @ torch.ones(2, 1)

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x3 and 2x1)

Read it: (2x3 and 2x1). The inner numbers — 3 and 2 — differ. The product needs [2, 3] @ [3, ...]. This error is your friend: it shows both shapes, and all that is left is for you to say which one is wrong.

2. An integer cannot have a gradient

torch.tensor(3).requires_grad_()

RuntimeError: only Tensors of floating point dtype can require gradients

torch.tensor(3) is int64. From 3 to 3.0 is one dot, but that dot is the difference between "can learn" and "cannot".

3. backward() from a non-number

w = torch.tensor([1.0, 2.0], requires_grad=True)
(w * 2).backward()

RuntimeError: grad can be implicitly created only for scalar outputs

A gradient answers "how would this one number change". The loss is always one number, so that is how it should be — if you got this error, your "loss" is a tensor with more than one element. Usually you forgot .mean() or .sum().

4. A second backward() on the same computation

w = torch.tensor(3.0, requires_grad=True)
loss = (w * 2) ** 2
loss.backward()
loss.backward()

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). ...

After the first backward() PyTorch throws away the intermediate values it was following — they are no longer needed. For the next computation you need a new loss. (The message suggests retain_graph=True; that is a rare case and not what you are doing here.)

5. The silent one: gradients accumulate

This one raises no error.

w = torch.tensor(3.0, requires_grad=True)
for i in range(3):
    loss = (w * 2) ** 2      # d loss / d w = 8 * w = 24
    loss.backward()
    print(f"after backward #{i + 1}: w.grad = {w.grad.item()}")

after backward #1: w.grad = 24.0
after backward #2: w.grad = 48.0
after backward #3: w.grad = 72.0

Remember this one: in lesson 4 it becomes one of the most important things, because the same code is repeated hundreds of times.

6. numpy() from a tracked number

torch.tensor([1.0], requires_grad=True).numpy()

RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.

The message tells you what to do: detach() — "cut it loose from tracking". It gives you a copy without history.

7. Integers and the mean

a = torch.tensor([1, 2, 3])
(a * 0.5).dtype     # torch.float32 — the product became float
a.mean()

RuntimeError: mean(): could not infer output dtype. Input dtype must be either a floating point or complex dtype. Got: Long

The same dtype principle applies, from the other side: Long is int64, and a mean is not defined for it. The fix: a.float().mean(). fashion.py also converts the pictures to float32 explicitly: .float() / 255.0.

What to take away

Six of the seven messages tell you what and where. They state shapes, types, names. The fifth is not an error but silence, and the only way to catch it is to look at the numbers: is w.grad really what you expect?