In the last step you saw two things: how autograd computes a gradient, and how [28, 28] becomes [784]. Now you write them as Python — and one line is missing in each.

The setup

Make one folder for the whole course project (the one where check_setup.py already lives). Every file of this course will live in it, side by side — later lessons use them.

And run the tests from that folder, not from another one:

python -m unittest

unittest is part of Python's standard library, nothing to install. It finds every test*.py file in the folder and runs it.

Your job

Create tensors_demo.py and test_tensors_demo.py (see the right-hand panel).

tensors_demo.py is complete and runs, but gives the wrong answers: two places are marked HOLE 1 and HOLE 2.

• HOLE 1 — loss_and_grads. The loss is already computed, but the gradients were never requested. One line is enough: ask PyTorch for them from loss. (You saw which method that is in the last step.)
• HOLE 2 — flatten_image. Turn [28, 28] into a single axis of 784 values. reshape accepts -1, which means "work this size out yourself".

test_tensors_demo.py is the verdict — you do not write it, you pass it. The test recomputes the same loss = 9, w.grad = −12, b.grad = −6 you saw in the table.

Run the test before filling anything

First as it stands. It is not a mistake, it is the starting point:

$ python -m unittest
FFFF
======================================================================
FAIL: test_gradient_is_zero_when_the_prediction_is_perfect (...)
AssertionError: unexpectedly None : w.grad is None - did you call backward() on the loss?
FAIL: test_gradients_match_the_hand_calculation (...)
AssertionError: unexpectedly None : w.grad is None - did you call backward() on the loss?
FAIL: test_flatten_gives_one_dimension_of_784 (...)
AssertionError: Tuples differ: (28, 28) != (784,) : got shape (28, 28), want (784,)
FAIL: test_flatten_keeps_row_order (...)
AssertionError: 2 != 1 : got 2 dimensions, want 1
----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=4)

(The real output also has long tracebacks; only the important lines are kept.)

Notice that the code did not crash. w.grad is None — PyTorch does not complain that you never asked for gradients; it simply did not make any. That is why the test says it in plain words.

Fill both holes and run again — you should see OK.

When it does not pass

All of this was tried, not imagined:

• w.grad is None although you wrote something with backward — most likely you forgot the parentheses: loss.backward without () is the name of the method, not a call of it. Python does not complain.
• 1.0 != 0.0 (the gradient in the "zero error" case) — you called .backward() on the wrong object, on w. A gradient is taken of the loss, not of a parameter: w.backward() fills w.grad = 1, and the only thing that can say is "how w changes itself".
• got 2 dimensions, want 1 with reshape(784, 1) — you got a column, not a row. [784, 1] and [784] are different things, and the product with the weights later will throw an error.
• 28.0 != 1.0 ... second value should be pixel (0, 1) with image.T.reshape(-1) — you transposed. The pixel count is the same, but the order is now column by column rather than row by row.