Four tasks. You can do them all in a new file or in the console, without touching the project files (but you may import from model import ...). For each — what you should see. "My machine" numbers may differ on yours; the direction is what matters.

---

1. Count the parameters by hand

MLP(hidden=(256, 128)), input 784, output 10.

Write down each layer's number of weights and biases and add them up. Then check with count_parameters.

You should see: 784·256 + 256 = 200 960, 256·128 + 128 = 32 896, 128·10 + 10 = 1 290; 235 146 in total.

---

2. Fold two layers into one

lin1, lin2 = nn.Linear(784, 128), nn.Linear(128, 10)
x = torch.rand(5, 784)

Build one nn.Linear(784, 10) that gives the same as lin2(lin1(x)). A hint: weight = lin2.weight @ lin1.weight, and you work out the bias yourself — think about what lin1.bias does after the second layer.

Check with torch.allclose(...) and compare the parameter counts.

You should see: True (difference ~10⁻⁷), 7 850 numbers instead of 101 770. Now put torch.relu between lin1 and lin2 — and see that the same trick no longer works.

---

3. XOR

Four points: (0,0)→0, (0,1)→1, (1,0)→1, (1,1)→0.

Train LinearClassifier(2, 2) and MLP(2, (8,), 2) — full batch (all 4 points), Adam, lr = 0.05, 500 steps. Print the final loss and accuracy.

My machine (seed 42):

linear   loss 0.6931  accuracy 50.0%
mlp      loss 0.0001  accuracy 100.0%

0.6931 = ln 2: the linear model cannot push the loss below a "coin toss". With other seeds the accuracy may be 75 % (a line can classify three of the four points correctly), but never 100 %.

---

4. How wide must it be?

With the circles.py data (seed 42, batch 32, lr = 0.01, 20 epochs) train MLP(2, (h,), 2) for h = 1, 2, 3, 4, 8, 16.

What is the smallest h that solves the rings?

My machine:

width 1   params 7    loss 0.6930  acc 51.0%
width 2   params 12   loss 0.6894  acc 51.7%
width 3   params 17   loss 0.0869  acc 99.8%
width 4   params 22   loss 0.1437  acc 97.9%
width 8   params 42   loss 0.0208  acc 100.0%
width 16  params 82   loss 0.0056  acc 100.0%

A sudden jump between 2 and 3: 51.7 % → 99.8 %. One can guess that a closed shape in the plane needs at least three "edges", so three neurons are the first threshold — but that is a guess, and I tried one seed. Try others. Notice that h = 4 (97.9 %) loses to h = 3 (99.8 %) — one run cannot prove that "more is better".