Four tasks. The first two on paper and in the console, the last ones with your train.py. For each — what you should see. "My machine" numbers may differ on yours; the direction is what matters.

---

1. When does it stop?

Here is the baseline validation loss of step 7, for each of the 30 epochs:

0.59 0.51 0.54 0.48 0.45 0.44 0.45 0.42 0.40 0.44 0.52 0.41 0.42 0.42 0.42
0.41 0.44 0.44 0.45 0.43 0.45 0.45 0.44 0.48 0.51 0.50 0.51 0.48 0.50 0.49

Without running code, answer: at which epoch would early stopping stop with patience = 1, 3, 5 and 10? From which epoch would the best weights come?

Then write an 8-line function stop_epoch(losses, patience) and check.

You should see: 3, 12, 14, 19. The best weights are from epoch 9 (0.40) in every case, except patience = 1: it stops at 3, and the best until then was at 2.

---

2. Dropout by hand

import torch.nn.functional as F
torch.manual_seed(0)
x = torch.ones(10)
F.dropout(x, p=0.5, training=True)
F.dropout(x, p=0.5, training=False)

Guess what each line prints. Why are the surviving values not 1 but something else? Then run it, and then take torch.ones(100_000) and look at the mean.

My machine:

tensor([0., 0., 2., 0., 0., 0., 2., 2., 0., 2.])
tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])
mean of 100 000 values: 0.9986

The survivors are doubled (1 / (1 − p) = 2) so that the mean stays ~1. And the measuring mode does nothing: it returns what it received. That is why eval() does not change accuracy by itself — it only switches the randomness off.

---

3. Write gap(history)

The function takes the list fit returned and returns the last row's train_acc − val_acc. Run the baseline (10 000 images, 30 epochs, no regularisation, seed 42) and print it.

You should see: about +0.104 (97.1 % − 86.7 %).

Now repeat with patience=5 and with dropout=0.3, weight_decay=1e-4. Notice the trap: fit with patience puts back the weights of the best epoch, but the last row of the history is still the last epoch. If you compute the gap from it, you measure not the model you hold. The correct number is evaluate(model, ...) after fit.

---

4. What patience?

The baseline MLP (256, 256), 30 epochs, Adam lr=0.001, patience = 1, 3, 5, 10. For each print how many epochs it ran and the final validation accuracy and loss (with evaluate).

My machine:

patience 1   epochs 3    train 84.1% val 82.2% val loss 0.5091
patience 3   epochs 12   train 90.7% val 85.8% val loss 0.3956
patience 5   epochs 14   train 90.7% val 85.8% val loss 0.3956
patience 10  epochs 19   train 90.7% val 85.8% val loss 0.3956

The last three rows give the same model (epoch 9's), differing only in how long it trained before that was clear. patience = 1 stops too early — the first worse step is not yet "it is over".

And one more observation: that 85.8 % is lower than 30 epochs with no stopping (86.7 %). Stopping on loss saves time but does not raise validation accuracy here — the same observation as in step 2: loss and accuracy disagree.