Two tasks: measure how much chance "wobbles" on your machine, and settle on your own baseline.

1. The spread of chance on your machine

Run the untrained model with torch.manual_seed(s) for each s from 0 to 9, and print the 10 accuracies.

You should get:

3.3%  11.4%  8.0%  10.9%  11.3%  9.2%  15.2%  9.0%  8.8%  6.5%    mean 9.4%

If your numbers differ — do not worry: it is another PyTorch version or another processor. What matters is that the numbers are scattered around 10 %, not clustered at 10 %.

Answer in one sentence: by how many percentage points does accuracy have to rise after training for you to say "the model learned something", not "got lucky"?

2. Your baseline

Go back to your categories from lesson 1. Answer in writing:

• How many classes will your classifier have? What is its "random" accuracy (1 / number of classes)?
• Suppose one class has 3 times more pictures than each of the others. What is the "always say the most common class" accuracy? Write the formula.
• On FashionMNIST this number for the first 10,000 pictures is 10.27 % (Trouser — 1,027 of 10,000). Check: torch.bincount(y).max() / len(y). Why is it a little higher than the "random" 10 %?

What to hand in

The ten numbers from part 1 and the answers to part 2. All of it is the first line of your report: every later accuracy number of your classifier will get a "baseline" column beside it.

Tip. In the next lesson the neurons start learning for the first time. One step — and accuracy jumps from 10.2 % to more than 28 %. Keep this lesson's neuron.py and fashion.py — you will need them.