There is no code here. This is what a folder reader does with the stand-in dataset — every number really printed by the finished version of our data.py (which you write in steps 4–6).

The folder

standin_data/
  coat/       0000.png ... 0199.png
  pullover/   0000.png ... 0199.png
  sneaker/    0000.png ... 0199.png
  tshirt/     0000.png ... 0199.png

The label is the folder name. There is no file anywhere saying "this picture is a coat".

class counts: {'coat': 200, 'pullover': 200, 'sneaker': 200, 'tshirt': 200}
classes: ['coat', 'pullover', 'sneaker', 'tshirt']
class_to_idx: {'coat': 0, 'pullover': 1, 'sneaker': 2, 'tshirt': 3}

The numbers come from the alphabet, not from what you think of the classes: coat = 0 only because c sorts before p. Remember that — it will matter in lesson 11.

One picture

one image: (1, 28, 28) torch.float32  min 0.00 max 1.00  label 0 coat

Shape (1, 28, 28): 1 channel (gray), 28 × 28. Pixels run from 0 to 1. This is exactly the shape the CNN from the last lesson accepts.

The split

train 640   val 160
val per class: {'coat': 40, 'pullover': 40, 'sneaker': 40, 'tshirt': 40}

Chosen at random for each class separately: 20 % of every class → 40 each for validation. So validation holds every class equally, rather than "however it came out". One validation picture = 1/160 = 0.625 points.

Normalisation

train mean 0.3125  std 0.3588
all-images mean 0.3147  std 0.3595

Before handing numbers to the network we subtract the mean and divide by the standard deviation, so the values sit around 0 with a spread of about 1. The mean and std are measured only on the training pictures (the top line). The difference to the line below — all pictures — is small here (0.3125 vs 0.3147); we left it measured and shown so you can see that however small, validation pictures must not take part in the calculation. They have to be truly "unseen".

The first batch (32 pictures) after normalisation:

batch: (32, 1, 28, 28)  labels [2, 1, 0, 0, 3, 0, 1, 3]
normalised batch mean 0.038  std 1.029

(The batch mean is not exactly 0 — it is only 32 pictures out of 640.)

Augmentation

The same file, loaded twice:

same file, augmented twice: max abs difference 2.525
same file, plain twice:     max abs difference 0.000

Without augmentation the picture is identical every time. With it, every time a little different (flipped, rotated, shifted). The network sees the same number of files, but each epoch a different variant of them.

Three observations

1. A folder is not a dataset. A dataset appears only when you decide: how to split, how to normalise, what to change at random.

2. Splitting and normalising are decisions, not technique. They have to protect the validation set. Mistakes are not reported.

3. Augmentation is for training only. Validation pictures must stay as they are: plain twice gives the same result every time.

What next

In the next step you prepare your own photographs. After that — a data.py that does everything you just saw.