In lesson 9 your CNN learned from 640 pictures. It had to learn everything from scratch: what an edge is, what a corner is, what a texture is, and only then how a coat differs from a pullover.

But edges, corners and textures are the same in all photographs. Someone has already taught a network to look for them — on over a million ImageNet photographs, across a thousand categories. That network is publicly available.

Transfer learning means: take it, pull out its "eyes" — all the convolutional layers — and train only the final layer, the one that says "this is a coat / this is a shoe", on your categories.

Two variants

• Freeze the features (feature extraction). All the borrowed layers stay as they are; only the new last layer is trained. The network has 2,052 trainable parameters out of 11 million.
• Fine-tune. Besides the head you unlock the last block (layer4) and let it adapt a little. Now ~8.4 million parameters are trained — but they start out already good, not random.

What this lesson promises — and what it does not

The lesson description promised "the biggest jump in the course". We measured it — and on our main dataset it did not appear:

• Our stand-in dataset is 4 clothing classes, 28 × 28 gray pictures. A small CNN from scratch already scored about 92 % there. Frozen ResNet features — the same; fine-tuning added roughly +2–3 points.
• On natural colour photographs (CIFAR-10: cat, dog, horse, deer) the same code lifted accuracy from ~50 % to 74–79 % — more than 25 points.

In other words: the jump depends on how much your pictures resemble the ones the network was trained on. A 28 × 28 gray picture of a sweater does not look much like an ImageNet photograph; a colour photograph looks more so.

That is not a flaw in the lesson — it is the most important thing to know before you believe that "transfer learning solves everything".

The lesson climbs in steps

1. Watch. What is inside ResNet-18, how many parameters, what "freeze" means.
2. Fill in the holes. build_resnet: freeze, replace the head — two lines missing.
3. Write it yourself. count_trainable and the RGB picture path in data.py.
4. Run it on the stand-in dataset.
5. Natural photographs — where the jump really appears.
6. Six mistakes you will make (and four of them are silent).

Guess before you read on

1. Training only 2,052 parameters on 640 pictures — will you get accuracy similar to lesson 9's CNN (92.5 %)? Better? Worse?
2. If you freeze nothing and train all 11 million with the same learning rate, will it be better than the frozen network?