Theory

What is inside ResNet-18

There is no code here. This is what you see when you open the resnet18 that torchvision downloads pretrained (about 45 MB, once). Every number was really printed.

What is inside

total parameters: 11,689,512
conv1      Conv2d                    9,408
bn1        BatchNorm2d                 128
relu       ReLU                          0
maxpool    MaxPool2d                     0
layer1     Sequential              147,968
layer2     Sequential              525,568
layer3     Sequential            2,099,712
layer4     Sequential            8,393,728
avgpool    AdaptiveAvgPool2d             0
fc         Linear                  513,000

The same idea as your CNN from lesson 8, only deeper: conv1 looks for the simplest patterns (edges), layer1 … layer4 for ever more complex ones, and fc is the last layer, which decides the class from the features.

fc: Linear(in_features=512, out_features=1000, bias=True)
output for 2 images: (2, 1000)
features before fc: (2, 512)
  • Before fc every picture is already 512 numbers — "what this network sees in this photograph".
  • fc turns those 512 numbers into 1,000 ImageNet class scores. We need 4 (or as many as you have categories).

Freezing and a new head

after build_resnet(4): fc = Linear(in_features=512, out_features=4, bias=True)
                       trainable, total = (2052, 11178564)
with layer4:           (8395780, 11178564)
layer4 params:         8,393,728

The new head: 512 × 4 + 4 = 2,052 parameters. The whole network now has 11,178,564 (going from 1,000 to 4 classes shrank fc from 513,000 to 2,052). Only those 2,052 are trainable. With layer4 — 8,395,780, because layer4 alone has 8,393,728.

What the input must look like

ImageClassification(
    resize_size=[256]  crop_size=[224]
    mean=[0.485, 0.456, 0.406]
    std=[0.229, 0.224, 0.225]
)

This network was trained on 224 × 224 colour pictures whose pixel values were normalised with exactly those three means and three standard deviations (one per channel — red, green, blue). Your 28 × 28 gray PNG matches none of the three. That is one part of this lesson: pictures have to be prepared the way the network saw them.

Three observations

1. Only 0.018 % of the network is really new. 2,052 of 11,178,564. The rest are weights that someone has already learned.

2. The fourth block is three quarters of everything. layer4 holds 8.4 million of 11.2. "Fine-tune layer4" means training 3/4 of the network on 640 pictures — which is why the learning rate must be small (0.001, not 0.01).

3. The new head starts out random. Before training our validation accuracy was 24.4 % — four classes, a random head, a random guess (chance = 25 %). Everything it "learns" depends on what the 512 numbers can offer.

What next

In the next step you write that build_resnet: two lines that do everything you just saw.