Convolutional Networks (CNN)

Code

Does the network see the picture? Shuffled pixels

Back to the question from step 1: which network gets worse if you shuffle the pixels?

This is not rhetoric — it is an experiment you can run. The idea is simple:

Draw one random order of the 784 pixel indices, the same for every picture (perm).
Rearrange the pixels of every picture (training and validation) in that order: x[:, perm].
Train the network and compare.

To a person the picture becomes noise. But you lost no information in any pixel — the very same numbers, just in different places.

Position does not matter to an MLP: it already treated every pixel as a separate input. If the CNN uses neighbourhood, it must get worse.

The file

Create shuffled.py (see the right panel). It is an experiment tool, not an exercise, so it is complete — you only run it. It uses the fit, MLP and CNN you already have:

python shuffled.py

Our run (10 epochs, 10,000 training and 5,000 validation pictures, seed 42):

MLP (256, 256)  val acc normal 85.0%   pixels shuffled 84.9%
CNN (16, 32)    val acc normal 87.2%   pixels shuffled 84.3%

The program is silent for a long time: it trains four networks one after another (an MLP and a CNN, each twice). A few minutes is normal.

What you see

MLP: 85.0 % → 84.9 %. A difference of 0.1 points, which is noise. The MLP did not notice the picture was shuffled, because it never knew what was next to what.
CNN: 87.2 % → 84.3 %. It loses 2.9 points — and drops to the same level as the MLP. All of its lead was the neighbourhood that the shuffle destroyed.

So the answer to the step 1 question: the CNN gets worse. And that is the best evidence that it does what it was meant to do — it uses the structure of the picture, rather than just having fewer parameters.

Honestly. This is one seed and 5,000 validation pictures, so ±1 point is normal noise. A 2.9-point drop is noticeable, but you should not trust a single number — change torch.manual_seed(42) in val_accuracy, run it again, and see whether the direction holds.

What next

The CNN is the first network in this course that actually knows it is working with pictures. The next lessons feed it your own photographs — and there the data and how you prepare it decide more than the architecture does.

shuffled.py CREATE

Create this file in ./

"""shuffled.py — does the network use the picture, or only the numbers in it?

Apply ONE fixed permutation to the 784 pixel positions of every image. A
person can no longer recognise anything; to an MLP nothing changed, because it
never knew which pixel was next to which.
"""

import torch

from fashion import load_fashion
from model import CNN, MLP
from train import fit, make_loader


def val_accuracy(make_model, x, y, xv, yv) -> float:
    torch.manual_seed(42)
    model = make_model()
    loader = make_loader(x, y, batch_size=64, shuffle=True)
    val_loader = make_loader(xv, yv, batch_size=256)
    history = fit(model, loader, epochs=10, lr=0.001, val_loader=val_loader, verbose=False)
    return history[-1]["val_acc"]


def main() -> None:
    x, y = load_fashion("train", n=10_000)
    xv, yv = load_fashion("val", n=5_000)

    # the same permutation for every image, train and validation alike
    perm = torch.randperm(784, generator=torch.Generator().manual_seed(0))

    models = [
        ("MLP (256, 256)", lambda: MLP(hidden=(256, 256))),
        ("CNN (16, 32)", lambda: CNN()),
    ]
    for name, make_model in models:
        normal = val_accuracy(make_model, x, y, xv, yv)
        shuffled = val_accuracy(make_model, x[:, perm], y, xv[:, perm], yv)
        print(f"{name:<15} val acc normal {normal:.1%}   pixels shuffled {shuffled:.1%}")


if __name__ == "__main__":
    main()