Nothing to write here. Here you watch one neuron and then ten — with real printed values.

First a small example you can check with a pencil

Three inputs, three weights, one bias:

x = [1.0, 2.0, 3.0]
w = [0.5, -1.0, 2.0]
b = 0.25

score  z = 1·0.5 + 2·(-1.0) + 3·2.0 + 0.25
         = 0.5 - 2.0 + 6.0 + 0.25
         = 4.75

That is the whole neuron: multiply each input by its weight, add them up, add the bias. The bias is a number that lets the neuron "lean" one way even when every input is zero.

The score is not yet an "answer". One activation (an activation function) turns it into something useful. The simplest is the sigmoid, which squashes any number into (0, 1):

sigmoid(-4.0) = 0.0180      a very sure "no"
sigmoid( 0.0) = 0.5000      undecided
sigmoid( 4.0) = 0.9820      a very sure "yes"
sigmoid(4.75) = 0.9914

Now a real neuron, a real picture

The weights are torch.randn(784, 10) * 0.01, i.e. random numbers whose typical size is about 0.01. The bias is zeros. For a single neuron we take the first column:

x[0]          shape [784]       the first picture, flattened
W[:, 0]       shape [784]       the first neuron's weights

one neuron, one image: score=-0.2554 sigmoid=0.4365

One neuron: score −0.2554, through the sigmoid — 0.4365. Nothing useful has been said yet; the weights are random.

Now all ten at once — x[0] @ W + b:

class         score z
T-shirt      -0.2554
Trouser       0.0748
Pullover      0.2654
Dress         0.0845
Coat         -0.0591
Sandal        0.0135
Shirt         0.0098
Sneaker      -0.0974
Bag           0.4544
Ankle boot    0.4461

Softmax: from scores to probabilities

Scores can be negative and add up to nothing meaningful. We need ten probabilities that add up to 1. That is what softmax does: raise e to the power of each and divide by the sum. A small illustration with three scores:

scores          [2.0,   1.0,   0.1]
exp(scores)     [7.389, 2.718, 1.105]       sum = 11.2125
divide by sum   [0.6590, 0.2424, 0.0986]    sum = 1.0

The first picture with all ten:

probs shape: torch.Size([10000, 10]) row sums to 1.0
Bag         0.1400
Ankle boot  0.1388
Pullover    0.1159
(the other seven — from 0.0688 to 0.0967)
first image: predicted Bag, truth Ankle boot

The untrained model says "bag" with 14 % — and "ankle boot" (the correct answer) trails by only 0.0012. It is nearly an even guess among ten.

All 10,000 pictures

untrained accuracy: 10.2%   (chance = 10.0%)

10.2 % — almost exactly what a random guess among ten would give. But look at what the model actually says:

how many times each class was predicted:
T-shirt 0   Trouser 14   Pullover 84   Dress 79   Coat 0
Sandal 15   Shirt 241    Sneaker 2     Bag 4509   Ankle boot 5056

It spreads almost all its answers — 9,565 of 10,000 — between two classes: bag and ankle boot. The other eight are almost never chosen.

Three observations

1. A neuron is a single line: x @ w + b. Ten neurons — the same line with [784, 10]. There is no other magic, and the entire "intelligence" part is which numbers get written into w.

2. Softmax decides nothing — it only normalises. The biggest score stays the biggest; softmax only turns "numbers" into "probabilities that add up to 1".

3. Random weights are not "neutral". They do not give every class an equal chance; they simply pick something. Here — bag and boot. The accuracy stays at ~10 % not because the model is "equally indifferent", but because its favourite answers only happen to coincide with the truth now and then.

That is exactly what question 2 in "Guess before reading on" was getting at: some class will be a favourite.

What next

In the next step you write neuron and softmax as Python — and one line is missing from each.