In the last step the gap shrank and the validation number did not. Careful: that is one table, one seed. Does "did not rise" mean "does not help", or just "the improvement is smaller than the noise"? Let us check — and compare with another way to improve the result.

1. Regularisation against noise

Two settings, three seeds each (30 epochs, no early stopping, the same 256-256 network, 5 000 validation images):

baseline                 train 95.8% 97.3% 97.0%  val 85.8% 87.3% 86.9%  mean val 86.7%
dropout 0.3 + wd 1e-4    train 93.1% 92.7% 94.0%  val 86.7% 86.2% 87.7%  mean val 86.9%

Training accuracy is systematically lower (~96–97 % → ~93 %): regularisation does what it promised. But the validation mean is 86.7 vs 86.9 %: a difference of 0.2 points, while the seeds themselves differ by 1.5 (85.8 … 87.3). And in each row a different seed is the best.

I ran the same experiment earlier (a different random-number sequence): 86.2 % → 86.5 %. The direction is the same, and so is the size: too small to tell from noise. With three seeds this is neither proof nor refutation.

The honest conclusion: for this network and this amount of data, dropout + weight decay do not raise validation accuracy in a way you would notice. They narrow the gap — that is true — but it is not the same thing.

2. More data

The same network (256-256), no regularisation, different numbers of training images, the same 5 000 validation images:

training images	epochs	training accuracy	validation	gap
1 000	20	95.1 %	80.8 %	+14.3
2 000	20	95.7 %	83.0 %	+12.6
5 000	20	93.6 %	84.3 %	+9.3
10 000	20	94.7 %	86.9 %	+7.8
50 000	8	91.5 %	88.8 %	+2.8

(The last row is only 8 epochs, because 50 000 images take ~5× longer; do not compare it apples-to-apples.)

Here validation accuracy climbs — from 80.8 to 88.8 %. The gap also shrinks, but now not because we pushed the training number down, but because the model has something to learn from.

What to take away

• More data raises the real number. Regularisation brings the training number closer to the real one. The first is a real gain, the second is better diagnostics.
• When you cannot get more data (usually your case), regularisation is still worth it: it makes your training number more honest, and early stopping saves time (12 epochs instead of 30 in the table of the previous step).