Guess · miss · gradients · nudge
What backpropagation actually does
A neural network is a pile of dials. Learning means turning them. Backpropagation is how the network works out which way to turn each one. Below, a real network learns to tell ripe fruit from unripe, one guess at a time.
Why the backwards trip matters
You could estimate a gradient by trial and error: wiggle one dial, measure how the miss changes, then repeat for every dial. Even this tiny network has 21 trainable dials: 16 connection weights and five biases. With the billions of parameters in a modern model, testing them one at a time would be impractical.
Backpropagation computes the gradient for every dial in one backwards sweep. Starting with the miss at the output, the chain rule combines each layer’s local sensitivity with the gradients arriving from the layer after it. By the end, every weight and bias has a number describing how a small change to it would change the miss.
Gradient descent then nudges each dial in the direction that lowers the miss, and training repeats with more examples. Backpropagation makes those gradients efficient to compute; the data, objective, architecture and optimiser determine what the network ultimately learns.