Backpropagation

What Backpropagation is

Backpropagation makes training deep networks tractable: one backward pass yields gradients for every parameter, no matter how many layers there are.

How it works

The forward pass caches intermediate activations. The backward pass propagates the derivative of the loss layer by layer, multiplying local derivatives along the way. Automatic differentiation in modern frameworks does this without hand-derived formulas.

Why it matters

Without it, deep learning would be computationally hopeless. It also explains vanishing and exploding gradients, which motivated residual connections and normalisation.

Common uses

• →Training all deep neural networks