Regularization

What Regularization is

Regularisation adds pressure towards simpler solutions — smaller weights, fewer active units, smoother functions.

How it works

L1 and L2 penalties shrink weights, dropout randomly disables units during training, weight decay is folded into the optimiser, data augmentation enlarges the effective dataset, and early stopping halts before memorisation sets in.

Why it matters

It is the standard toolkit for closing the gap between training and validation performance without collecting more data.

Common uses

• →Deep network training
• →Linear models with many correlated features
• →Fine-tuning small datasets