What is regularisation, and what is the difference between L1 and L2?

Regularisation adds a penalty on model complexity to the loss function, discouraging the model from fitting noise. Instead of minimising error alone, it minimises error plus a penalty on the size of the coefficients.

• L1 (Lasso) adds the sum of absolute coefficient values. Its distinctive property is that it drives some coefficients to exactly zero, performing automatic feature selection. Useful when you suspect many features are irrelevant and want a sparse, interpretable model.
• L2 (Ridge) adds the sum of squared coefficients. It shrinks coefficients towards zero but never to zero. It handles correlated features gracefully by distributing weight among them, where L1 would arbitrarily pick one.
• Elastic Net combines both, giving sparsity while handling correlated groups sensibly.

The strength is controlled by a hyperparameter — often alpha or lambda, or inverted as C in scikit-learn — tuned by cross-validation. Too little and you overfit; too much and you underfit.

Note: Two practical points. Features must be scaled before applying L1 or L2, because the penalty depends on coefficient magnitude and unscaled features are penalised unequally. And regularisation takes other forms outside linear models: dropout and weight decay in neural networks, and max depth, min samples per leaf, and subsampling in tree ensembles all serve the same purpose.