Explain the bias-variance trade-off and how it guides model selection.

Prediction error decomposes into three parts:

• Bias — error from wrong assumptions. A model too simple to represent the true relationship has high bias and underfits. Fitting a straight line to a curve is the canonical example.
• Variance — error from sensitivity to the particular training sample. A model too flexible fits noise, so a different sample would produce a very different model. It overfits.
• Irreducible error — noise inherent in the problem. No model removes this, and recognising it prevents chasing performance that is not achievable.

The trade-off: increasing model complexity reduces bias and raises variance. Total error falls, reaches a minimum, then rises again. The goal is that minimum, not maximum flexibility.

How it guides decisions:

• High bias diagnosed (poor on both train and validation) — use a more expressive model, add features, reduce regularisation. More data will not help.
• High variance diagnosed (good on train, poor on validation) — more data, stronger regularisation, a simpler model, or ensembling. More data will help.
• Ensembles attack each side: bagging and Random Forest reduce variance; boosting reduces bias.

Note: Worth acknowledging that very large neural networks exhibit "double descent" — error rises then falls again as capacity grows well past the interpolation point, which complicates the classical picture. Mentioning it shows you know the trade-off is a useful model rather than a complete law.