Machine Learning Interview Questions

Note: Lead with the business problem, not the model. Interviewers hear "I built an XGBoost classifier" constantly and "we were losing 8% of customers a quarter and could not tell which ones" rarely.

Structure it as:

• The problem and why ML was the right tool. Some problems are better solved with a rule or a report — being able to say why this one was not is a strong signal.
• The data. Where it came from, how much, and what was wrong with it. Realistically most of your time went here, so say so.
• How you framed it. Classification, regression, ranking, or something else, and — critically — what metric you optimised and why that metric matched the business cost.
• What you tried and what you shipped. Include the baseline. A simple model you beat is what makes the result meaningful.
• Deployment and outcome. Whether it reached production, how it was served, and what it actually changed.

If it never reached production, say why. That is a very common and instructive story.

Translate into decisions and costs, and never lead with the algorithm.

• Frame performance in business terms. Not "87% accuracy" but "of the 100 customers it flags each month, about 70 genuinely are at risk, and it catches roughly half of everyone who leaves". Precision and recall become meaningful when expressed as consequences.
• Be explicit about the two kinds of error and what each costs. A false positive on a fraud model means inconveniencing a real customer; a false negative means losing money. The threshold is a business decision, not a technical one, and framing it that way is what earns trust.
• Use feature importance or SHAP to explain drivers, but be careful to describe them as associations rather than causes — stakeholders will act on them as causes otherwise.
• State the limits plainly. What the model does not know, where it should not be relied on, and how it may degrade.

Note: Being willing to recommend a simpler, more interpretable model when the stakes require explanation — credit decisions, medical triage — shows maturity. Accuracy is not the only requirement.

Everyone has this story, and interviewers are checking whether you diagnose or flail.

Good scenarios to describe:

• Excellent validation performance, poor production performance. The usual cause is data leakage — a feature that would not be available at prediction time, or preprocessing fitted on the whole dataset before splitting. This is the most instructive version of the story.
• Performance decayed over time, from data drift or a change upstream in how a field was populated.
• High accuracy that was worthless because the classes were imbalanced and the model simply predicted the majority.
• The model worked but nobody used it, because it did not fit the existing workflow.

How to tell it: what you observed, how you isolated the cause, what you changed, and — most importantly — what you put in place so it would be caught earlier next time. Monitoring on input distributions and prediction distributions, a held-out temporal validation set, or a proper train-serve consistency check.

Note: Admitting that you initially blamed the model and the problem was in the data is a credible and common arc.

Interviewers value people who say no to ML, because unnecessary models are expensive and fragile.

ML is appropriate when:

• The pattern is genuinely complex and cannot be expressed as rules a human would write.
• You have enough labelled, relevant historical data — and the future will resemble it.
• Errors are tolerable and their cost is understood. A model that is wrong 10% of the time is fine for recommendations and unacceptable for some safety decisions.
• The decision is repeated often enough to justify building and maintaining a system.

ML is the wrong choice when:

• A rule would work. If a domain expert can write down the logic, write it down. It is cheaper, explainable, and does not drift.
• You lack labels, and acquiring them is impractical.
• The decision must be fully explainable for regulatory reasons and a simple model will not suffice.
• Nobody has committed to acting on the output.

Note: Always propose a baseline first — a heuristic, a simple rule, or a logistic regression. It sets the bar, it is often good enough, and if the sophisticated model cannot beat it, that is a finding rather than a failure.

How you keep current: papers for the areas you work in, practitioner blogs and library release notes for what is actually usable, and implementing something rather than only reading it. Reproducing a result teaches you the assumptions the paper glosses over.

How you filter — and this is the part that matters:

• Fundamentals over trends. Understanding bias-variance, validation methodology, and how to interrogate data has not changed in decades and determines whether your work is correct. Most production ML failures are methodology failures, not architecture failures.
• Does it fit a problem you have? A new architecture that needs a hundred times the data you possess is interesting, not useful.
• Is it reproducible and maintainable? A published state-of-the-art result with no released code and enormous compute requirements is not a candidate for production.

Note: A strong, honest position: most business ML value still comes from good features, clean data, and gradient-boosted trees on tabular problems — not from the latest architecture. Being able to say that, while still following the field, shows judgement rather than fashion-following.

• Supervised learning — you have labelled examples, and the model learns to map inputs to known outputs. Split into classification (discrete labels: spam or not, which of five categories) and regression (continuous values: price, temperature, demand). This covers the large majority of applied ML because business problems usually come with historical outcomes.
• Unsupervised learning — no labels; the model finds structure in the data itself. Clustering (k-means, DBSCAN, hierarchical) groups similar records; dimensionality reduction (PCA, t-SNE, UMAP) compresses features while preserving structure; association rules find items that co-occur. Used for customer segmentation, anomaly detection, and exploration.
• Reinforcement learning — an agent takes actions in an environment and learns from rewards. There are no correct answers given, only feedback on outcomes, and the agent must balance exploring new actions against exploiting what it knows. Used in robotics, game playing, recommendation sequencing, and RLHF for language models.

Note: Semi-supervised and self-supervised learning are worth mentioning. Self-supervised in particular — creating labels from the data itself, such as predicting a masked word — is what made modern language models possible, since it removes the labelling bottleneck entirely.

Overfitting means the model has learned noise as well as signal. Training error is low, validation error is high, and it generalises badly. Underfitting means the model is too simple to capture the pattern — both training and validation error are high.

This is the bias-variance trade-off. Underfitting is high bias; overfitting is high variance.

How to detect it: compare training and validation performance. A large gap means overfitting; both poor means underfitting. Learning curves plotted against training set size distinguish them clearly — if validation error is still falling as you add data, more data will help; if the curves have converged and both are poor, the model is too simple.

Fixing overfitting:

• More training data — the most reliable fix when available.
• Regularisation — L1 (Lasso, which drives coefficients to zero and performs feature selection) or L2 (Ridge, which shrinks them).
• Simplify the model — fewer parameters, shallower trees, fewer features.
• Early stopping on a validation set.
• Dropout in neural networks; pruning and depth limits in trees.
• Cross-validation for reliable estimates.
• Ensembling, which averages away variance.

Fixing underfitting: a more expressive model, better features, less regularisation, or longer training.

Note: The most dangerous case is not overfitting to training data but overfitting to the validation set through repeated tuning. That is why a genuinely held-out test set touched once matters.

If 99% of transactions are legitimate, a model predicting "legitimate" every time achieves 99% accuracy and catches zero fraud. Accuracy measures the majority class and hides total failure on the class you care about.

Better metrics, from the confusion matrix:

• Precision = TP / (TP + FP). Of everything flagged, how much was right. Matters when false positives are costly — flagging a legitimate customer's card.
• Recall (sensitivity) = TP / (TP + FN). Of everything that was positive, how much did you catch. Matters when misses are costly — a missed disease diagnosis.
• F1 — the harmonic mean of the two, when you need a single balanced number.
• ROC-AUC — ranking quality across all thresholds, but it is optimistic on heavily imbalanced data because the true negative count dominates.
• PR-AUC — precision-recall AUC, which is the better summary metric for rare positives.

Handling the imbalance itself: class weights in the loss function (usually the first thing to try), resampling with SMOTE or undersampling, threshold tuning on the predicted probability, or collecting more minority examples.

Note: The most important point is that the threshold is a business decision. The model outputs a probability; where you cut it depends on the relative cost of a false positive versus a false negative. Presenting that as a curve for stakeholders to choose from is better than picking 0.5 by default.

The three splits serve three distinct purposes:

• Training set — the model learns parameters from it.
• Validation set — used to tune hyperparameters and choose between models.
• Test set — touched once, at the very end, to estimate real-world performance.

The reason for three rather than two is subtle and important: every time you look at the validation set and adjust something, you leak a little information into your choices. After fifty experiments, validation performance is optimistic. The test set stays clean because you never optimise against it.

K-fold cross-validation splits the training data into k parts, trains k times each holding out a different fold, and averages the results. It gives a more reliable estimate than a single split — especially on small datasets — and it uses all the data for both training and validation across the folds. Stratified k-fold preserves class proportions in each fold and should be the default for classification.

Where standard cross-validation is wrong:

• Time series. Random splitting lets the model train on the future and predict the past. Use forward-chaining, where each fold trains on everything before a cutoff and validates after it.
• Grouped data. If one patient or customer has multiple rows, they must not appear in both train and validation, or you are leaking. Use group-aware splitting.

Both are ensemble methods combining weak learners, but they build them differently.

Bagging (Bootstrap Aggregating) trains many models independently and in parallel on bootstrap samples of the data, then averages or votes. Because the models are independent, averaging reduces variance without increasing bias.

Random Forest is bagging with decision trees plus one addition: at each split, only a random subset of features is considered. This decorrelates the trees, which is what makes the averaging effective. It is robust, needs little tuning, handles mixed data types, and rarely overfits badly.

Boosting trains models sequentially, each one focused on the errors of those before it. It reduces bias, building a strong learner from weak ones.

Gradient boosting fits each new tree to the residuals — more precisely, the negative gradient of the loss — of the current ensemble. XGBoost, LightGBM, and CatBoost are optimised implementations, and they remain the strongest general-purpose approach for tabular data.

The trade-offs:

• Random Forest is harder to overfit, trains in parallel, and needs less tuning.
• Gradient boosting usually achieves better accuracy but is sensitive to hyperparameters — particularly learning rate and tree depth — and will overfit if pushed too far. Use early stopping on a validation set.

Note: For most tabular business problems, a well-tuned gradient boosting model beats a neural network. Saying so demonstrates practical rather than academic judgement.

Feature engineering is creating the input variables a model learns from. It routinely matters more than the choice of algorithm — a good feature set with a simple model usually beats a poor feature set with a sophisticated one.

Common techniques:

• Categorical encoding — one-hot for low cardinality, target or ordinal encoding for high cardinality. Target encoding must be computed within cross-validation folds or it leaks.
• Scaling — standardisation or normalisation, essential for distance-based and gradient-descent models, irrelevant for tree-based ones.
• Date and time decomposition — day of week, month, hour, is-holiday, days-since-last-event. Raw timestamps are almost useless; their components are highly predictive.
• Aggregations — count, mean, and recency of a customer's past behaviour. These are usually the most powerful features in business problems.
• Binning and interactions — grouping continuous values, or products and ratios of existing features.
• Text and domain-specific transforms — TF-IDF, embeddings, or ratios that a domain expert would recognise.

Handling missing values is part of this: understand why a value is missing before imputing. Missingness is often itself informative, and a binary "was missing" flag frequently helps.

Note: The critical discipline is that every feature must be computable at prediction time with data available then. A feature built from information that only exists after the outcome is leakage, and it is the most common reason a model performs brilliantly in testing and fails in production.

Regularisation adds a penalty on model complexity to the loss function, discouraging the model from fitting noise. Instead of minimising error alone, it minimises error plus a penalty on the size of the coefficients.

• L1 (Lasso) adds the sum of absolute coefficient values. Its distinctive property is that it drives some coefficients to exactly zero, performing automatic feature selection. Useful when you suspect many features are irrelevant and want a sparse, interpretable model.
• L2 (Ridge) adds the sum of squared coefficients. It shrinks coefficients towards zero but never to zero. It handles correlated features gracefully by distributing weight among them, where L1 would arbitrarily pick one.
• Elastic Net combines both, giving sparsity while handling correlated groups sensibly.

The strength is controlled by a hyperparameter — often alpha or lambda, or inverted as C in scikit-learn — tuned by cross-validation. Too little and you overfit; too much and you underfit.

Note: Two practical points. Features must be scaled before applying L1 or L2, because the penalty depends on coefficient magnitude and unscaled features are penalised unequally. And regularisation takes other forms outside linear models: dropout and weight decay in neural networks, and max depth, min samples per leaf, and subsampling in tree ensembles all serve the same purpose.

Data leakage is when information unavailable at prediction time influences training. It produces excellent validation scores and a model that fails in production, and it is the single most common serious mistake in applied machine learning.

The main forms:

• Target leakage — a feature that is a consequence of the outcome. Including "number of collection calls" when predicting default, or "discharge medication" when predicting diagnosis. The giveaway is suspiciously high performance and one feature dominating importance.
• Train-test contamination — fitting a scaler, imputer, or encoder on the full dataset before splitting. The transformer has then seen the test data's distribution.
• Temporal leakage — training on data from after the prediction point. Random splits on time series always do this.
• Group leakage — the same entity appearing in both train and test, so the model memorises rather than generalises.
• Duplicate rows spread across splits.

How to prevent it:

• Split first, then fit everything on the training split only. Use a scikit-learn Pipeline so preprocessing is fitted inside each cross-validation fold automatically — this eliminates the most common form structurally.
• Split by time for temporal problems, and by group where entities repeat.
• Interrogate every feature: would this value exist, with this content, at the moment of prediction?
• Treat implausibly good results as a bug report, not a success.

Serving patterns:

• Batch prediction — scoring on a schedule and writing results to a table. Simplest and sufficient for churn scores, lead ranking, and most business use cases.
• Real-time API — a service exposing an endpoint. Necessary when the prediction depends on the current request, such as fraud scoring at checkout.
• Embedded — the model runs on device, for latency or privacy reasons.

What must be in place:

• Reproducibility — versioned model artefacts, training data, and code, so any prediction can be traced to the model that made it.
• Train-serve consistency. The same preprocessing must run in both places. Shipping the whole pipeline as one artefact, or using a feature store, prevents a whole class of silent errors.
• A rollback path, and staged rollout — shadow mode or a canary before full traffic.

Monitoring, at three levels:

• Operational — latency, error rate, throughput.
• Data — input distributions compared against training. Data drift means the inputs have changed; catching it is possible immediately.
• Model — prediction distribution, and accuracy once labels arrive. Concept drift means the relationship between inputs and target has changed.

Note: The key practical difficulty is that ground truth often arrives late or never, so you must monitor input and prediction distributions as leading indicators rather than waiting for accuracy to confirm a problem.

Prediction error decomposes into three parts:

• Bias — error from wrong assumptions. A model too simple to represent the true relationship has high bias and underfits. Fitting a straight line to a curve is the canonical example.
• Variance — error from sensitivity to the particular training sample. A model too flexible fits noise, so a different sample would produce a very different model. It overfits.
• Irreducible error — noise inherent in the problem. No model removes this, and recognising it prevents chasing performance that is not achievable.

The trade-off: increasing model complexity reduces bias and raises variance. Total error falls, reaches a minimum, then rises again. The goal is that minimum, not maximum flexibility.

How it guides decisions:

• High bias diagnosed (poor on both train and validation) — use a more expressive model, add features, reduce regularisation. More data will not help.
• High variance diagnosed (good on train, poor on validation) — more data, stronger regularisation, a simpler model, or ensembling. More data will help.
• Ensembles attack each side: bagging and Random Forest reduce variance; boosting reduces bias.

Note: Worth acknowledging that very large neural networks exhibit "double descent" — error rises then falls again as capacity grows well past the interpolation point, which complicates the classical picture. Mentioning it shows you know the trade-off is a useful model rather than a complete law.