Describe a time a model did not work as expected. What did you do?

Everyone has this story, and interviewers are checking whether you diagnose or flail.

Good scenarios to describe:

• Excellent validation performance, poor production performance. The usual cause is data leakage — a feature that would not be available at prediction time, or preprocessing fitted on the whole dataset before splitting. This is the most instructive version of the story.
• Performance decayed over time, from data drift or a change upstream in how a field was populated.
• High accuracy that was worthless because the classes were imbalanced and the model simply predicted the majority.
• The model worked but nobody used it, because it did not fit the existing workflow.

How to tell it: what you observed, how you isolated the cause, what you changed, and — most importantly — what you put in place so it would be caught earlier next time. Monitoring on input distributions and prediction distributions, a held-out temporal validation set, or a proper train-serve consistency check.

Note: Admitting that you initially blamed the model and the problem was in the data is a credible and common arc.