MLA-C01 · Question #157
An ML model is deployed in production. The model has performed well and has met its metric thresholds for months. An ML engineer who is monitoring the model observes a sudden degradation. The…
The correct answer is B. Drift in production data distribution. When a model performs well for months and then suddenly degrades, the most common root cause is data drift-the statistical distribution of incoming production data has shifted away from the distribution the model was trained on. Real-world data changes over time (seasonal…
Question
An ML model is deployed in production. The model has performed well and has met its metric thresholds for months. An ML engineer who is monitoring the model observes a sudden degradation. The performance metrics of the model are now below the thresholds. What could be the cause of the performance degradation?
Options
- ALack of training data
- BDrift in production data distribution
- CCompute resource constraints
- DModel overfitting
How the community answered
(21 responses)- A5% (1)
- B90% (19)
- D5% (1)
Explanation
When a model performs well for months and then suddenly degrades, the most common root cause is data drift-the statistical distribution of incoming production data has shifted away from the distribution the model was trained on. Real-world data changes over time (seasonal patterns, economic shifts, new customer behaviors), and a model trained on old data becomes less accurate. Option A (lack of training data) would have been a problem at model creation time, not months later. Option C (compute resource constraints) would cause latency or timeouts, not a drop in prediction accuracy below metric thresholds. Option D (model overfitting) is a training-time problem that would be detected during evaluation before deployment, not appear suddenly after months of good production performance.
Topics
Community Discussion
No community discussion yet for this question.