nerdexam
Amazon

MLA-C01 · Question #157

An ML model is deployed in production. The model has performed well and has met its metric thresholds for months. An ML engineer who is monitoring the model observes a sudden degradation. The…

The correct answer is B. Drift in production data distribution. When a model performs well for months and then suddenly degrades, the most common root cause is data drift-the statistical distribution of incoming production data has shifted away from the distribution the model was trained on. Real-world data changes over time (seasonal…

ML Solution Monitoring, Maintenance, and Security

Question

An ML model is deployed in production. The model has performed well and has met its metric thresholds for months. An ML engineer who is monitoring the model observes a sudden degradation. The performance metrics of the model are now below the thresholds. What could be the cause of the performance degradation?

Options

  • ALack of training data
  • BDrift in production data distribution
  • CCompute resource constraints
  • DModel overfitting

How the community answered

(21 responses)
  • A
    5% (1)
  • B
    90% (19)
  • D
    5% (1)

Explanation

When a model performs well for months and then suddenly degrades, the most common root cause is data drift-the statistical distribution of incoming production data has shifted away from the distribution the model was trained on. Real-world data changes over time (seasonal patterns, economic shifts, new customer behaviors), and a model trained on old data becomes less accurate. Option A (lack of training data) would have been a problem at model creation time, not months later. Option C (compute resource constraints) would cause latency or timeouts, not a drop in prediction accuracy below metric thresholds. Option D (model overfitting) is a training-time problem that would be detected during evaluation before deployment, not appear suddenly after months of good production performance.

Topics

#Model monitoring#Data drift#Production model issues#Performance degradation

Community Discussion

No community discussion yet for this question.

Full MLA-C01 Practice