You deployed an ML model into production a year ago. Every month, you collect all raw requests that were sent to your model prediction service during the previous month. You send a subset of these req

Question

Options

• ATrain an anomaly detection model on the training dataset, and run all incoming requests through
• BIdentify temporal patterns in your model's performance over the previous year. Based on these
• CCompare the cost of the labeling service with the lost revenue due to model performance
• DRun training-serving skew detection batch jobs every few days to compare the aggregate

• A
  15% (5)
• B
  6% (2)
• C
  9% (3)
• D
  70% (23)

Explanation

Training-serving skew detection compares the statistical distribution of incoming production requests against the training data distribution. When these distributions diverge significantly, it is a strong proxy signal that model performance is degrading - without requiring expensive human labeling for every batch. Running this every few days provides timely detection while minimizing labeling costs. Option A (anomaly detection on all requests) doesn't directly indicate when to retrain. Option B (temporal patterns) is less precise and actionable. Option C (cost vs. revenue comparison) is a business analysis, not a monitoring strategy that triggers retraining.

No community discussion yet for this question.