A machine learning engineer is in the process of implementing a concept drift monitoring solution. They are planning to use the following steps: 1. Deploy a model to production and compute predicted…

Question

Options

• AObtain the observed values (actual) feature values
• BMeasure the latency of the prediction time
• CRetrain the model
• DNone of these should be completed as Step #3
• ECompute the evaluation metric using the observed and predicted values

• A
  7% (3)
• B
  2% (1)
• C
  13% (6)
• D
  76% (34)
• E
  2% (1)

Explanation

Option D is correct because Step #3 in a proper concept drift monitoring pipeline should be computing the residuals (i.e., the individual differences between predicted and actual values). These residuals are the input signal that feeds the statistical test in Step #4 - tests like CUSUM, DDM, or ADWIN operate on individual error values over time, not on aggregate metrics.

Why the distractors fail:

• A is wrong because obtaining feature values is relevant to data drift (covariate shift) monitoring, not concept drift, which focuses on changes in P(Y|X)
• B is wrong because prediction latency is a system performance metric entirely unrelated to drift detection
• C is wrong because retraining is a response to detected drift - it comes after Step #4, not before it
• E is the trickiest distractor: computing an aggregate evaluation metric (like overall accuracy or RMSE) loses the individual-level granularity needed for statistical drift tests to detect when and how the error pattern changes over time

Memory tip: Think of the pipeline as "predict → observe → residualize → test." The statistical test needs a stream of individual errors, not a single summarized number - so any option describing aggregation or unrelated steps doesn't fit between observation and testing.

No community discussion yet for this question.