CERTIFIED-MACHINE-LEARNING-PROFESSIONAL · Question #44
CERTIFIED-MACHINE-LEARNING-PROFESSIONAL Question #44: Real Exam Question with Answer & Explanation
The correct answer is D: None of these should be completed as Step #3. Option D is correct because Step #3 in a proper concept drift monitoring pipeline should be computing the residuals (i.e., the individual differences between predicted and actual values). These residuals are the input signal that feeds the statistical test in Step #4 - tests like
Question
A machine learning engineer is in the process of implementing a concept drift monitoring solution. They are planning to use the following steps: 1. Deploy a model to production and compute predicted values 2. Obtain the observed (actual) label values 3. _____ 4. Run a statistical test to determine if there are changes over time Which of the following should be completed as Step #3?
Options
- AObtain the observed values (actual) feature values
- BMeasure the latency of the prediction time
- CRetrain the model
- DNone of these should be completed as Step #3
- ECompute the evaluation metric using the observed and predicted values
Explanation
Option D is correct because Step #3 in a proper concept drift monitoring pipeline should be computing the residuals (i.e., the individual differences between predicted and actual values). These residuals are the input signal that feeds the statistical test in Step #4 - tests like CUSUM, DDM, or ADWIN operate on individual error values over time, not on aggregate metrics.
Why the distractors fail:
- A is wrong because obtaining feature values is relevant to data drift (covariate shift) monitoring, not concept drift, which focuses on changes in P(Y|X)
- B is wrong because prediction latency is a system performance metric entirely unrelated to drift detection
- C is wrong because retraining is a response to detected drift - it comes after Step #4, not before it
- E is the trickiest distractor: computing an aggregate evaluation metric (like overall accuracy or RMSE) loses the individual-level granularity needed for statistical drift tests to detect when and how the error pattern changes over time
Memory tip: Think of the pipeline as "predict → observe → residualize → test." The statistical test needs a stream of individual errors, not a single summarized number - so any option describing aggregation or unrelated steps doesn't fit between observation and testing.
Community Discussion
No community discussion yet for this question.