PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #184
You received a training-serving skew alert from a Vertex AI Model Monitoring job running in production. You retrained the model with more recent training data, and deployed it back to the Vertex AI…
The correct answer is B. Update the model monitoring job to use the more recent training data that was used to retrain the. If a training-serving skew alert persists after model retraining with new data, the model monitoring job's baseline must be updated to reflect the new training data distribution.
Question
Options
- AUpdate the model monitoring job to use a lower sampling rate.
- BUpdate the model monitoring job to use the more recent training data that was used to retrain the
- CTemporarily disable the alert. Enable the alert again after a sufficient amount of new production
- DTemporarily disable the alert until the model can be retrained again on newer training data.
How the community answered
(42 responses)- A12% (5)
- B76% (32)
- C5% (2)
- D7% (3)
Why each option
If a training-serving skew alert persists after model retraining with new data, the model monitoring job's baseline must be updated to reflect the new training data distribution.
Using a lower sampling rate would reduce the amount of data analyzed, potentially masking actual skew or making detection less reliable, rather than resolving the root cause of the alert.
The model monitoring job compares live serving data against a baseline derived from training data; if the model was retrained with more recent data, the monitoring job's baseline must also be updated to this new dataset to accurately detect skew and avoid false positives from comparing against an outdated reference.
Temporarily disabling the alert only hides the problem without addressing the underlying misconfiguration of the monitoring job's baseline or the potential for actual skew.
Disabling the alert is a temporary workaround that does not fix the monitoring configuration, which is comparing serving data against an incorrect or outdated training data baseline.
Concept tested: Vertex AI Model Monitoring, training-serving skew, updating monitoring baselines
Source: https://cloud.google.com/vertex-ai/docs/model-monitoring/overview
Topics
Community Discussion
No community discussion yet for this question.