nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #336

You are building an ML model to predict customer churn for a subscription service. You have trained your model on Vertex AI using historical data, and deployed it to a Vertex AI endpoint for…

The correct answer is A. Monitor the training/serving skew of feature values for requests sent to the endpoint. A significant drop in model performance in production often indicates training-serving skew - differences between the feature distributions during training and those seen at serving time. Monitoring this skew helps identify data drift or inconsistencies causing the performance

Submitted by haru.x· Apr 18, 2026Monitoring, optimizing, and maintaining ML solutions

Question

You are building an ML model to predict customer churn for a subscription service. You have trained your model on Vertex AI using historical data, and deployed it to a Vertex AI endpoint for real-time predictions. After a few weeks, you notice that the model's performance, measured by AUC (area under the ROC curve), has dropped significantly in production compared to its performance during training. How should you troubleshoot this problem?

Options

  • AMonitor the training/serving skew of feature values for requests sent to the endpoint.
  • BMonitor the resource utilization of the endpoint, such as CPU and memory usage, to identify
  • CEnable Vertex Explainable AI feature attribution to analyze model predictions and understand the
  • DMonitor the latency of the endpoint to determine whether predictions are being served within the

How the community answered

(29 responses)
  • A
    72% (21)
  • B
    3% (1)
  • C
    7% (2)
  • D
    17% (5)

Explanation

A significant drop in model performance in production often indicates training-serving skew - differences between the feature distributions during training and those seen at serving time. Monitoring this skew helps identify data drift or inconsistencies causing the performance

Topics

#Training-Serving Skew#Model Monitoring#Data Drift#MLOps Troubleshooting

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice