nerdexam
Amazon

MLA-C01 · Question #87

A company deployed an ML model that uses the XGBoost algorithm to predict product failures. The model is hosted on an Amazon SageMaker endpoint and is trained on normal operating data. An AWS Lambda f

The correct answer is C. Schedule a monitoring job in SageMaker Model Monitor. Use the job to detect drift by analyzing. SageMaker Model Monitor is purpose-built to detect model drift by continuously analyzing inference data against a baseline (the training data distribution) - exactly what this scenario requires. Option C is correct because scheduling a Model Monitor job automates the drift detect

ML Solution Monitoring, Maintenance, and Security

Question

A company deployed an ML model that uses the XGBoost algorithm to predict product failures. The model is hosted on an Amazon SageMaker endpoint and is trained on normal operating data. An AWS Lambda function provides the predictions to the company's application. An ML engineer must implement a solution that uses incoming live data to detect decreased model accuracy over time. Which solution will meet these requirements?

Options

  • AUse Amazon CloudWatch to create a dashboard that monitors real-time inference data and model
  • BModify the Lambda function to calculate model drift by using real-time inference data and model
  • CSchedule a monitoring job in SageMaker Model Monitor. Use the job to detect drift by analyzing
  • DSchedule a monitoring job in SageMaker Debugger. Use the job to detect drift by analyzing the

How the community answered

(59 responses)
  • A
    3% (2)
  • B
    14% (8)
  • C
    75% (44)
  • D
    8% (5)

Explanation

SageMaker Model Monitor is purpose-built to detect model drift by continuously analyzing inference data against a baseline (the training data distribution) - exactly what this scenario requires. Option C is correct because scheduling a Model Monitor job automates the drift detection process using statistical comparisons between live inference data and the original training baseline, with no custom code needed.

Why the distractors fail:

  • A (CloudWatch dashboard): CloudWatch monitors infrastructure metrics (latency, invocations, errors), not model accuracy or data drift - it has no concept of ML feature distributions.
  • B (Lambda function with custom drift logic): While technically possible, this is an anti-pattern - it reinvents what Model Monitor already does natively, adds maintenance burden, and isn't a managed ML solution.
  • D (SageMaker Debugger): Debugger is designed for training-time debugging (vanishing gradients, overfit detection), not for monitoring deployed endpoint drift on live inference data.

Memory tip: Think of it this way - Debugger = debug training, Model Monitor = monitor production. The word "monitor" in the requirement ("detect decreased model accuracy over time") maps directly to the service name Model Monitor.

Topics

#SageMaker Model Monitor#Model Drift Detection#ML Monitoring#Real-time Inference

Community Discussion

No community discussion yet for this question.

Full MLA-C01 Practice