nerdexam
Amazon

MLS-C01 · Question #10

A Machine Learning Specialist deployed a model that provides product recommendations on a company's website. Initially, the model was performing very well and resulted in customers buying more…

The correct answer is D. The model should be periodically retrained using the original training data plus new data as. A product recommendation model's performance has degraded over time without any changes, suggesting data or concept drift that needs to be addressed.

Machine Learning Implementation and Operations

Question

A Machine Learning Specialist deployed a model that provides product recommendations on a company's website. Initially, the model was performing very well and resulted in customers buying more products on average. However, within the past few months, the Specialist has noticed that the effect of product recommendations has diminished and customers are starting to return to their original habits of spending less. The Specialist is unsure of what happened, as the model has not changed from its initial deployment over a year ago. Which method should the Specialist try to improve model performance?

Options

  • AThe model needs to be completely re-engineered because it is unable to handle product inventory
  • BThe model's hyperparameters should be periodically updated to prevent drift.
  • CThe model should be periodically retrained from scratch using the original data while adding a
  • DThe model should be periodically retrained using the original training data plus new data as

How the community answered

(32 responses)
  • A
    3% (1)
  • B
    16% (5)
  • C
    6% (2)
  • D
    75% (24)

Why each option

A product recommendation model's performance has degraded over time without any changes, suggesting data or concept drift that needs to be addressed.

AThe model needs to be completely re-engineered because it is unable to handle product inventory

Re-engineering the entire model without first identifying the root cause (data/concept drift) is an overly drastic and likely unnecessary step.

BThe model's hyperparameters should be periodically updated to prevent drift.

Periodically updating only hyperparameters without retraining with new data will not address data or concept drift, as the model's learned patterns are still based on old distributions.

CThe model should be periodically retrained from scratch using the original data while adding a

Retraining with only the original data will not incorporate new patterns or changes in data distribution over time, making this approach ineffective for addressing performance degradation due to drift.

DThe model should be periodically retrained using the original training data plus new data asCorrect

Model performance degradation over time, without changes to the model itself, is often due to data drift or concept drift, meaning the underlying data distribution or the relationship between features and targets has changed. Periodically retraining the model using a combination of the original training data and new, recent data captures these changes, allowing the model to adapt and maintain relevance.

Concept tested: Model retraining to address concept/data drift

Source: https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor-concept-drift.html

Topics

#Model Drift#Data Drift#Model Retraining#MLOps

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full MLS-C01 Practice