nerdexam
Amazon

MLS-C01 · Question #380

A data scientist is building a new model for an ecommerce company. The model will predict how many minutes it will take to deliver a package. During model training, the data scientist needs to evaluat

The correct answer is B. Mean squared error (MSE) C. Root mean squared error (RMSE). For a model predicting package delivery time, which is a regression problem, Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) are the appropriate performance evaluation metrics.

Modeling

Question

A data scientist is building a new model for an ecommerce company. The model will predict how many minutes it will take to deliver a package. During model training, the data scientist needs to evaluate model performance. Which metrics should the data scientist use to meet this requirement? (Choose two.)

Options

  • AInferenceLatency
  • BMean squared error (MSE)
  • CRoot mean squared error (RMSE)
  • DPrecision
  • EAccuracy

How the community answered

(21 responses)
  • A
    5% (1)
  • B
    90% (19)
  • D
    5% (1)

Why each option

For a model predicting package delivery time, which is a regression problem, Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) are the appropriate performance evaluation metrics.

AInferenceLatency

InferenceLatency measures the time taken for a model to make a prediction, which is an operational metric, not a measure of the model's predictive performance or accuracy.

BMean squared error (MSE)Correct

Mean Squared Error (MSE) is a standard regression metric that measures the average of the squared differences between predicted and actual values. It quantifies the average magnitude of the errors and is commonly used to evaluate the accuracy of regression models.

CRoot mean squared error (RMSE)Correct

Root Mean Squared Error (RMSE) is the square root of MSE, providing a metric in the same units as the target variable (minutes, in this case). RMSE is widely used in regression tasks for its interpretability and represents the typical magnitude of prediction errors.

DPrecision

Precision is a classification metric that measures the proportion of true positive predictions among all positive predictions, making it unsuitable for regression problems where continuous values are predicted.

EAccuracy

Accuracy is a classification metric that measures the overall proportion of correct predictions, which is not applicable to regression models that predict continuous numerical outcomes.

Concept tested: Regression model evaluation metrics

Source: https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-model-metrics.html

Topics

#Model evaluation#Regression metrics#Mean squared error (MSE)#Root mean squared error (RMSE)

Community Discussion

No community discussion yet for this question.

Full MLS-C01 Practice