nerdexam
Amazon

MLS-C01 · Question #75

A Data Scientist is developing a machine learning model to classify whether a financial transaction is fraudulent. The labeled data available for training consists of 100,000 non- fraudulent…

The correct answer is B. Increase the XGBoost scale_pos_weight parameter to adjust the balance of positive and negative D. Change the XGBoost eval_metric parameter to optimize based on AUC instead of error. The model has high accuracy but too many false negatives due to a severely imbalanced dataset, requiring strategies to improve detection of the minority fraudulent class.

Modeling

Question

A Data Scientist is developing a machine learning model to classify whether a financial transaction is fraudulent. The labeled data available for training consists of 100,000 non- fraudulent observations and 1,000 fraudulent observations. The Data Scientist applies the XGBoost algorithm to the data, resulting in the following confusion matrix when the trained model is applied to a previously unseen validation dataset. The accuracy of the model is 99.1%, but the Data Scientist has been asked to reduce the number of false negatives. Which combination of steps should the Data Scientist take to reduce the number of false positive predictions by the model? (Choose two.)

Options

  • AChange the XGBoost eval_metric parameter to optimize based on rmse instead of error.
  • BIncrease the XGBoost scale_pos_weight parameter to adjust the balance of positive and negative
  • CIncrease the XGBoost max_depth parameter because the model is currently underfitting the data.
  • DChange the XGBoost eval_metric parameter to optimize based on AUC instead of error.
  • EDecrease the XGBoost max_depth parameter because the model is currently overfitting the data.

How the community answered

(33 responses)
  • A
    12% (4)
  • B
    45% (15)
  • C
    36% (12)
  • E
    6% (2)

Why each option

The model has high accuracy but too many false negatives due to a severely imbalanced dataset, requiring strategies to improve detection of the minority fraudulent class.

AChange the XGBoost eval_metric parameter to optimize based on rmse instead of error.

Optimizing based on rmse (Root Mean Squared Error) is typically used for regression problems, not for binary classification tasks like fraud detection.

BIncrease the XGBoost scale_pos_weight parameter to adjust the balance of positive and negativeCorrect

Increasing the scale_pos_weight parameter in XGBoost is a technique to handle imbalanced datasets by assigning a higher weight to the minority class (fraudulent transactions). This makes the model penalize false negatives more heavily, encouraging it to predict the positive class more readily and thereby reducing the number of missed fraudulent transactions.

CIncrease the XGBoost max_depth parameter because the model is currently underfitting the data.

Increasing the XGBoost max_depth parameter without addressing class imbalance is unlikely to solve the core issue of missing the minority class, and the problem description does not indicate underfitting as the primary problem.

DChange the XGBoost eval_metric parameter to optimize based on AUC instead of error.Correct

Changing the XGBoost eval_metric to optimize for AUC (Area Under the Receiver Operating Characteristic curve) instead of error is crucial for imbalanced classification problems. AUC evaluates the model's ability to distinguish between positive and negative classes across various thresholds, providing a more robust measure of performance than accuracy for imbalanced data and allowing for better trade-offs between false positives and false negatives.

EDecrease the XGBoost max_depth parameter because the model is currently overfitting the data.

Decreasing the XGBoost max_depth parameter would simplify the model, making it more prone to underfitting and likely increasing false negatives, which is the opposite of the stated goal to reduce them.

Concept tested: Handling imbalanced classification datasets in XGBoost

Source: https://xgboost.readthedocs.io/en/stable/parameter.html#learning-task-parameters

Topics

#XGBoost#Class Imbalance#Hyperparameter Tuning#Evaluation Metrics

Community Discussion

No community discussion yet for this question.

Full MLS-C01 Practice