PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #152
You work for a magazine publisher and have been tasked with predicting whether customers will cancel their annual subscription. In your exploratory data analysis, you find that 90% of individuals rene
The correct answer is C. This is a good result because predicting those who cancel their subscription is more difficult,. In an imbalanced dataset where cancelations are rare, achieving 99% accuracy for the minority class (cancelers) is a very strong result, especially since this class is typically harder to predict accurately.
Question
Options
- AThis is not a good result because the model should have a higher accuracy for those who renew
- BThis is not a good result because the model is performing worse than predicting that people will
- CThis is a good result because predicting those who cancel their subscription is more difficult,
- DThis is a good result because the accuracy across both groups is greater than 80%.
How the community answered
(38 responses)- A24% (9)
- B11% (4)
- C61% (23)
- D5% (2)
Why each option
In an imbalanced dataset where cancelations are rare, achieving 99% accuracy for the minority class (cancelers) is a very strong result, especially since this class is typically harder to predict accurately.
In an imbalanced dataset, simply having higher accuracy for the majority class (renewers) is easier to achieve and less indicative of a good model than high accuracy for the minority class.
Predicting everyone renews would give 90% accuracy but would completely miss all cancelations; a model that correctly identifies 99% of the difficult-to-predict cancelers is performing significantly better than a naive majority-class predictor.
Predicting the minority class (customers who cancel, only 10% of the data) with 99% accuracy is generally considered a very good result, as this class is often much harder to identify correctly compared to the majority class. The overall utility of the model is often focused on identifying these rare, critical events, making high precision/recall for the minority class highly valuable.
While 80%+ accuracy across both groups seems good at first glance, this statement oversimplifies the problem; in imbalanced classification, evaluating minority class performance is critical, and 99% on the harder class is a strong indicator of value.
Concept tested: Model evaluation with imbalanced datasets
Source: https://developers.google.com/machine-learning/crash-course/classification/precision-and-recall
Topics
Community Discussion
No community discussion yet for this question.