PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #133
You recently built the first version of an image segmentation model for a self-driving car. After deploying the model, you observe a decrease in the area under the curve (AUC) metric. When analyzing…
The correct answer is A. The model is overfitting in areas with less traffic and underfitting in areas with more traffic. The model's observed decrease in AUC and failure in highly congested traffic while performing well in less traffic indicates it has overfit to simpler, common scenarios and underfit to the more complex or underrepresented conditions.
Question
Options
- AThe model is overfitting in areas with less traffic and underfitting in areas with more traffic.
- BAUC is not the correct metric to evaluate this classification model.
- CToo much data representing congested areas was used for model training.
- DGradients become small and vanish while backpropagating from the output to input nodes.
How the community answered
(42 responses)- A83% (35)
- B10% (4)
- C2% (1)
- D5% (2)
Why each option
The model's observed decrease in AUC and failure in highly congested traffic while performing well in less traffic indicates it has overfit to simpler, common scenarios and underfit to the more complex or underrepresented conditions.
If the image segmentation model performs well in 'less traffic' scenarios but fails in 'highly congested traffic,' it suggests the model has learned the simpler patterns effectively (overfitting to them) but failed to generalize to the more complex, potentially underrepresented, patterns of congested environments (underfitting to complexity). This differential performance explains the overall decreased AUC metric after deployment.
AUC (Area Under the Curve) is a widely accepted and appropriate metric for evaluating the performance of classification models, including image segmentation where each pixel is classified.
If too much data representing congested areas was used for training, the model should ideally perform *better* or at least not fail in those areas, which contradicts the observation that it specifically fails in highly congested traffic.
Vanishing gradients primarily refer to a problem during model *training* where gradients become too small to update network weights effectively, hindering learning, and does not directly explain the specific *deployment behavior* of differential performance across traffic conditions.
Concept tested: Overfitting and underfitting scenarios
Source: https://developers.google.com/machine-learning/crash-course/generalization/peril-of-overfitting
Topics
Community Discussion
No community discussion yet for this question.