A deep learning model achieves very high ROC-AUC but performs poorly in real-world deployment. Which is the MOST plausible explanation?

The correct answer is B. ROC-AUC is insensitive to class imbalance and decision thresholds. ROC-AUC measures a model's ability to rank positive instances above negative ones across all possible classification thresholds. However, it does not reflect how the model performs at the specific operating threshold used in production, nor does it adequately penalize poor perfor

NVIDIA Certified Associate (NCA) Core AI Concepts

Question

Options

AThe model has too many parameters
BROC-AUC is insensitive to class imbalance and decision thresholds
CThe optimizer converged too fast
DThe dataset is too large

How the community answered

(39 responses)

A
3% (1)
B
95% (37)
D
3% (1)

Explanation

ROC-AUC measures a model's ability to rank positive instances above negative ones across all possible classification thresholds. However, it does not reflect how the model performs at the specific operating threshold used in production, nor does it adequately penalize poor performance on minority classes in imbalanced datasets. A model can achieve a high AUC while still misclassifying most of the rare (but critical) positive cases at the deployment threshold. Metrics like Precision-Recall AUC, F1-score at the chosen threshold, or calibrated probabilities are often more informative for imbalanced real-world scenarios. Options A, C, and D (too many parameters, fast convergence, large dataset) do not directly explain why a high AUC would correlate with poor real-world performance.

Topics

#Model Evaluation#ROC-AUC#Class Imbalance#Decision Thresholds

Community Discussion

No community discussion yet for this question.

Full NCA-AIIO Practice