Which statistic calculated from a validation sample, can help decide which model to use for prediction of a binary target variable?

Question

Options

• AAdjusted R Square
• BMallow's Cp
• CChi Square
• DAverage Squared Error

• A
  7% (2)
• B
  4% (1)
• C
  15% (4)
• D
  74% (20)

Explanation

Average Squared Error (D) measures how well a model's predicted probabilities match actual binary outcomes on held-out validation data - lower values indicate better predictive accuracy, making it directly useful for comparing competing models on unseen data.

Why the distractors are wrong:

• A (Adjusted R²) is designed for continuous regression targets, not binary classification - it measures variance explained, which doesn't map meaningfully to 0/1 outcomes.
• B (Mallow's Cp) is a model selection criterion for linear regression that penalizes for extra predictors; it's not suited for binary prediction problems.
• C (Chi-Square) tests for association or goodness-of-fit but doesn't directly quantify predictive accuracy across models on a validation sample.

Memory tip: For binary targets, think about prediction error - you need a metric that works on probability outputs. "Average Squared Error on a validation sample" = a direct, model-agnostic measure of prediction quality, which is exactly what you want when comparing models on data they haven't seen.

No community discussion yet for this question.