nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #87

You are working on a classification problem with time series data. After conducting just a few experiments using random cross-validation, you achieved an Area Under the Receiver Operating Characterist

The correct answer is B. Address data leakage by applying nested cross-validation during model training.. A 99% AUC ROC on training data after only a few experiments with no tuning is a red flag - it's unrealistically high and strongly indicates data leakage rather than a genuinely good model. With time series data, using random cross-validation is the most common cause: future data

Submitted by haruto_sh· Apr 18, 2026ML model development

Question

You are working on a classification problem with time series data. After conducting just a few experiments using random cross-validation, you achieved an Area Under the Receiver Operating Characteristic Curve (AUC ROC) value of 99% on the training data. You haven't explored using any sophisticated algorithms or spent any time on hyperparameter tuning. What should your next step be to identify and fix the problem?

Options

  • AAddress the model overfitting by using a less complex algorithm and use k-fold cross-validation.
  • BAddress data leakage by applying nested cross-validation during model training.
  • CAddress data leakage by removing features highly correlated with the target value.
  • DAddress the model overfitting by tuning the hyperparameters to reduce the AUC ROC value.

How the community answered

(61 responses)
  • A
    7% (4)
  • B
    69% (42)
  • C
    20% (12)
  • D
    5% (3)

Explanation

A 99% AUC ROC on training data after only a few experiments with no tuning is a red flag - it's unrealistically high and strongly indicates data leakage rather than a genuinely good model. With time series data, using random cross-validation is the most common cause: future data points randomly end up in the training fold, leaking future information into the model. The correct fix is to use time-aware cross-validation (e.g., walk-forward or nested temporal cross-validation), ensuring the model only ever trains on past data and validates on future data. Removing correlated features (C) addresses feature leakage, not temporal leakage. Overfitting explanations (A, D) don't fit because overfitting would show a gap between training and validation scores, not an implausibly perfect training score from the start.

Topics

#Data Leakage#Cross-Validation#Time Series Data#Model Evaluation

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice