Which of the following description of the validation set is wrong?

Question

Options

• AThe verification set can coincide with the test set.
• BThe test set can coincide with the training set
• CThe subset used to pick hyperparameters is called a validation set
• DTypically 80% of the training data 1s used for training and 20% 1s used for verification.

• A
  73% (27)
• B
  16% (6)
• C
  8% (3)
• D
  3% (1)

Explanation

Option A is wrong because the validation (verification) set and test set must remain strictly separate - the validation set is used during development to tune hyperparameters, while the test set is held out for a final, unbiased evaluation. If they coincide, you're effectively tuning on your test data, which renders the test results meaningless as an independent measure of generalization.

Options C and D are true: C correctly defines the validation set's purpose (hyperparameter selection), and D accurately reflects the common 80/20 data split used in practice. Option B, while poor practice, is considered technically permissible in some frameworks - for instance, when no separate test set exists and the same data is reused, or when a final model is retrained on all available data before deployment.

Memory tip: Think of the test set as a "sealed envelope" - once you open it (use it during training/tuning), the result inside is no longer trustworthy. The validation set is your scratch paper; the test set is the final exam you only take once.

No community discussion yet for this question.