Your team is working on an NLP research project to predict political affiliation of authors based on articles they have written. You have a large training dataset that is structured like this: You…

Question

Exhibit

Options

• ADistribute texts randomly across the train-test-eval subsets:
• BDistribute authors randomly across the train-test-eval subsets: (*)
• CDistribute sentences randomly across the train-test-eval subsets:
• DDistribute paragraphs of texts (i.e., chunks of consecutive sentences) across the train-test-eval

• A
  6% (3)
• B
  82% (40)
• C
  10% (5)
• D
  2% (1)

Explanation

The unit of splitting must be the author, not individual texts, sentences, or paragraphs. If articles from the same author appear in both the training and test sets, the model can memorize author-specific writing quirks (vocabulary, style) rather than learning generalizable signals of political affiliation. This constitutes data leakage at the author level, artificially inflating evaluation metrics. Splitting randomly by text (A), sentence (C), or paragraph (D) all risk placing different excerpts from the same author across multiple splits. Distributing by author ensures the model is evaluated on authors it has never seen, which reflects real-world deployment conditions.

No community discussion yet for this question.