nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #119

You are experimenting with a built-in distributed XGBoost model in Vertex AI Workbench user- managed notebooks. You use BigQuery to split your data into training and validation sets using the…

The correct answer is C. The tables that you created to hold your training and validation records share some records, and. The method of splitting data into training and validation sets using independent RAND() conditions in BigQuery can lead to overlapping records, causing data leakage and artificially inflated model performance during development.

Submitted by ngozi_ng· Apr 18, 2026Data processing and feature engineering

Question

You are experimenting with a built-in distributed XGBoost model in Vertex AI Workbench user- managed notebooks. You use BigQuery to split your data into training and validation sets using the following queries: CREATE OR REPLACE TABLE 'myproject.mydataset.training' AS (SELECT * FROM 'myproject.mydataset.mytable' WHERE RAND() <= 0.8); CREATE OR REPLACE TABLE 'myproject.mydataset.validation' AS (SELECT * FROM 'myproject.mydataset.mytable' WHERE RAND() <= 0.2); After training the model, you achieve an area under the receiver operating characteristic curve (AUC ROC) value of 0.8, but after deploying the model to production, you notice that your model performance has dropped to an AUC ROC value of 0.65. What problem is most likely occurring?

Options

  • AThere is training-serving skew in your production environment.
  • BThere is not a sufficient amount of training data.
  • CThe tables that you created to hold your training and validation records share some records, and
  • DThe RAND() function generated a number that is less than 0.2 in both instances, so every record

How the community answered

(32 responses)
  • A
    25% (8)
  • B
    13% (4)
  • C
    56% (18)
  • D
    6% (2)

Why each option

The method of splitting data into training and validation sets using independent `RAND()` conditions in BigQuery can lead to overlapping records, causing data leakage and artificially inflated model performance during development.

AThere is training-serving skew in your production environment.

While training-serving skew is a common problem in production, the specific data splitting logic described provides a more direct and certain explanation for the observed discrepancy between training/validation AUC and production AUC.

BThere is not a sufficient amount of training data.

There is no information in the problem statement to suggest that the amount of training data is insufficient; the primary issue is the integrity of the data split.

CThe tables that you created to hold your training and validation records share some records, andCorrect

Using `RAND() <= 0.8` for training and a separate `RAND() <= 0.2` for validation independently means a record could satisfy both conditions (e.g., if RAND() generates 0.1), leading to data leakage where training and validation sets contain common records. This overlap causes the model to appear to perform better on the validation set than it would on truly unseen production data, resulting in a performance drop upon deployment.

DThe RAND() function generated a number that is less than 0.2 in both instances, so every record

The claim that `RAND()` generated a number less than 0.2 in both instances for *every* record is a statistically improbable scenario and an overgeneralization; the fundamental problem is the potential for *any* overlap due to independent random sampling.

Concept tested: Data leakage from incorrect data splitting

Source: https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#rand

Topics

#Data splitting#Data leakage#Model evaluation#BigQuery RAND()

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice