Google
PROFESSIONAL-DATA-ENGINEER · Question #167
After migrating ETL jobs to run on BigQuery, you need to verify that the output of the migrated jobs is the same as the output of the original. You've loaded a table containing the output of the origi
The correct answer is C. Use a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table. Explanation/Reference: Full comparison with this option, rest are comparison on sample which doesn't ensure all the data will be ok.
Submitted by wei.xz· Mar 30, 2026Ensuring solution quality
Question
After migrating ETL jobs to run on BigQuery, you need to verify that the output of the migrated jobs is the same as the output of the original. You've loaded a table containing the output of the original job and want to compare the contents with output from the migrated job to show that they are identical. The tables do not contain a primary key column that would enable you to join them together for comparison. What should you do?
Options
- ASelect random samples from the tables using the RAND() function and compare the samples.
- BSelect random samples from the tables using the HASH() function and compare the samples.
- CUse a Dataproc cluster and the BigQuery Hadoop connector to read the data from each table and calculate a hash from non-timestamp columns of the table
- DCreate stratified random samples using the OVER() function and compare equivalent samples from each table.
How the community answered
(35 responses)- A20% (7)
- B6% (2)
- C63% (22)
- D11% (4)
Explanation
Explanation/Reference: Full comparison with this option, rest are comparison on sample which doesn't ensure all the data will be ok.
Topics
#ETL validation#data comparison#BigQuery hashing#Dataproc
Community Discussion
No community discussion yet for this question.