PROFESSIONAL-DATA-ENGINEER · Question #353
You are monitoring your organization's data lake hosted on BigQuery. The ingestion pipelines read data from Pub/Sub and write the data into tables on BigQuery. After a new version of the ingestion…
The correct answer is C. 1. Check for duplicate rows in the BigQuery tables that have the daily partition data size doubled. Option C is correct because the clues point directly to duplicate row writes: Pub/Sub volume is unchanged (ruling out upstream data growth), yet only some tables doubled - a pattern consistent with a pipeline code bug that writes records twice to specific tables after the…
Question
Options
- A
- Check for duplicate rows in the BigQuery tables that have the daily partition data size doubled.
- B
- Check for code errors in the deployed pipelines.
- C
- Check for duplicate rows in the BigQuery tables that have the daily partition data size doubled.
- D
- Roll back the last deployment.
How the community answered
(16 responses)- A6% (1)
- B13% (2)
- C75% (12)
- D6% (1)
Explanation
Option C is correct because the clues point directly to duplicate row writes: Pub/Sub volume is unchanged (ruling out upstream data growth), yet only some tables doubled - a pattern consistent with a pipeline code bug that writes records twice to specific tables after the deployment. Confirming duplicates in the affected partitions first gives you hard evidence of the root cause before acting, which is the correct investigative order.
Option A likely differs from C in its follow-up step - for example, it may suggest rolling back or taking an action that doesn't address the underlying code defect, making it incomplete even if the first step is the same. Option B (checking for code errors) skips empirical confirmation; you should verify the symptom (duplicates) before diving into code, not assume a code error without data. Option D (rolling back immediately) is reactive, not investigative - it might stop the bleeding but leaves the bug unidentified and likely to recur in the next deployment.
Memory tip: When BigQuery storage grows but upstream (Pub/Sub) data doesn't, and only some tables are affected after a deployment, think "selective duplicate writes." The pattern "same input → double output in specific tables" = duplication bug, not volume spike.
Topics
Community Discussion
No community discussion yet for this question.