nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #194

You have historical data covering the last three years in BigQuery and a data pipeline that delivers new data to BigQuery daily. You have noticed that when the Data Science team runs a query…

The correct answer is A. Re-create the tables using DDL. Partition the tables by a column containing a TIMESTAMP or DATE Type. Partitioning a BigQuery table by a TIMESTAMP or DATE column (option A) solves the root cause: when a query filters on a date column, BigQuery's partition pruning eliminates irrelevant partitions from the scan, so only the relevant days are read instead of the entire table…

Submitted by omar99· Mar 30, 2026Designing data processing systems

Question

You have historical data covering the last three years in BigQuery and a data pipeline that delivers new data to BigQuery daily. You have noticed that when the Data Science team runs a query filtered on a date column and limited to 3090 days of data, the query scans the entire table. You also noticed that your bill is increasing more quickly than you expected. You want to resolve the issue as cost-effectively as possible while maintaining the ability to conduct SQL queries. What should you do?

Options

  • ARe-create the tables using DDL. Partition the tables by a column containing a TIMESTAMP or DATE Type.
  • BRecommend that the Data Science team export the table to a CSV file on Cloud Storage and use Cloud Datalab to explore the data by reading the files directly.
  • CModify your pipeline to maintain the last 3090 days of data in one table and the longer history in a different table to minimize full table scans over the entire
  • DWrite an Apache Beam pipeline that creates a BigQuery table per day. Recommend that the Data Science team use wildcards on the table name suffixes to

How the community answered

(38 responses)
  • A
    71% (27)
  • B
    8% (3)
  • C
    18% (7)
  • D
    3% (1)

Explanation

Partitioning a BigQuery table by a TIMESTAMP or DATE column (option A) solves the root cause: when a query filters on a date column, BigQuery's partition pruning eliminates irrelevant partitions from the scan, so only the relevant days are read instead of the entire table - directly reducing both query time and cost.

Why the distractors fail:

  • B breaks the SQL requirement - Cloud Datalab reading raw CSV files is not a SQL workflow and adds operational friction for the Data Science team.
  • C is a manual workaround that adds pipeline complexity and still doesn't prevent full scans within each table; it also doesn't scale cleanly as data grows.
  • D per-day tables with wildcard queries are a legacy pattern that predates native partitioning; wildcards still risk scanning all matching tables and add naming/management overhead without the cost controls partitioning provides.

Memory tip: Think of partitioning as giving BigQuery a filing cabinet with labeled drawers - a date filter lets it open only the drawer it needs instead of dumping the entire cabinet on the floor. Anytime a GCP exam mentions "full table scans" + "cost" + "date filtering," partitioning by DATE/TIMESTAMP is almost always the answer.

Topics

#BigQuery Partitioning#BigQuery Cost Optimization#Data Warehouse Design#Performance Tuning

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-DATA-ENGINEER Practice