nerdexam
Google

PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #143

You work on a data science team at a bank and are creating an ML model to predict loan default risk. You have collected and cleaned hundreds of millions of records worth of training data in a BigQuery

The correct answer is D. Use TensorFlow I/O's BigQuery Reader to directly read the data.. To minimize data ingestion bottlenecks for TensorFlow and Vertex AI training on hundreds of millions of BigQuery records, the most scalable and efficient approach is to directly read data from BigQuery.

Submitted by sofia.br· Apr 18, 2026ML pipeline operationalization

Question

You work on a data science team at a bank and are creating an ML model to predict loan default risk. You have collected and cleaned hundreds of millions of records worth of training data in a BigQuery table, and you now want to develop and compare multiple models on this data using TensorFlow and Vertex AI. You want to minimize any bottlenecks during the data ingestion state while considering scalability. What should you do?

Options

  • AUse the BigQuery client library to load data into a dataframe, and use
  • BExport data to CSV files in Cloud Storage, and use tf.data.TextLineDataset() to read them.
  • CConvert the data into TFRecords, and use tf.data.TFRecordDataset() to read them.
  • DUse TensorFlow I/O's BigQuery Reader to directly read the data.

How the community answered

(51 responses)
  • A
    14% (7)
  • B
    4% (2)
  • C
    6% (3)
  • D
    76% (39)

Why each option

To minimize data ingestion bottlenecks for TensorFlow and Vertex AI training on hundreds of millions of BigQuery records, the most scalable and efficient approach is to directly read data from BigQuery.

AUse the BigQuery client library to load data into a dataframe, and use

Loading hundreds of millions of records into a single dataframe using the BigQuery client library would likely cause out-of-memory errors and be a significant bottleneck due to the sheer volume of data.

BExport data to CSV files in Cloud Storage, and use tf.data.TextLineDataset() to read them.

Exporting hundreds of millions of records to CSV files and then reading them with `tf.data.TextLineDataset()` introduces an unnecessary intermediate storage step and can be slower due to parsing CSVs.

CConvert the data into TFRecords, and use tf.data.TFRecordDataset() to read them.

Converting the data into TFRecords, while efficient, still requires an initial export and conversion step, which is an additional overhead compared to directly reading from BigQuery, and necessitates managing the TFRecord files.

DUse TensorFlow I/O's BigQuery Reader to directly read the data.Correct

TensorFlow I/O's BigQuery Reader is specifically designed to efficiently stream large datasets directly from BigQuery into TensorFlow `tf.data.Dataset` format, optimizing for performance and scalability by avoiding intermediate storage steps and leveraging BigQuery's data processing capabilities. This direct integration minimizes I/O bottlenecks and simplifies the data pipeline when working with vast amounts of data stored in BigQuery.

Concept tested: Efficient BigQuery data ingestion for TensorFlow

Source: https://www.tensorflow.org/io/tutorials/bigquery

Topics

#Data Ingestion#BigQuery#TensorFlow I/O#Scalability

Community Discussion

No community discussion yet for this question.

Full PROFESSIONAL-MACHINE-LEARNING-ENGINEER Practice