You are training a TensorFlow model on a structured dataset with 100 billion records stored in several CSV files. You need to improve the input/output execution performance. What should you do?

Question

Options

• ALoad the data into BigQuery, and read the data from BigQuery.
• BLoad the data into Cloud Bigtable, and read the data from Bigtable.
• CConvert the CSV files into shards of TFRecords, and store the data in Cloud Storage.
• DConvert the CSV files into shards of TFRecords, and store the data in the Hadoop Distributed File

• A
  2% (1)
• B
  5% (2)
• C
  84% (37)
• D
  9% (4)

Explanation

TFRecords are TensorFlow's native binary serialization format, purpose-built for efficient training data pipelines. They enable highly optimized sequential reads, work seamlessly with the tf.data API for prefetching and parallel loading, and eliminate the parsing overhead of CSV (which is text-based and requires type inference). Sharding the TFRecords into many files allows tf.data to read multiple shards in parallel, saturating I/O bandwidth. Storing them in Cloud Storage makes them accessible to distributed training across multiple workers. Option A (BigQuery) requires a BigQuery Storage connector, adds query overhead, and is optimized for analytics queries rather than sequential ML training reads. Option B (Cloud Bigtable) is a key-value store optimized for random low-latency lookups, not the high-throughput sequential reads needed for ML training. Option D (HDFS) is an on-premises Hadoop filesystem-using it for GCP-native training adds unnecessary infrastructure complexity and is not a cloud-native solution.

No community discussion yet for this question.