PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #222
You need to develop a custom TensorFlow model that will be used for online predictions. The training data is stored in BigQuery You need to apply instance-level data transformations to the data for…
The correct answer is D. Create an Apache Beam pipeline to read the data from BigQuery and preprocess it by using. For instance-level data transformations on BigQuery data with a custom TensorFlow model, an Apache Beam pipeline ensures consistent preprocessing for both training and online serving.
Question
Options
- ACreate a BigQuery script to preprocess the data, and write the result to another BigQuery table.
- BCreate a pipeline in Vertex AI Pipelines to read the data from BigQuery and preprocess it using a
- CCreate a preprocessing function that reads and transforms the data from BigQuery. Create a
- DCreate an Apache Beam pipeline to read the data from BigQuery and preprocess it by using
How the community answered
(36 responses)- A3% (1)
- B8% (3)
- C14% (5)
- D75% (27)
Why each option
For instance-level data transformations on BigQuery data with a custom TensorFlow model, an Apache Beam pipeline ensures consistent preprocessing for both training and online serving.
A BigQuery script can preprocess data but lacks the ability to easily export the transformation logic for consistent application during TensorFlow model serving.
Vertex AI Pipelines can orchestrate preprocessing, but creating a general pipeline component alone doesn't directly address the specific need for consistent instance-level preprocessing logic exportable for online serving.
Creating a preprocessing function is a good step, but packaging it for consistent application in a custom TensorFlow model for online predictions, especially with BigQuery data and the need for a scalable pipeline, is best handled by Apache Beam/tf.Transform.
An Apache Beam pipeline, often executed via Google Cloud Dataflow, is ideal for reading and transforming large datasets from BigQuery. It allows for the definition of transformations that can then be exported as a TensorFlow Transform graph, which can be used during both model training and serving, ensuring consistency and preventing training-serving skew.
Concept tested: Consistent data preprocessing with Apache Beam/tf.Transform
Source: https://cloud.google.com/dataflow/docs/guides/dataflow-ml-preprocessing-with-tft
Topics
Community Discussion
No community discussion yet for this question.