Google
PROFESSIONAL-DATA-ENGINEER · Question #151
You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and…
The correct answer is C. Use Cloud Dataflow to run your transformations. Monitor the job system lag with Stackdriver. Use the default autoscaling setting for worker instances. Explanation/Reference: Dataflow is good with autoscaling and stackdriver to monitor CPU and Storage.
Submitted by femi9· Mar 30, 2026Building and operationalizing data processing systems
Question
You are selecting services to write and transform JSON messages from Cloud Pub/Sub to BigQuery for a data pipeline on Google Cloud. You want to minimize service costs. You also want to monitor and accommodate input data volume that will vary in size with minimal manual intervention. What should you do?
Options
- AUse Cloud Dataproc to run your transformations. Monitor CPU utilization for the cluster. Resize the number of worker nodes in your cluster via the command
- BUse Cloud Dataproc to run your transformations. Use the diagnose command to generate an operational output archive. Locate the bottleneck and adjust
- CUse Cloud Dataflow to run your transformations. Monitor the job system lag with Stackdriver. Use the default autoscaling setting for worker instances.
- DUse Cloud Dataflow to run your transformations. Monitor the total execution time for a sampling of jobs.
How the community answered
(35 responses)- A9% (3)
- B6% (2)
- C83% (29)
- D3% (1)
Explanation
Explanation/Reference: Dataflow is good with autoscaling and stackdriver to monitor CPU and Storage.
Topics
#Cloud Dataflow#autoscaling#Pub/Sub to BigQuery#pipeline cost optimization
Community Discussion
No community discussion yet for this question.