PROFESSIONAL-DATA-ENGINEER · Question #220
You have a data pipeline with a Cloud Dataflow job that aggregates and writes time series metrics to Cloud Bigtable. This data feeds a dashboard used by thousands of users across the organization…
The correct answer is B. Increase the maximum number of Cloud Dataflow workers by setting maxNumWorkers in PipelineOptions C. Increase the number of nodes in the Cloud Bigtable cluster. Explanation/Reference: If you need to change DataFlow pipeline, better using Combine than CoGroupByKey according Google recommendations: Combine is orders of magnitude faster than GroupByKey because Dataflow knows how to parallelize a combine step. Combine allows Dataflow to…
Question
Options
- AConfigure your Cloud Dataflow pipeline to use local execution
- BIncrease the maximum number of Cloud Dataflow workers by setting maxNumWorkers in PipelineOptions
- CIncrease the number of nodes in the Cloud Bigtable cluster
- DModify your Cloud Dataflow pipeline to use the Flatten transform before writing to Cloud Bigtable
- EModify your Cloud Dataflow pipeline to use the CoGroupByKey transform before writing to Cloud Bigtable
How the community answered
(37 responses)- A11% (4)
- B78% (29)
- D8% (3)
- E3% (1)
Explanation
Explanation/Reference: If you need to change DataFlow pipeline, better using Combine than CoGroupByKey according Google recommendations: Combine is orders of magnitude faster than GroupByKey because Dataflow knows how to parallelize a combine step. Combine allows Dataflow to distribute a key to multiple workers and process it in parallel.
Topics
Community Discussion
No community discussion yet for this question.