PROFESSIONAL-MACHINE-LEARNING-ENGINEER · Question #153
You have built a model that is trained on data stored in Parquet files. You access the data through a Hive table hosted on Google Cloud. You preprocessed these data with PySpark and exported it as a…
The correct answer is C. Add a ContainerOp to your pipeline that spins a Dataproc cluster, runs a transformation, and then. The recommended approach to parametrize the model training in Kubeflow Pipelines would be to add a ContainerOp to the pipeline that spins up a Dataproc cluster, runs the PySpark transformation step, and saves the transformed data in Cloud Storage. This approach allows for easy…
Question
Options
- ARemove the data transformation step from your pipeline.
- BContainerize the PySpark transformation step, and add it to your pipeline.
- CAdd a ContainerOp to your pipeline that spins a Dataproc cluster, runs a transformation, and then
- DDeploy Apache Spark at a separate node pool in a Google Kubernetes Engine cluster. Add a
How the community answered
(35 responses)- A6% (2)
- B9% (3)
- C83% (29)
- D3% (1)
Explanation
The recommended approach to parametrize the model training in Kubeflow Pipelines would be to add a ContainerOp to the pipeline that spins up a Dataproc cluster, runs the PySpark transformation step, and saves the transformed data in Cloud Storage. This approach allows for easy integration of PySpark transformations with Kubeflow Pipelines while taking advantage of the scalability and efficiency of Dataproc.
Topics
Community Discussion
No community discussion yet for this question.