PROFESSIONAL-MACHINE-LEARNING-ENGINEER Question #213: Real Exam Question with Answer & Explanation

Question

You work for a startup that has multiple data science workloads. Your compute infrastructure is currently on-premises, and the data science workloads are native to PySpark. Your team plans to migrate their data science workloads to Google Cloud. You need to build a proof of concept to migrate one data science job to Google Cloud. You want to propose a migration process that requires minimal cost and effort. What should you do first?

Options

• ACreate a n2-standard-4 VM instance and install Java, Scala, and Apache Spark dependencies on
• BCreate a Google Kubernetes Engine cluster with a basic node pool configuration, install Java,
• CCreate a Standard (1 master, 3 workers) Dataproc cluster, and run a Vertex AI Workbench
• DCreate a Vertex AI Workbench notebook with instance type n2-standard-4.

Explanation

For a minimal cost and effort proof of concept to migrate an on-premises PySpark data science job to Google Cloud, the first step should be to create a Vertex AI Workbench notebook.

Common mistakes.

• A. Manually creating a VM instance and installing all Spark dependencies is significantly higher effort and generally not recommended as a first step for a managed cloud environment PoC when managed services are available.
• B. Setting up Spark on a Google Kubernetes Engine (GKE) cluster is a complex task requiring deep Kubernetes knowledge and is not suitable for a 'minimal effort' initial proof of concept.
• C. While creating a Dataproc cluster and a Vertex AI Workbench notebook is an excellent solution for running PySpark at scale, it is a higher initial cost and effort than simply starting with a Workbench notebook for a proof of concept of a single job.

Concept tested. PySpark migration PoC (minimal effort)

Reference. https://cloud.google.com/vertex-ai/docs/workbench/overview

No community discussion yet for this question.