Your organization stores customer data in an on-premises Apache Hadoop cluster in Apache Parquet format. Data is processed on a daily basis by Apache Spark jobs that run on the cluster. You are…

Question

Options

• AMigrate your data to Cloud Storage and migrate the metadata to Dataproc Metastore (DPMS). Refactor Spark pipelines to write and read data on Cloud
• BMigrate your data to Cloud Storage and register the bucket as a Dataplex asset. Refactor Spark pipelines to write and read data on Cloud Storage, and run
• CMigrate your data to BigQuery. Refactor Spark pipelines to write and read data on BigQuery, and run them on Dataproc Serverless.
• DMigrate your data to BigLake. Refactor Spark pipelines to write and read data on Cloud Storage, and run them on Dataproc on Compute Engine.

• A
  61% (22)
• B
  22% (8)
• C
  6% (2)
• D
  11% (4)

Explanation

Option A is correct because Cloud Storage is the natural HDFS replacement for Parquet files, and Dataproc Metastore (DPMS) - a managed Hive Metastore - preserves the table metadata so that both Spark and BigQuery can discover and query the same Parquet data without duplicating or transforming it. This satisfies the BigQuery requirement via external tables while requiring minimal Spark code changes (GCS is accessed just like HDFS).

Option B (Dataplex) is a data governance/cataloging layer, not a metastore; it adds organizational overhead but doesn't natively expose Parquet files to BigQuery as queryable tables the way DPMS does.

Option C migrating data into BigQuery native storage forces significant Spark pipeline refactoring and eliminates the Parquet-on-GCS pattern entirely - the opposite of "minimizing ETL changes."

Option D uses Dataproc on Compute Engine (cluster-based, not serverless), which has higher provisioning overhead and cost compared to the serverless variant implied by option A, and BigLake alone doesn't fulfill the metastore role that keeps Spark jobs working with minimal changes.

Memory tip: Think of DPMS as "Hive Metastore as a managed service" - it's the glue that lets both Spark and BigQuery share the same Parquet files on GCS without moving or transforming the data, which is exactly what "minimize ETL changes" means on the exam.

No community discussion yet for this question.