nerdexam
Google

PROFESSIONAL-DATA-ENGINEER · Question #312

You need to modernize your existing on-premises data strategy. Your organization currently uses: - Apache Hadoop clusters for processing multiple large data sets, including on-premises Hadoop Distribu

The correct answer is B. Use Dataproc to migrate Hadoop clusters to Google Cloud, and Cloud Storage to handle any HDFS use cases. Orchestrate your pipelines with Cloud. Option B is correct because Dataproc is Google Cloud's managed Hadoop and Spark service, enabling a near-direct lift-and-shift of existing Hadoop clusters, while Cloud Storage serves as the recommended HDFS replacement. Most critically, Cloud Composer is Google Cloud's managed Ap

Submitted by satoshi_tk· Mar 30, 2026Designing data processing systems

Question

You need to modernize your existing on-premises data strategy. Your organization currently uses: - Apache Hadoop clusters for processing multiple large data sets, including on-premises Hadoop Distributed File System (HDFS) for data replication. - Apache Airflow to orchestrate hundreds of ETL pipelines with thousands of job steps. You need to set up a new architecture in Google Cloud that can handle your Hadoop workloads and requires minimal changes to your existing orchestration processes. What should you do?

Options

  • AUse Bigtable for your large workloads, with connections to Cloud Storage to handle any HDFS use cases. Orchestrate your pipelines with Cloud Composer.
  • BUse Dataproc to migrate Hadoop clusters to Google Cloud, and Cloud Storage to handle any HDFS use cases. Orchestrate your pipelines with Cloud
  • CUse Dataproc to migrate Hadoop clusters to Google Cloud, and Cloud Storage to handle any HDFS use cases. Convert your ETL pipelines to Dataflow.
  • DUse Dataproc to migrate your Hadoop clusters to Google Cloud, and Cloud Storage to handle any HDFS use cases. Use Cloud Data Fusion to visually design

How the community answered

(16 responses)
  • A
    6% (1)
  • B
    88% (14)
  • D
    6% (1)

Explanation

Option B is correct because Dataproc is Google Cloud's managed Hadoop and Spark service, enabling a near-direct lift-and-shift of existing Hadoop clusters, while Cloud Storage serves as the recommended HDFS replacement. Most critically, Cloud Composer is Google Cloud's managed Apache Airflow service - your existing Airflow DAGs and pipeline definitions migrate with minimal modification, which directly satisfies the "minimal changes to orchestration" requirement.

Option A is wrong because Bigtable is a NoSQL wide-column database, not a Hadoop processing engine - it cannot run Hadoop workloads or replace your cluster-based processing.

Option C is wrong because Dataflow is based on Apache Beam, which would require rewriting your ETL pipelines in a new paradigm - the opposite of "minimal changes."

Option D is wrong because Cloud Data Fusion is a visual pipeline design tool that would require rebuilding your pipelines from scratch using a GUI, again violating the minimal-change constraint.

Memory tip: Map the tools: Dataproc = managed Hadoop/Spark, Cloud Storage = HDFS replacement, Cloud Composer = managed Airflow. When a question asks for Hadoop migration with minimal orchestration changes, "Composer" is the giveaway - it's literally Airflow, just hosted.

Topics

#Data Migration#Hadoop#ETL Orchestration#Google Cloud Dataproc

Community Discussion

No community discussion yet for this question.

Sign up to join the discussion
Full PROFESSIONAL-DATA-ENGINEER Practice