PROFESSIONAL-DATA-ENGINEER · Question #19
Your company's on-premises Apache Hadoop servers are approaching end-of-life, and IT has decided to migrate the cluster to Google Cloud Dataproc. A like-for- like migration of the cluster would requir
A is correct because Cloud Dataproc is natively integrated with Google Cloud Storage (GCS), which costs significantly less than Persistent Disk - GCS is object storage priced at a fraction of block storage rates. By storing all Hadoop data in GCS instead of on-node Persistent Dis
Question
Options
- APut the data into Google Cloud Storage.
- BUse preemptible virtual machines (VMs) for the Cloud Dataproc cluster.
- CTune the Cloud Dataproc cluster so that there is just enough disk for all data.
- DMigrate some of the cold data into Google Cloud Storage, and keep only the hot data in Persistent Disk.
Explanation
A is correct because Cloud Dataproc is natively integrated with Google Cloud Storage (GCS), which costs significantly less than Persistent Disk - GCS is object storage priced at a fraction of block storage rates. By storing all Hadoop data in GCS instead of on-node Persistent Disk, you achieve the same functionality at a much lower cost, and Dataproc's Cloud Storage connector handles HDFS-compatible access transparently.
B is wrong because preemptible VMs reduce compute costs, not storage costs - they don't address the 50 TB per node Persistent Disk concern at all.
C is wrong because tuning disk capacity is a marginal optimization; if the data genuinely requires 50 TB per node, you can't tune your way out of that need - you'd still be paying for expensive block storage.
D is wrong because it's a partial solution - splitting data between GCS (cold) and Persistent Disk (hot) still incurs high block storage costs for the hot tier. Option A eliminates Persistent Disk for data storage entirely, which is the maximum cost reduction.
Memory tip: Think of Dataproc as "HDFS replaced by GCS" - just as HDFS was the storage layer for on-prem Hadoop, GCS plays that role in the cloud, and object storage is always cheaper than block storage at scale.
Topics
Community Discussion
No community discussion yet for this question.