CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #50
A production cluster has 3 executor nodes and uses the same virtual machine type for the driver and executor. When evaluating the Ganglia Metrics for this cluster, which indicator would signal a…
The correct answer is E. Overall cluster CPU utilization is around 25%. The cluster has 4 nodes total (1 driver + 3 executors), all using the same VM type. Ganglia reports aggregate metrics across the entire cluster. If the driver is experiencing a bottleneck (e.g., collect() calls, large broadcast variables, Python UDF serialization, or heavy…
Question
A production cluster has 3 executor nodes and uses the same virtual machine type for the driver and executor. When evaluating the Ganglia Metrics for this cluster, which indicator would signal a bottleneck caused by code executing on the driver?
Options
- AThe five Minute Load Average remains consistent/flat
- BBytes Received never exceeds 80 million bytes per second
- CTotal Disk Space remains constant
- DNetwork I/O never spikes
- EOverall cluster CPU utilization is around 25%
How the community answered
(31 responses)- A3% (1)
- B6% (2)
- C3% (1)
- D16% (5)
- E71% (22)
Explanation
The cluster has 4 nodes total (1 driver + 3 executors), all using the same VM type. Ganglia reports aggregate metrics across the entire cluster. If the driver is experiencing a bottleneck (e.g., collect() calls, large broadcast variables, Python UDF serialization, or heavy driver-side computation), the driver's CPU will be pegged near 100% while the 3 executor nodes remain largely idle. The overall cluster CPU utilization would therefore be approximately 1/4 = 25%. A flat ~25% cluster-wide CPU reading indicates only one of the four nodes is doing meaningful work - a classic signature of driver-side bottleneck. If executors were the bottleneck, cluster CPU would be much higher (75%–100%).
Topics
Community Discussion
No community discussion yet for this question.