CERTIFIED-DATA-ENGINEER-PROFESSIONAL · Question #125
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM. Given an extremely long-running job for which completion mus
The correct answer is B. - Total VMs: 16. Fault tolerance in Spark is achieved through task re-execution on surviving nodes. With 16 VMs, each VM holds a smaller fraction of the total work (1/16th), so the failure of one or even several VMs still leaves enough surviving executors to recompute lost tasks and complete the
Question
Each configuration below is identical to the extent that each cluster has 400 GB total of RAM 160 total cores and only one Executor per VM. Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?
Options
- A
- Total VMs: 8
- B
- Total VMs: 16
- C
- Total VMs: 1
- D
- Total VMs: 4
- E
- Total VMs: 2
How the community answered
(30 responses)- A7% (2)
- B83% (25)
- C7% (2)
- E3% (1)
Explanation
Fault tolerance in Spark is achieved through task re-execution on surviving nodes. With 16 VMs, each VM holds a smaller fraction of the total work (1/16th), so the failure of one or even several VMs still leaves enough surviving executors to recompute lost tasks and complete the job. In contrast, configurations with fewer, larger VMs (e.g., 1, 2, or 4 VMs) are more vulnerable-losing a single VM with fewer total nodes means losing a much larger percentage of capacity, and the job may fail to complete. More VMs = smaller blast radius per failure = higher fault tolerance, even though total resources are identical across all options.
Topics
Community Discussion
No community discussion yet for this question.