Which of the following statements about Spark's stability is incorrect?

Question

Options

• ASpark is designed to support the loss of any set of worker nodes.
• BSpark will rerun any failed tasks due to failed worker nodes.
• CSpark will recompute data cached on failed worker nodes.
• DSpark will spill data to disk if it does not fit in memory.
• ESpark will reassign the driver to a worker node if the driver's node fails.

• A
  3% (1)
• B
  3% (1)
• C
  8% (3)
• D
  17% (6)
• E
  69% (25)

Explanation

Option E is incorrect because the Spark driver is a single point of failure - if the driver node fails, the entire application crashes and must be restarted from scratch. Spark has no built-in mechanism to migrate or reassign the driver to another node mid-application.

The other options are all true statements about Spark's resilience:

• A is correct: Spark's RDD lineage graph allows it to recover from the loss of any worker nodes by recomputing lost partitions.
• B is correct: The task scheduler detects failed tasks and automatically reschedules them on surviving workers.
• C is correct: Cached/persisted data on a failed node is recomputed by replaying the RDD lineage - no data is permanently lost just because a cache was on a dead node.
• D is correct: Spark uses a tiered memory model where data that doesn't fit in RAM spills to disk, preventing OOM crashes.

Memory tip: Think of the driver as the "CEO" of a Spark job - you can replace workers (employees) on the fly, but if the CEO's office burns down, the company halts until leadership is re-established from outside. Workers are fault-tolerant; the driver is not.

No community discussion yet for this question.