Which of the following statements about the Spark driver is true?

Question

Options

• ASpark driver is horizontally scaled to increase overall processing throughput.
• BSpark driver is the most coarse level of the Spark execution hierarchy.
• CSpark driver is fault tolerant - if it fails, it will recover the entire Spark application.
• DSpark driver is responsible for scheduling the execution of data by various worker nodes in cluster
• ESpark driver is only compatible with its included cluster manager.

• A
  4% (1)
• B
  8% (2)
• D
  88% (23)

Explanation

The Spark driver is the central coordinator of a Spark application. It is responsible for maintaining the SparkContext, building the DAG of transformations, creating a logical and physical execution plan, and scheduling tasks across worker nodes/executors. Option A is false - the driver is a single process and cannot be horizontally scaled. Option B is false - Jobs are the coarsest level in the execution hierarchy, not the driver. Option C is false - the driver is a single point of failure; if it crashes, the entire Spark application fails. Option E is false - Spark supports multiple cluster managers including YARN, Kubernetes, Mesos, and Standalone.

No community discussion yet for this question.