DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #121
Which of the following describes the relationship between nodes and executors?
The correct answer is C. An executor is a processing engine running on a node. In Apache Spark, an executor is a JVM process that runs on a worker node and is responsible for executing tasks assigned to it by the driver.
Question
Which of the following describes the relationship between nodes and executors?
Options
- AExecutors and nodes are not related.
- BAnode is a processing engine running on an executor.
- CAn executor is a processing engine running on a node.
- DThere are always the same number of executors and nodes.
- EThere are always more nodes than executors.
How the community answered
(41 responses)- B2% (1)
- C88% (36)
- D2% (1)
- E7% (3)
Why each option
In Apache Spark, an executor is a JVM process that runs on a worker node and is responsible for executing tasks assigned to it by the driver.
Executors and nodes are directly related - executors run on nodes as JVM processes.
This reverses the relationship - a node is not a processing engine running on an executor; nodes are the underlying machines.
An executor is a processing engine - specifically a JVM process - that is launched on a worker node. The node is the physical or virtual machine in the cluster, while the executor is the software process running on top of it that performs computation and stores data for the application.
There is no requirement for a 1:1 ratio - a single node can host multiple executors depending on resource configuration.
There is no rule enforcing more nodes than executors - in fact, a single node can run multiple executors.
Concept tested: Spark cluster architecture - executors and nodes
Source: https://spark.apache.org/docs/latest/cluster-overview.html
Topics
Community Discussion
No community discussion yet for this question.