Which of the following lists the units of work performed by Spark from largest to smallest?

The correct answer is C. Job, Stage, Task. Spark's execution hierarchy from largest to smallest is: Job → Stage → Task. A Job is created each time an action (e.g., count(), collect(), write()) is called on a DataFrame. Each Job is divided into one or more Stages, where stage boundaries are determined by shuffle…

Spark Architecture and Execution

Question

Options

ATask, Stage, Job
BStage, Job, Task
CJob, Stage, Task
DStage, Task, Job
EJob, Task, Stage

How the community answered

(34 responses)

A
3% (1)
B
3% (1)
C
91% (31)
D
3% (1)

Explanation

Spark's execution hierarchy from largest to smallest is: Job → Stage → Task. A Job is created each time an action (e.g., count(), collect(), write()) is called on a DataFrame. Each Job is divided into one or more Stages, where stage boundaries are determined by shuffle operations - wide transformations like groupBy() or join() that require data to be exchanged between partitions create new stages. Each Stage is further broken down into Tasks, where each task processes exactly one partition of data. Tasks are the smallest unit of work and run in parallel across the cluster's executor slots.

Topics

#Spark Execution Model#Spark Architecture#Jobs Stages Tasks#Spark Fundamentals

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice