DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #98
Which of the following types of processes induces a stage boundary?
The correct answer is A. Shuffle. A shuffle operation induces a stage boundary in Spark. A shuffle occurs when data must be redistributed across partitions between nodes (e.g., during groupBy, join, or repartition). Because data from multiple upstream partitions must be written to disk, transferred over the…
Question
Which of the following types of processes induces a stage boundary?
Options
- AShuffle
- BCaching
- CExecutor failure
- DJob delegation
- EApplication failure
How the community answered
(58 responses)- A88% (51)
- B7% (4)
- C2% (1)
- E3% (2)
Explanation
A shuffle operation induces a stage boundary in Spark. A shuffle occurs when data must be redistributed across partitions between nodes (e.g., during groupBy, join, or repartition). Because data from multiple upstream partitions must be written to disk, transferred over the network, and read by downstream tasks, Spark cannot pipeline across a shuffle - it must complete the upstream stage before starting the downstream stage. This boundary between stages is defined by the shuffle. Caching, executor failures, job delegation, and application failures do not define stage boundaries in the execution DAG.
Topics
Community Discussion
No community discussion yet for this question.