Which of the following operations is least likely to result in a shuffle?

The correct answer is B. DataFrame.fliter(). filter() (the correct spelling - 'fliter' in the option is a typo) is a narrow transformation: it evaluates each row independently against a condition and keeps or discards it. No data needs to move between partitions, so no shuffle occurs. All other listed operations are wide…

Understand Spark's execution model, including transformations, actions, and the implications of data shuffling for performance.

Question

Options

ADataFrame.join()
BDataFrame.fliter()
CDataFrame.orderBy()
DDataFrame.distinct()
EDataFrame.intersect()

How the community answered

(65 responses)

B
94% (61)
C
2% (1)
D
3% (2)
E
2% (1)

Explanation

filter() (the correct spelling - 'fliter' in the option is a typo) is a narrow transformation: it evaluates each row independently against a condition and keeps or discards it. No data needs to move between partitions, so no shuffle occurs. All other listed operations are wide transformations that require data from multiple partitions to be exchanged across the network: orderBy() requires a global sort across all partitions; distinct() must compare rows across partitions to remove duplicates; intersect() must find common rows across partitions; join() (without broadcast) must co-locate matching rows from both DataFrames. Narrow transformations like filter(), map(), and select() never trigger a shuffle.

Topics

#Spark DataFrame Operations#Data Shuffling#Wide Transformations#Performance Considerations

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice