Which of the following operations will fail to trigger evaluation?

The correct answer is D. DataFrame.join(). DataFrame.join() is a transformation, not an action, so it does not trigger evaluation (execution). It is lazy - it simply adds a join step to the logical plan (DAG) and returns a new DataFrame. All the other options are actions that do trigger evaluation: collect() retrieves…

Spark DataFrame Operations and Execution

Question

Options

ADataFrame.collect()
BDataFrame.count()
CDataFrame.first()
DDataFrame.join()
EDataFrame.take()

How the community answered

(70 responses)

A
1% (1)
C
3% (2)
D
94% (66)
E
1% (1)

Explanation

DataFrame.join() is a transformation, not an action, so it does not trigger evaluation (execution). It is lazy - it simply adds a join step to the logical plan (DAG) and returns a new DataFrame. All the other options are actions that do trigger evaluation: collect() retrieves all rows to the driver, count() returns the number of rows, first() returns the first row, and take(n) returns the first n rows. Only actions force Spark to execute the accumulated transformations.

Topics

#Spark Lazy Evaluation#Spark Actions#Spark Transformations#DataFrame Operations

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice