Which of the following DataFrame operations is classified as an action?

The correct answer is C. DataFrame.take(). DataFrame.take(n) is an action because it triggers execution of the Spark DAG and returns actual data to the driver - specifically, an array of the first n Row objects. Actions are what force Spark to leave lazy evaluation mode and compute results. The distractors are all…

Core Spark concepts - distinguishing between DataFrame actions and transformations

Question

Options

ADataFrame.drop()
BDataFrame.coalesce()
CDataFrame.take()
DDataFrame.join()
EDataFrame.filter()

How the community answered

(29 responses)

B
3% (1)
C
93% (27)
D
3% (1)

Explanation

DataFrame.take(n) is an action because it triggers execution of the Spark DAG and returns actual data to the driver - specifically, an array of the first n Row objects. Actions are what force Spark to leave lazy evaluation mode and compute results.

The distractors are all transformations, which only build up the logical execution plan without moving data:

drop() - removes columns, returns a new DataFrame
coalesce() - reduces partitions, returns a new DataFrame
join() - combines DataFrames, returns a new DataFrame
filter() - subsets rows, returns a new DataFrame

Memory tip: If the operation returns a DataFrame, it's a transformation. If it returns something else (a list, a count, None, writes to disk), it's an action. take() returns a Python list - that's your giveaway.

Topics

#actions vs transformations#DataFrame API#lazy evaluation#Spark fundamentals

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice