nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #32

The code block shown below contains an error. The code block intended to return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column…

The correct answer is E. The column key is the second parameter to join() and the type of join in the third parameter to. The PySpark DataFrame.join() signature is join(other, on=None, how=None). The on parameter (key column) is the second argument and the how parameter (join type) is the third. In the erroneous code StoresDF.join(employeesDF, "inner", "storeID"), the arguments are reversed…

Spark DataFrame Transformations

Question

The code block shown below contains an error. The code block intended to return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId. Identify the error. Code block:

StoresDF.join(employeesDF, "inner", "storeID")

Options

  • AThe key column storeID needs to be wrapped in the col() operation.
  • BThe key column storeID needs to be in a list like ["storeID"].
  • CThe key column storeID needs to be specified in an expression of both DataFrame columns like
  • DThere is no DataFrame.join() operation - DataFrame.merge() should be used instead.
  • EThe column key is the second parameter to join() and the type of join in the third parameter to

How the community answered

(16 responses)
  • A
    6% (1)
  • D
    6% (1)
  • E
    88% (14)

Explanation

The PySpark DataFrame.join() signature is join(other, on=None, how=None). The on parameter (key column) is the second argument and the how parameter (join type) is the third. In the erroneous code StoresDF.join(employeesDF, "inner", "storeID"), the arguments are reversed: "inner" is passed as the key column and "storeID" is passed as the join type. The correct call should be storesDF.join(employeesDF, "storeID", "inner"). The capitalization issue (StoresDF) is secondary; the core error is the swapped parameter order.

Topics

#Spark DataFrames#Join operations#PySpark API#Error identification

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice