The code block shown below should return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId and column employeeId. Choose the…

The correct answer is E. 1. storesDF.storeId. When joining two DataFrames on conditions that may involve columns with the same name in both DataFrames (causing ambiguity), you must use DataFrame-qualified column references such as storesDF.storeId and employeesDF.employeeId. The correct fill is: storesDF.join(employeesDF…

Transforming Data with Spark DataFrames

Question

The code block shown below should return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId and column employeeId. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. Code block:

storesDF.join(employeesDF, [1 == 2, 3 == 4])

Options

A
1. storesDF.storeId
B
1. col("storeId")
C
1. storeId
D
1. col("storeId")
E
1. storesDF.storeId

How the community answered

(35 responses)

B
6% (2)
C
3% (1)
E
91% (32)

Explanation

When joining two DataFrames on conditions that may involve columns with the same name in both DataFrames (causing ambiguity), you must use DataFrame-qualified column references such as storesDF.storeId and employeesDF.employeeId. The correct fill is: storesDF.join(employeesDF, [storesDF.storeId == employeesDF.storeId, storesDF.employeeId == employeesDF.employeeId]). Option B uses col('storeId') which is ambiguous when both DataFrames have a column with the same name. Option C uses unquoted bare names which are not valid Python expressions. Option E correctly disambiguates column references by prefixing them with the DataFrame object.

Topics

#Spark DataFrames#Joins#DataFrame API#Column Expressions

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice