The code block shown below contains an error. The code block is intended to return a new DataFrame that is the result of a left join between DataFrame storesDF and DataFrame employeesDF on column…

Question

Options

• AThe key column storeId needs to be in a list like ["storeId"].
• BThe key column storeId needs to be wrapped in the col() operation.
• CThere is no DataFrame.join() operation - DataFrame.merge() should be used instead.
• DThe default argument to the how parameter is "inner" - an additional argument of "1eft" must be
• EThe key column storeId needs to be specified in an expression of both DataFrame columns like

• A
  3% (1)
• C
  3% (1)
• D
  88% (30)
• E
  6% (2)

Explanation

In PySpark, the DataFrame.join() method has a third parameter called how that specifies the join type. Its default value is "inner", meaning that if you omit the how argument, PySpark will perform an inner join - not a left join. Since the intent of the code is to perform a left join, the missing argument causes the wrong type of join to be executed silently with no error message.

The corrected code should be:

storesDF.join(employeesDF, "storeId", "left")

Why the other options are wrong:

• A is incorrect: PySpark accepts a single column name as a string for the join key - wrapping it in a list ["storeId"] is valid but not required, and is not the bug here.
• B is incorrect: Using a plain string column name is perfectly valid in PySpark joins; col() is not required.
• C is incorrect: DataFrame.join() is a valid PySpark operation. DataFrame.merge() is a pandas API method, not standard PySpark.
• E is incorrect (as described): Specifying the key as a string is the idiomatic way to perform equi-joins on a shared column name in PySpark; there is no requirement to use a column expression from both DataFrames unless the column names differ between the two DataFrames.

No community discussion yet for this question.