Which of the following pairs of arguments cannot be used in DataFrame.join() to perform an inner join on two DataFrames, named and aliased with "a" and "b" respectively, to specify two key columns?

Question

Options

• Aon = [a.column1 == b.column1, a.column2 == b.column2]
• Bon = [col("column1"), col("column2")]
• Con = [col("a.column1") == col("b.column1"), col("a.column2") == col("b.column2")]
• DAll of these options can be used to perform an inner join with two key columns.
• Eon = ["column1", "column2"]

• B
  88% (23)
• D
  8% (2)
• E
  4% (1)

Explanation

Passing on = [col("column1"), col("column2")] - a list of unqualified Column objects without equality expressions - is invalid for a join when both DataFrames contain columns with the same names, because Spark cannot determine which DataFrame each column belongs to and will raise an AnalysisException for ambiguous references. The correct alternatives are: a list of strings like ["column1", "column2"] (E), a list of equality expressions using aliased DataFrames like [a.column1 == b.column1, ...] (A), or qualified column expressions using alias-dot notation like [col("a.column1") == col("b.column1"), ...] (C).

No community discussion yet for this question.