Which of the following code blocks fails to return a new DataFrame that is the result of an inner join between DataFrame storesDF and DataFrame employeesDF on column storeId and column employeeId?

Question

Options

• AstoresDF.join(employeesDF, Seq(col("storeId"), col("employeeId")))
• BstoresDF.join(employeesDF, Seq("storeId", "employeeId"))
• CstoresDF.join(employeesDF, storesDF("storeId") === employeesDF("storeId") and
• DstoresDF.join(employeesDF, Seq("storeId", "employeeId"), "inner")
• EstoresDF.alias("s").join(employeesDF.alias("e"), col("s.storeId") === col("e.storeId") and

• A
  74% (35)
• B
  15% (7)
• C
  6% (3)
• D
  2% (1)
• E
  2% (1)

Explanation

Option A fails because Seq(col("storeId"), col("employeeId")) passes a Seq[Column] to the join method, but the join overload that accepts a Seq requires Seq[String]. Spark has no join overload for Seq[Column], so this results in a compile-time type mismatch - you cannot mix col() objects inside a Seq when using the "using columns" join form.

Why the distractors are valid:

• B correctly passes Seq[String], which matches the join(right, Seq[String]) overload and defaults to an inner join.
• C uses a Column expression (Boolean expression via === and and), which matches the join(right, joinExprs: Column) overload.
• D is the most explicit form - Seq[String] plus the join type "inner" - and is perfectly valid.
• E uses DataFrame aliases with col("alias.column") notation in a Column expression, which is also valid syntax.

Memory tip: Think "Seq = Strings Only." If you use Seq() in a Spark join, every element must be a plain string column name - the moment you wrap a column in col() inside a Seq, you've broken the type contract. Reserve col() for the direct Column-expression form of join (no Seq wrapper).

No community discussion yet for this question.