nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #7

Which of the following code blocks returns a new DataFrame where column productCategories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF? A sample of…

The correct answer is A. storesDF.withColumn("productCategories", explode(col("productCategories"))). The explode() function from pyspark.sql.functions takes an array or map column and creates one new row per element, increasing the total row count. This is exactly the behavior described. Option A - storesDF.withColumn("productCategories", explode(col("productCategories")))…

Transforming DataFrames

Question

Which of the following code blocks returns a new DataFrame where column productCategories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF? A sample of storesDF is displayed below:

Options

  • AstoresDF.withColumn("productCategories", explode(col("productCategories")))
  • BstoresDF.withColumn("productCategories", split(col("productCategories")))
  • CstoresDF.withColumn("productCategories", col("productCategories").explode())
  • DstoresDF.withColumn("productCategories", col("productCategories").split())
  • EstoresDF.withColumn("productCategories", explode("productCategories"))

How the community answered

(62 responses)
  • A
    82% (51)
  • B
    10% (6)
  • C
    2% (1)
  • D
    5% (3)
  • E
    2% (1)

Explanation

The explode() function from pyspark.sql.functions takes an array or map column and creates one new row per element, increasing the total row count. This is exactly the behavior described. Option A - storesDF.withColumn("productCategories", explode(col("productCategories"))) - is correct. Option B uses split(), which converts a string to an array but does not expand rows. Options C and D incorrectly try to call .explode() or .split() as methods on a Column object - neither method exists in the Column API. Option E passes a plain string to explode() instead of a Column object; explode() requires a col() reference.

Topics

#Spark DataFrame API#explode function#withColumn#Data Transformation

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice