The code block shown below should return a new DataFrame where column product補tegories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF. Choose the…

The correct answer is E. 1. withColumn. The goal is to explode a string column into one word per row, which requires: (1) withColumn - the correct PySpark method to add/replace a column; (2) the column name string 'productCategories'; (3) explode - which converts each array element into a separate row; (4) split…

Spark DataFrame Transformations

Question

The code block shown below should return a new DataFrame where column product補tegories only has one word per row, resulting in a DataFrame with many more rows than DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task. A sample of storesDF is displayed below:

Code block:

storesDF.1(2, 3(4(5)))

Options

A
1. newColumn
B
1. withColumn
C
1. withColumn
D
1. newColumn
E
1. withColumn

How the community answered

(50 responses)

B
4% (2)
C
6% (3)
D
2% (1)
E
88% (44)

Explanation

The goal is to explode a string column into one word per row, which requires: (1) withColumn - the correct PySpark method to add/replace a column; (2) the column name string 'productCategories'; (3) explode - which converts each array element into a separate row; (4) split - which splits the string into an array of words; and (5) col('productCategories') plus a delimiter (e.g., ' '). The full expression is: storesDF.withColumn('productCategories', explode(split(col('productCategories'), ' '))). The 'newColumn' variant (seen in wrong choices) does not exist in the PySpark API.

Topics

#Spark DataFrame API#DataFrame Transformations#explode function#Column Operations

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice