nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #8

Which of the following code blocks returns a new DataFrame with column storeDescription where the pattern "Description: " has been removed from the beginning of column storeDescription in DataFrame…

The correct answer is E. storesDF.withColumn("storeDescription", regexp_replace(col("storeDescription"), "^Description: ". regexp_replace(column, pattern, replacement) is the correct PySpark function to replace a regex pattern with a string. Its first argument must be a Column object (via col()), the second is the regex pattern, and the third is the replacement string (empty string "" to delete the…

Transforming Data with DataFrames

Question

Which of the following code blocks returns a new DataFrame with column storeDescription where the pattern "Description: " has been removed from the beginning of column storeDescription in DataFrame storesDF? A sample of DataFrame storesDF is below:

Options

  • AstoresDF.withColumn("storeDescription", regexp_replace(col("storeDescription"), "^Description:
  • BstoresDF.withColumn("storeDescription", col("storeDescription").regexp_replace("^Description: ",
  • CstoresDF.withColumn("storeDescription", regexp_extract(col("storeDescription"), "^Description: ",
  • DstoresDF.withColumn("storeDescription", regexp_replace("storeDescription", "^Description: ", ""))
  • EstoresDF.withColumn("storeDescription", regexp_replace(col("storeDescription"), "^Description: ",

How the community answered

(38 responses)
  • A
    3% (1)
  • B
    11% (4)
  • C
    5% (2)
  • D
    3% (1)
  • E
    79% (30)

Explanation

regexp_replace(column, pattern, replacement) is the correct PySpark function to replace a regex pattern with a string. Its first argument must be a Column object (via col()), the second is the regex pattern, and the third is the replacement string (empty string "" to delete the match). Option E - regexp_replace(col("storeDescription"), "^Description: ", "") - is correct. Option C uses regexp_extract, which captures a match rather than removing it. Option D passes a plain string instead of a col() object as the first argument. Option B incorrectly calls .regexp_replace() as a method on a Column object, which is not valid in PySpark's Python API. Option A appears truncated but also lacks the replacement argument.

Topics

#Spark DataFrame API#Column operations#String manipulation#Regular expressions

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice