nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #153

Which of the following code blocks returns a new DataFrame with column storeReview where the pattern "End" has been removed from the end of column storeReview in DataFrame storesDF? A sample…

The correct answer is B. storesDF.withColumn("storeReview", regexp_replace(col("storeReview"), " End$", "")). regexp_replace is a standalone function from pyspark.sql.functions - it is NOT a method chained off a Column object (eliminating A). It requires exactly three arguments: the column expression, the regex pattern, and the replacement string (eliminating C which omits the…

Working with Spark DataFrames

Question

Which of the following code blocks returns a new DataFrame with column storeReview where the pattern "End" has been removed from the end of column storeReview in DataFrame storesDF? A sample DataFrame storesDF is below:

Options

  • AstoresDF.withColumn("storeReview", col("storeReview").regexp_replace(" End$", ""))
  • BstoresDF.withColumn("storeReview", regexp_replace(col("storeReview"), " End$", ""))
  • CstoresDF.withColumn("storeReview? regexp_replace(col("storeReview"), " End$"))
  • DstoresDF.withColumn("storeReview", regexp_replace("storeReview", " End$", ""))
  • EstoresDF.withColumn("storeReview", regexp_extract(col("storeReview"), " End$", ""))

How the community answered

(22 responses)
  • B
    73% (16)
  • C
    5% (1)
  • D
    14% (3)
  • E
    9% (2)

Explanation

regexp_replace is a standalone function from pyspark.sql.functions - it is NOT a method chained off a Column object (eliminating A). It requires exactly three arguments: the column expression, the regex pattern, and the replacement string (eliminating C which omits the replacement). Passing a plain string 'storeReview' instead of col('storeReview') as the first argument is incorrect (eliminating D). regexp_extract is for capturing matches, not removing them (eliminating E). Option B is correct: regexp_replace(col('storeReview'), ' End$', '') uses the '$' anchor to match only at the end of the string and replaces it with an empty string.

Topics

#Spark DataFrames#Column Functions#Regular Expressions#String Manipulation

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice