Which of the following code blocks creates a Python UDF assessPerformanceUDF() using the integer-returning Python function assessPerformance() and applies it to Column customerSatisfaction in…

The correct answer is B. assessPerformanceUDF = udf(assessPerformance, IntegerType()) storesDF.withColumn("result". When registering a UDF with an explicit return type, the type must be an instantiated DataType object - IntegerType() with parentheses, not the uninstantiated class IntegerType (eliminating A). Options C and D omit the return type entirely, which causes Spark to default to…

Implementing User-Defined Functions (UDFs) in PySpark

Question

Which of the following code blocks creates a Python UDF assessPerformanceUDF() using the integer-returning Python function assessPerformance() and applies it to Column customerSatisfaction in DataFrame storesDF?

Options

AassessPerformanceUDF = udf(assessPerformance, IntegerType) storesDF.withColumn("result",
BassessPerformanceUDF = udf(assessPerformance, IntegerType()) storesDF.withColumn("result",
CassessPerformanceUDF - udf(assessPerformance) storesDF.withColumn("result",
DassessPerformanceUDF = udf(assessPerformance) storesDF.withColumn("result",
EassessPerformanceUDF = udf(assessPerformance, IntegerType()) storesDF.withColumn("result",

How the community answered

(43 responses)

A
7% (3)
B
88% (38)
C
2% (1)
D
2% (1)

Explanation

When registering a UDF with an explicit return type, the type must be an instantiated DataType object - IntegerType() with parentheses, not the uninstantiated class IntegerType (eliminating A). Options C and D omit the return type entirely, which causes Spark to default to StringType, producing wrong results for an integer-returning function. Option E is syntactically similar to B but may differ in how the UDF is applied. Option B is correct: udf(assessPerformance, IntegerType()) properly wraps the function with an instantiated IntegerType, then withColumn() applies it to the target column.

Topics

#PySpark UDFs#DataFrame Transformations#Spark SQL Data Types#Python for Spark

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice