DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #78
The code block shown below contains an error. The code block is intended to create the Scala UDF assessPerformanceUDF() and apply it to the integer column customers1t1sfaction in Data Frame…
The correct answer is B. The return type of assessPerformanceUDF() must be specified. When defining a UDF in Spark Scala using the udf() function, the return type of the UDF must be explicitly specified. While Scala can often infer types, Spark's udf() wrapper requires a declared return type so it can serialize/deserialize values correctly across the distributed…
Question
The code block shown below contains an error. The code block is intended to create the Scala UDF assessPerformanceUDF() and apply it to the integer column customers1t1sfaction in Data Frame storesDF. Identify the error. Code block:
Exhibit
Options
- AThe input type of customerSatisfaction is not specified in the udf() operation.
- BThe return type of assessPerformanceUDF() must be specified.
- CThe withColumn() operation is not appropriate here - UDFs should be applied by iterating over
- DThe assessPerformanceUDF() must first be defined as a Scala function and then converted to a
- EUDFs can only be applied via SQL and not through the Data Frame API.
How the community answered
(34 responses)- A3% (1)
- B94% (32)
- E3% (1)
Explanation
When defining a UDF in Spark Scala using the udf() function, the return type of the UDF must be explicitly specified. While Scala can often infer types, Spark's udf() wrapper requires a declared return type so it can serialize/deserialize values correctly across the distributed execution environment. Without specifying the return type (e.g., IntegerType or StringType), the UDF definition is incomplete and Spark cannot properly encode the output. The fix is to add the return type as a type parameter or additional argument to the udf() call.
Topics
Community Discussion
No community discussion yet for this question.