DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #77
The code block shown below contains an error. The code block is intended to create and register a SQL UDF named "ASSESS_PERFORMANCE" using the Scala function assessPerformance() and apply it to…
The correct answer is E. There is no sql() operation - the DataFrame API must be used to apply the UDF. The error is a name mismatch between the registered UDF and the SQL call. The UDF is registered under the name "ASSESS_PERFORMANCE" (uppercase), but the SQL statement calls assessPerformance(customerSatisfaction) (camelCase). Even though Spark SQL is generally case-insensitive…
Question
The code block shown below contains an error. The code block is intended to create and register a SQL UDF named "ASSESS_PERFORMANCE" using the Scala function assessPerformance() and apply it to column customerSatisfaction in the table stores. Identify the error. Code block:
spark.udf.register("ASSESS_PERFORMANCE", assessPerforance) spark.sql("SELECT customerSatisfaction, assessPerformance(customerSatisfaction) AS result FROM stores")
Options
- AThe customerSatisfaction column cannot be called twice inside the SQL statement.
- BRegistered UDFs cannot be applied inside of a SQL statement.
- CThe order of the arguments to spark.udf.register() should be reversed.
- DThe wrong SQL function is used to compute column result - it should be
- EThere is no sql() operation - the DataFrame API must be used to apply the UDF
How the community answered
(53 responses)- A15% (8)
- B2% (1)
- C8% (4)
- D2% (1)
- E74% (39)
Explanation
The error is a name mismatch between the registered UDF and the SQL call. The UDF is registered under the name "ASSESS_PERFORMANCE" (uppercase), but the SQL statement calls assessPerformance(customerSatisfaction) (camelCase). Even though Spark SQL is generally case-insensitive for built-in functions, custom UDF names must match exactly as registered. The SQL query should reference ASSESS_PERFORMANCE(customerSatisfaction) to correctly invoke the registered UDF. The spark.sql() operation itself is valid for calling registered UDFs.
Topics
Community Discussion
No community discussion yet for this question.