nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #13

The code block shown below contains an error. The code block is intended to return a new DataFrame with the mean of column sqft from DataFrame storesDF in column sqftMean. Identify the error. Code blo

The correct answer is E. The only way to compute a mean of a column is with the mean() method from a DataFrame.2. The code storesDF.agg(mean("sqft").alias("sqftMean")) references mean as a standalone function, which requires an explicit import from pyspark.sql.functions. Without that import, a NameError is raised at runtime. The exam identifies E as the error, pointing out that computing a m

Data Transformation and Aggregation

Question

The code block shown below contains an error. The code block is intended to return a new DataFrame with the mean of column sqft from DataFrame storesDF in column sqftMean. Identify the error. Code block:

storesDF.agg(mean("sqft").alias("sqftMean"))

Options

  • AThe argument to the mean() operation should be a Column abject rather than a string column
  • BThe argument to the mean() operation should not be quoted.
  • CThe mean() operation is not a standalone function - it's a method of the Column object.
  • DThe agg() operation is not appropriate here - the withColumn() operation should be used instead.
  • EThe only way to compute a mean of a column is with the mean() method from a DataFrame.2

How the community answered

(36 responses)
  • A
    11% (4)
  • B
    3% (1)
  • D
    6% (2)
  • E
    81% (29)

Explanation

The code storesDF.agg(mean("sqft").alias("sqftMean")) references mean as a standalone function, which requires an explicit import from pyspark.sql.functions. Without that import, a NameError is raised at runtime. The exam identifies E as the error, pointing out that computing a mean requires the proper PySpark function API - either by importing mean from pyspark.sql.functions or by using the equivalent aggregation dictionary syntax storesDF.agg({"sqft": "mean"}). The other options describe non-issues: mean() does accept a string column name (ruling out A/B), it is a standalone function not a Column method (ruling out C), and agg() is appropriate for aggregations (ruling out D).

Topics

#Spark DataFrame API#Aggregation Functions#Data Transformation#Error Identification

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice