nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #54

Which of the following code blocks returns a new DataFrame where column sqft from DataFrame storesDF has had its missing values replaced with the value 30,000? A sample of DataFrame storesDF is below:

The correct answer is E. storesDF.na.fill(30000, "sqft"). In PySpark, DataFrame.na.fill(value, subset) fills null/NaN values. The subset parameter accepts a string column name or a list of strings. Option E - storesDF.na.fill(30000, 'sqft') - correctly passes the fill value and the column name as a string. Option A uses Seq('sqft')…

Handling Missing Data in Spark DataFrames

Question

Which of the following code blocks returns a new DataFrame where column sqft from DataFrame storesDF has had its missing values replaced with the value 30,000? A sample of DataFrame storesDF is below:

Options

  • AstoresDF.na.fill(30000, Seq("sqft"))
  • BstoresDF.nafill(30000, col("sqft"))
  • CstoresDF.na.fill(30000, col("sqft"))
  • DstoresDF.fillna(30000, col("sqft"))
  • EstoresDF.na.fill(30000, "sqft")

How the community answered

(25 responses)
  • B
    4% (1)
  • C
    4% (1)
  • E
    92% (23)

Explanation

In PySpark, DataFrame.na.fill(value, subset) fills null/NaN values. The subset parameter accepts a string column name or a list of strings. Option E - storesDF.na.fill(30000, 'sqft') - correctly passes the fill value and the column name as a string. Option A uses Seq('sqft'), which is Scala syntax (not valid in Python). Option B uses a non-existent method nafill. Option C passes a Column object col('sqft') as the subset, which is invalid - it must be a string or list of strings. Option D uses fillna (a valid alias) but also incorrectly passes a Column object rather than a string.

Topics

#Spark DataFrame API#Missing Values#Data Cleaning#na.fill

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice