nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #4

Which of the following code blocks returns a new DataFrame with a new column employeesPerSqft that is the quotient of column numberOfEmployees and column sqft, both of which are from DataFrame…

The correct answer is A. storesDF.withColumn("employeesPerSqft", col("numberOfEmployees") / col("sqft")). withColumn(name, expression) is the correct method to add or replace a column in a DataFrame. The first argument must be a string (the new column name), and the second must be a Column expression. Option A - storesDF.withColumn("employeesPerSqft", col("numberOfEmployees") /…

Data Manipulation with Spark DataFrames

Question

Which of the following code blocks returns a new DataFrame with a new column employeesPerSqft that is the quotient of column numberOfEmployees and column sqft, both of which are from DataFrame storesDF? Note that column employeesPerSqft is not in the original DataFrame storesDF.

Options

  • AstoresDF.withColumn("employeesPerSqft", col("numberOfEmployees") / col("sqft"))
  • BstoresDF.withColumn("employeesPerSqft", "numberOfEmployees" / "sqft")
  • CstoresDF.select("employeesPerSqft", "numberOfEmployees" / "sqft")
  • DstoresDF.select("employeesPerSqft", col("numberOfEmployees") / col("sqft"))
  • EstoresDF.withColumn(col("employeesPerSqft"), col("numberOfEmployees") / col("sqft"))

How the community answered

(40 responses)
  • A
    90% (36)
  • B
    5% (2)
  • D
    3% (1)
  • E
    3% (1)

Explanation

withColumn(name, expression) is the correct method to add or replace a column in a DataFrame. The first argument must be a string (the new column name), and the second must be a Column expression. Option A - storesDF.withColumn("employeesPerSqft", col("numberOfEmployees") / col("sqft")) - satisfies both requirements. Option B fails because Python string literals cannot be divided. Options C and D use select(), which doesn't add a column to an existing DataFrame the way described - it would require an alias and would drop all other columns. Option E passes a col() object as the first argument to withColumn, but that parameter must be a plain string.

Topics

#DataFrame Transformations#Adding Columns#Spark SQL Functions#Column Operations

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice