nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #147

The code block shown below contains an error. The code block is intended to return a DataFrame containing all columns from DataFrame storesDF except for column sqft and column customerSatisfaction…

The correct answer is D. The sqft and customerSatisfaction column names should be quoted like "sqft" and. In Python (PySpark), when you write sqft and customerSatisfaction without quotes, the interpreter treats them as variable names, not string literals. If those variables are not defined in scope, Python raises a NameError. Column names passed to drop() must be quoted string…

Performing DataFrame Transformations

Question

The code block shown below contains an error. The code block is intended to return a DataFrame containing all columns from DataFrame storesDF except for column sqft and column customerSatisfaction. Identify the error. Code block:

storesDF.drop(sqft, customerSatisfaction)

Options

  • AThe drop() operation only works if one column name is called at a time - there should be two calls
  • BThe drop() operation only works if column names are wrapped inside the col() function like
  • CThere is no drop() operation for storesDF.
  • DThe sqft and customerSatisfaction column names should be quoted like "sqft" and
  • EThe sqft and customerSatisfaction column names should be subset from the DataFrame

How the community answered

(35 responses)
  • A
    3% (1)
  • B
    6% (2)
  • D
    89% (31)
  • E
    3% (1)

Explanation

In Python (PySpark), when you write sqft and customerSatisfaction without quotes, the interpreter treats them as variable names, not string literals. If those variables are not defined in scope, Python raises a NameError. Column names passed to drop() must be quoted string literals: storesDF.drop('sqft', 'customerSatisfaction'). Spark's drop() does accept multiple column names in a single call, so there is no need for separate calls. Wrapping in col() is also valid but not required for drop().

Topics

#Spark DataFrame API#Column Operations#PySpark Syntax#DataFrame Transformations

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice