nerdexam
Databricks

DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #48

Which of the following code blocks returns a DataFrame containing only the rows from DataFrame storesDF where the value in column sqft is less than or equal to 25,000 OR the value in column…

The correct answer is B. storesDF.filter(col("sqft") <= 25000 | col("customerSatisfaction") >= 30). In PySpark, the bitwise OR operator | must be used to combine Column expressions with logical OR. Python's built-in or keyword short-circuits and returns one of the operands rather than combining Column expressions, so options A and E using 'and'/'or' will not produce the…

Performing Data Transformations with Spark DataFrames

Question

Which of the following code blocks returns a DataFrame containing only the rows from DataFrame storesDF where the value in column sqft is less than or equal to 25,000 OR the value in column customerSatisfaction is greater than or equal to 30?

Options

  • AstoresDF.filter(col("sqft") <= 25000 and col("customerSatisfaction") >= 30)
  • BstoresDF.filter(col("sqft") <= 25000 | col("customerSatisfaction") >= 30)
  • CstoresDF.filter(col(sqft) <= 25000 or col(customerSatisfaction) >= 30)
  • DstoresDF.filter(sqft <= 25000 | customerSatisfaction >= 30)
  • EstoresDF.filter(col("sqft") <= 25000 or col("customerSatisfaction") >= 30)

How the community answered

(16 responses)
  • A
    6% (1)
  • B
    88% (14)
  • E
    6% (1)

Explanation

In PySpark, the bitwise OR operator | must be used to combine Column expressions with logical OR. Python's built-in or keyword short-circuits and returns one of the operands rather than combining Column expressions, so options A and E using 'and'/'or' will not produce the correct boolean Column. Option B - storesDF.filter(col('sqft') <= 25000 | col('customerSatisfaction') >= 30) - correctly uses | between two Column conditions. Option C omits quotes around column names. Option D omits col() wrappers.

Topics

#PySpark DataFrames#DataFrame Filtering#Logical Operators#Column Expressions

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice