Which of the following code blocks returns a DataFrame containing only the rows from DataFrame storesDF where the value in column sqft is less than or equal to 25,000 AND the value in column…

Question

Options

• AstoresDF.filter(col("sqft") <= 25000 and col("customerSatisfaction") >= 30)
• BstoresDF.filter(col("sqft") <= 25000 or col("customerSatisfaction") >= 30)
• CstoresDF.filter(sqft) <= 25000 and customerSatisfaction >= 30)
• DstoresDF.filter(col("sqft") <= 25000 & col("customerSatisfaction") >= 30)
• EstoresDF.filter(sqft <= 25000) & customerSatisfaction >= 30)

• A
  6% (1)
• B
  6% (1)
• D
  89% (16)

Explanation

Option D is correct because PySpark's filter() method requires the bitwise & operator to combine Column expressions - Python's built-in and keyword cannot evaluate the truthiness of two Column objects element-wise and will raise a ValueError or produce incorrect results.

Why the distractors fail:

• A & B use Python's and/or keywords on Column objects, which PySpark cannot evaluate as intended - and/or work on Python booleans, not distributed Column expressions.
• B also uses or instead of and, which would return rows matching either condition, not both.
• C has mismatched parentheses and references sqft/customerSatisfaction as bare Python variables rather than wrapping them in col() - this would raise a NameError.
• E closes the filter() call too early (after sqft <= 25000), applies & to the returned DataFrame object rather than the column condition, and also uses bare variable names without col().

Memory tip: In PySpark, think "columns get symbols, not words" - use & for AND, | for OR, and ~ for NOT when chaining col() expressions, and always wrap each condition in parentheses to avoid Python's operator precedence surprises (e.g., (col("sqft") <= 25000) & (col("customerSatisfaction") >= 30)).

No community discussion yet for this question.