Which of the following code blocks will always return a new 4-partition DataFrame from the 8- partition DataFrame storesDF without inducing a shuffle?

Question

Options

• AstoresDF.repartition(4, "sqft")
• BstoresDF.repartition()
• CstoresDF.coalesce(4)
• DstoresDF.repartition(4)
• EstoresDF.coalesce

• A
  8% (3)
• B
  3% (1)
• C
  83% (30)
• E
  6% (2)

Explanation

coalesce(4) reduces partitions from 8 to 4 by merging existing partitions on the same executor, avoiding a full network shuffle - it's a narrow transformation. Options A and D both use repartition, which always triggers a full shuffle regardless of whether you're increasing or decreasing partition count; adding a column like "sqft" in option A makes it even more clearly a shuffle. Option B (repartition() with no arguments) is invalid syntax and would throw an error at runtime. Option E (storesDF.coalesce without parentheses and no argument) never calls the method - it just references the function object and returns nothing useful, not a 4-partition DataFrame.

Memory tip: Think of coalesce as "collapse inward" - it squishes nearby partitions together without moving data across the network. repartition "re-deals the deck" from scratch, always shuffling. When reducing partitions, reach for coalesce; only use repartition when you need an even distribution or are increasing partitions.

No community discussion yet for this question.