The code block shown below contains an error. The code block is intended to return a new 12- partition DataFrame from the 8-partition DataFrame storesDF by inducing a shuffle. Identify the error…

Question

Options

• AThe coalesce() operation cannot guarantee the number of target partitions - the repartition()
• BThe coalesce() operation does not induce a shuffle and cannot increase the number of partitions -
• CThe coalesce() operation will only work if the DataFrame has been cached to memory - the
• DThe coalesce() operation requires a column by which to partition rather than a number of
• EThe number of resulting partitions, 12, is not achievable for an 8-partition DataFrame.

• A
  7% (2)
• B
  90% (27)
• D
  3% (1)

Explanation

coalesce() is a narrow transformation that can only reduce the number of partitions by merging adjacent partitions-it cannot increase the partition count. Going from 8 partitions to 12 requires adding partitions, which mandates a shuffle. repartition(12) performs a full shuffle and can both increase and decrease the number of partitions. Since the intent is to go from 8 to 12 partitions via a shuffle, coalesce(12) is the wrong operation.

No community discussion yet for this question.