DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #144
Which of the following operations can be used to create a new DataFrame that has 12 partitions from an original DataFrame df that has 8 partitions?
The correct answer is A. df.repartition(12). repartition(n) performs a full shuffle of the data and can both increase and decrease the number of partitions to any target value. Since you need to go from 8 to 12 partitions (an increase), repartition(12) is the correct choice. coalesce(n), by contrast, can only reduce the…
Question
Which of the following operations can be used to create a new DataFrame that has 12 partitions from an original DataFrame df that has 8 partitions?
Options
- Adf.repartition(12)
- Bdf.cache()
- Cdf.partitionBy(1.5)
- Ddf.coalesce(12)
- Edf.partitionBy(12)
How the community answered
(43 responses)- A91% (39)
- C2% (1)
- D2% (1)
- E5% (2)
Explanation
repartition(n) performs a full shuffle of the data and can both increase and decrease the number of partitions to any target value. Since you need to go from 8 to 12 partitions (an increase), repartition(12) is the correct choice. coalesce(n), by contrast, can only reduce the number of partitions and avoids a full shuffle by merging existing partitions; calling coalesce(12) on an 8-partition DataFrame would have no effect since you cannot coalesce upward. partitionBy() is a write operation used with DataFrameWriter, not a transformation on a DataFrame object. df.cache() stores data in memory but does not change partition count.
Topics
Community Discussion
No community discussion yet for this question.