DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #173
Which of the following code blocks attempts to cache the partitions of DataFrame storesDF only in Spark's memory?
The correct answer is D. storesDF.persist(StorageLevel.MEMORY_ONLY).count(). In PySpark, persist(StorageLevel.MEMORY_ONLY) explicitly caches DataFrame partitions in memory only. cache() (choice C) is equivalent to persist(StorageLevel.MEMORY_AND_DISK) for DataFrames in Spark 2.0+, so it does not restrict caching to memory only. persist() with no…
Question
Which of the following code blocks attempts to cache the partitions of DataFrame storesDF only in Spark's memory?
Options
- AstoresDF.cache(StorageLevel.MEMORY_ONLY).count()
- BstoresDF.persist().count()
- CstoresDF.cache().count()
- DstoresDF.persist(StorageLevel.MEMORY_ONLY).count()
- EstoresDF.persist("MEMORY_ONLY").count()
How the community answered
(41 responses)- B2% (1)
- C5% (2)
- D90% (37)
- E2% (1)
Explanation
In PySpark, persist(StorageLevel.MEMORY_ONLY) explicitly caches DataFrame partitions in memory only. cache() (choice C) is equivalent to persist(StorageLevel.MEMORY_AND_DISK) for DataFrames in Spark 2.0+, so it does not restrict caching to memory only. persist() with no arguments (choice B) also defaults to MEMORY_AND_DISK. Choice E passes a string instead of a StorageLevel object, which is invalid syntax. The .count() action is needed to materialize and actually trigger the caching.
Topics
Community Discussion
No community discussion yet for this question.