Which of the following storage levels should be used to store as much data as possible in memory on two cluster nodes while storing any data that does not fit in memory on disk to be read in when…

The correct answer is D. MEMORY_AND_DISK_2. MEMORY_AND_DISK_2 satisfies all three requirements: (1) it stores data in memory first, (2) it spills to disk when memory is insufficient, and (3) the '_2' suffix means the data is replicated across two cluster nodes. MEMORY_AND_DISK would use memory and disk but only on one…

Data Persistence and Caching

Question

Which of the following storage levels should be used to store as much data as possible in memory on two cluster nodes while storing any data that does not fit in memory on disk to be read in when needed?

Options

AMEMORY_ONLY_2
BMEMORY_AND_DISK_SER
CMEMORY_AND_DISK
DMEMORY_AND_DISK_2
EMEMORY_ONLY

How the community answered

(25 responses)

A
4% (1)
B
4% (1)
D
80% (20)
E
12% (3)

Explanation

MEMORY_AND_DISK_2 satisfies all three requirements: (1) it stores data in memory first, (2) it spills to disk when memory is insufficient, and (3) the '_2' suffix means the data is replicated across two cluster nodes. MEMORY_AND_DISK would use memory and disk but only on one node. MEMORY_ONLY_2 replicates on two nodes but has no disk fallback. MEMORY_AND_DISK_SER serializes the data (more compact but slower to read) and only on one node. MEMORY_ONLY uses only memory on one node with no replication or disk fallback.

Topics

#Spark Storage Levels#RDD Persistence#Data Caching#Fault Tolerance

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice