In what order should the below lines of code be run in order to write DataFrame storesDF to file path filePath as parquet and partition by values in column division? Lines of code: 1. .write() \ 2…

The correct answer is C. 4, 6, 2, 3. The correct sequence is lines 4, 6, 2, 3: storesDF → .write → .partitionBy("division") → .parquet(filePath). write is a property (not a method), so line 1 (.write() with parentheses) is invalid. Line 5 (.repartition("division")) physically repartitions data in memory/cluster…

Writing DataFrames to Storage

Question

In what order should the below lines of code be run in order to write DataFrame storesDF to file path filePath as parquet and partition by values in column division? Lines of code: 1. .write() \ 2. .partitionBy("division") \ 3. .parquet(filePath) 4. .storesDF \ 5. .repartition("division") 6. .write \ 7. .path(filePath, "parquet")

Options

A4, 1, 2, 3
B4, 1, 5, 7
C4, 6, 2, 3
D4, 1, 5, 3
E4, 6, 2, 7

How the community answered

(46 responses)

A
7% (3)
B
2% (1)
C
89% (41)
E
2% (1)

Explanation

The correct sequence is lines 4, 6, 2, 3: storesDF → .write → .partitionBy("division") → .parquet(filePath). write is a property (not a method), so line 1 (.write() with parentheses) is invalid. Line 5 (.repartition("division")) physically repartitions data in memory/cluster but does not create on-disk partition directories - .partitionBy() on the DataFrameWriter is required for that. Line 7 (.path(filePath, "parquet")) does not exist as a valid DataFrameWriter method. Only option C assembles the correct chain using the property form of write.

Topics

#Spark DataFrame API#Data Writing#Parquet Format#Data Partitioning

Community Discussion

No community discussion yet for this question.

Full DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK Practice