DP-203 · Question #403
Drag and Drop Question You have an Azure Databricks deployment and a local file named /tmp/file1 that contains the following code. You need to read /tmp/file1 into a data frame by using Scala. How…
The correct answer is Multiline; json. When reading a JSON file that spans multiple lines (as indicated by the file containing structured JSON data), you must use the 'Multiline' option set to true to tell Spark that a single JSON record can span multiple lines. The format must be specified as 'json' because the…
Question
Answer Area
Drag items
Correct arrangement
- Multiline
- json
Explanation
When reading a JSON file that spans multiple lines (as indicated by the file containing structured JSON data), you must use the 'Multiline' option set to true to tell Spark that a single JSON record can span multiple lines. The format must be specified as 'json' because the file contains JSON data and Spark needs to know which reader/parser to apply. Together, spark.read.option('Multiline', 'true').json('/tmp/file1') correctly reads a multi-line JSON file into a DataFrame in Scala on Azure Databricks.
Topics
Community Discussion
No community discussion yet for this question.