DATABRICKS-CERTIFIED-ASSOCIATE-DEVELOPER-FOR-APACHE-SPARK · Question #166
The code block shown below contains an error. The code block is intended to read JSON at the file path filePath into a DataFrame with the specified schema schema. Identify the error. Code block…
The correct answer is A. The schema operation from read takes a schema object rather than a string - the argument should. The error is that DataFrameReader.schema() requires a StructType object (or a DDL-formatted string representation), not a plain variable name passed as a string literal "schema". The correct usage is spark.read.schema(schema).format('json').load(filePath) where schema is a…
Question
The code block shown below contains an error. The code block is intended to read JSON at the file path filePath into a DataFrame with the specified schema schema. Identify the error. Code block:
spark.read.schema("schema").format("json").load(filePath)
Options
- AThe schema operation from read takes a schema object rather than a string - the argument should
- BThere is no load() operation for DataFrameReader - it should be replaced with the json() operation.
- CThe spark.read operation should be followed by parentheses in order to return a
- DThere is no read property of spark - spark should be replaced with DataFrame.
- EThe schema operation from read takes a column rather than a string - the argument should be
How the community answered
(29 responses)- A93% (27)
- B3% (1)
- D3% (1)
Explanation
The error is that DataFrameReader.schema() requires a StructType object (or a DDL-formatted string representation), not a plain variable name passed as a string literal "schema". The correct usage is spark.read.schema(schema).format('json').load(filePath) where schema is a previously defined StructType object. Passing the string "schema" would either raise an error or be misinterpreted as a DDL string. Options B, C, D, and E are all incorrect: load() is a valid DataFrameReader method, spark.read is a valid property (no parentheses needed), and spark is the correct SparkSession entry point.
Topics
Community Discussion
No community discussion yet for this question.