A data engineer has developed a data pipeline to ingest data from a JSON source using Auto Loader, but the engineer has not provided any type inference or schema hints in their pipeline. Upon…

Question

Options

• AThere was a type mismatch between the specific schema and the inferred schema
• BJSON data is a text-based format
• CAuto Loader only works with string data
• DAll of the fields had at least one null value
• EAuto Loader cannot infer the schema of ingested data

• B
  94% (17)
• D
  6% (1)

Explanation

JSON is a text-based, human-readable format. While JSON values can semantically represent booleans, numbers, and nulls, they are ultimately stored as character sequences. When Auto Loader processes JSON files without schema hints or the cloudFiles.inferColumnTypes option enabled, it conservatively defaults to representing all fields as strings - because reading them as strings is always safe and lossless, whereas assuming a numeric or boolean type could cause errors if any record deviates. Auto Loader is fully capable of schema inference and does work with non-string data; the root cause here is specifically the text-based nature of JSON and the absence of explicit type inference configuration.

No community discussion yet for this question.