COF-C02 · Question #109
A table needs to be loaded. The input data is in JSON format and is a concatenation of multiple JSON documents. The file size is 3 GB. A warehouse size small is being used. The following COPY INTO…
The correct answer is B. Split the file into multiple files in the recommended size range (100 MB - 250 MB). The error 'Max LOB size (16777216) exceeded' means a single parsed JSON column value exceeds Snowflake's 16 MB limit for a VARIANT column. The source file is a 3 GB concatenation of many JSON documents being parsed as a massive single object. Splitting the file into multiple…
Question
A table needs to be loaded. The input data is in JSON format and is a concatenation of multiple JSON documents. The file size is 3 GB. A warehouse size small is being used. The following COPY INTO command was executed:
COPY INTO SAMPLE FROM @~/SAMPLE.JSON (TYPE=JSON) The load failed with this error:
Max LOB size (16777216) exceeded, actual size of parsed column is 17894470. How can this issue be resolved?
Options
- ACompress the file and load the compressed file.
- BSplit the file into multiple files in the recommended size range (100 MB - 250 MB).
- CUse a larger-sized warehouse.
- DSet STRIP_OUTER_ARRAY=TRUE in the COPY INTO command.
How the community answered
(30 responses)- A3% (1)
- B80% (24)
- C13% (4)
- D3% (1)
Explanation
The error 'Max LOB size (16777216) exceeded' means a single parsed JSON column value exceeds Snowflake's 16 MB limit for a VARIANT column. The source file is a 3 GB concatenation of many JSON documents being parsed as a massive single object. Splitting the file into multiple files in the 100 MB–250 MB recommended range allows Snowflake to parse each document within size limits and also enables parallel loading. STRIP_OUTER_ARRAY=TRUE applies to JSON arrays, not concatenated documents. A larger warehouse adds CPU/memory but does not raise the LOB size limit. Compression reduces transfer size but not the parsed in-memory size.
Topics
Community Discussion
No community discussion yet for this question.