SOL-C01 · Question #119
Your team has set up an automated data pipeline to load data into Snowflake using the *COPY INTO' command from an S3 bucket. After some time, you notice that some files in the bucket are not being loa
The correct answer is B. Query the 'COPY_HISTORY view in the 'SNOWFLAKE database to review the load status and C. Parse the output of the 'COPY INTO' command to capture the number of rows successfully loaded. B is correct because COPY_HISTORY (in SNOWFLAKE.ACCOUNT_USAGE or INFORMATION_SCHEMA) is Snowflake's native, purpose-built view for tracking file load history - it records each file's status (LOADED, LOAD_FAILED, PARTIALLY_LOADED), error messages, row counts, and timestamps, makin
Question
Your team has set up an automated data pipeline to load data into Snowflake using the *COPY INTO' command from an S3 bucket. After some time, you notice that some files in the bucket are not being loaded, and you suspect this is due to errors during the loading process. You want to implement a monitoring solution to track these failed file loads and take appropriate action. What are the BEST ways to achieve this? Choose all that apply
Options
- AExamine the 'VALIDATE function output after each *COPY INTO' operation to check for any
- BQuery the 'COPY_HISTORY view in the 'SNOWFLAKE database to review the load status and
- CParse the output of the 'COPY INTO' command to capture the number of rows successfully loaded
- DImplement an external monitoring tool to scan the S3 bucket for files that have not been modified
- EUtilize Snowflake's error notification system to receive alerts when a 'COPY INTO' command
How the community answered
(33 responses)- A6% (2)
- B82% (27)
- D9% (3)
- E3% (1)
Explanation
B is correct because COPY_HISTORY (in SNOWFLAKE.ACCOUNT_USAGE or INFORMATION_SCHEMA) is Snowflake's native, purpose-built view for tracking file load history - it records each file's status (LOADED, LOAD_FAILED, PARTIALLY_LOADED), error messages, row counts, and timestamps, making it ideal for ongoing monitoring. C is correct because the COPY INTO command returns a structured result set after each run showing rows loaded vs. rows with errors; programmatically parsing this output lets your pipeline detect failures immediately and take action (e.g., alerts, retries).
A is wrong because the VALIDATE function is a diagnostic tool that re-examines errors from a specific, already-completed load (referenced by query ID) - it doesn't provide ongoing monitoring or historical tracking across multiple loads. D is wrong because checking S3 for unmodified files conflates "not recently written" with "failed to load" - a file could be static simply because it's old, not because loading failed; this approach has no knowledge of Snowflake's load state. E is wrong because Snowflake does not have a built-in, out-of-the-box error notification system specifically triggered by COPY INTO failures - any alerting would require custom setup using Snowflake Alerts or an external tool, making it not a "best" native solution.
Memory tip: Think "B-C = Built-in Check" - always reach for Snowflake's native load tracking first: Browse the history view and Capture the command output. If you're going outside Snowflake (S3 scanning, external tools), you've already left the best path.
Topics
Community Discussion
No community discussion yet for this question.