SOL-C01 · Question #242
A Snowflake architect is designing a data lake solution using Snowflake's external tables. The data lake contains JSON files stored in AWS S3. The architect needs to ensure that Snowflake can automati
The correct answer is C. Use Snowflake's Snowpipe with auto-ingest configured to automatically load data from the S3. Snowpipe with auto-ingest is the most efficient and scalable solution. When configured, AWS S3 sends event notifications (via SQS) to Snowflake whenever new files are created. Snowpipe picks these up automatically and triggers metadata refresh (and optionally data loading) withou
Question
A Snowflake architect is designing a data lake solution using Snowflake's external tables. The data lake contains JSON files stored in AWS S3. The architect needs to ensure that Snowflake can automatically discover new JSON files added to the S3 bucket and make them available for querying through the external table. Which of the following approaches provides the MOST efficient and scalable solution for automatically refreshing the external table metadata? Schedule a daily task in Snowflake to execute 'ALTER EXTERNAL TABLE REFRESH' to manually update the metadata.
Options
- AConfigure an AWS Lambda function triggered by S3 events (e.g., `s3:ObjectCreated') to execute
- BEnable Snowflake's Event Tables and configure them to capture S3 events. Then, create a stream
- CUse Snowflake's Snowpipe with auto-ingest configured to automatically load data from the S3
- DUtilize Snowflake's automatic data clustering feature on the external table based on the file
How the community answered
(48 responses)- A6% (3)
- B10% (5)
- C79% (38)
- D4% (2)
Explanation
Snowpipe with auto-ingest is the most efficient and scalable solution. When configured, AWS S3 sends event notifications (via SQS) to Snowflake whenever new files are created. Snowpipe picks these up automatically and triggers metadata refresh (and optionally data loading) without any manual intervention or scheduled polling. This is event-driven, near-real-time, and requires no external orchestration. Option A (Lambda calling ALTER EXTERNAL TABLE) works but adds external infrastructure complexity. Option B (Event Tables + streams) is overly complex for this use case. Option D (auto-clustering) manages data organization, not metadata discovery of new files. The question's own preamble mentions a scheduled daily task as an insufficient baseline - Snowpipe's auto-ingest surpasses that.
Topics
Community Discussion
No community discussion yet for this question.