DEA-C01 Question #176: Real Exam Question with Answer & Explanation

Question

A data engineer configured an AWS Glue Data Catalog for data that is stored in Amazon S3 buckets. The data engineer needs to configure the Data Catalog to receive incremental updates. The data engineer sets up event notifications for the S3 bucket and creates an Amazon Simple Queue Service (Amazon SQS) queue to receive the S3 events. Which combination of steps should the data engineer take to meet these requirements with LEAST operational overhead? (Choose two.)

Options

• ACreate an S3 event-based AWS Glue crawler to consume events from the SQS queue.
• BDefine a time-based schedule to run the AWS Glue crawler, and perform incremental updates to
• CUse an AWS Lambda function to directly update the Data Catalog based on S3 events that the
• DManually initiate the AWS Glue crawler to perform updates to the Data Catalog when there is a
• EUse AWS Step Functions to orchestrate the process of updating the Data Catalog based on S3

Explanation

AWS Glue crawlers can be configured to work with S3 event notifications and consume events from SQS queues. This allows the crawler to automatically detect changes in the S3 bucket and update the Data Catalog incrementally, minimizing operational overhead by automating the You can use AWS Lambda to process the S3 events as they are received in the SQS queue. The Lambda function can be configured to automatically update the Glue Data Catalog, ensuring that changes are reflected in near real-time without manual intervention, which also reduces operational overhead. A time-based schedule does not meet the requirement for receiving incremental updates in real time. Scheduling adds unnecessary delay and operational complexity compared to event-driven Manually triggering the crawler would introduce high operational overhead and is not aligned with the requirement to reduce the manual process for updating the Data Catalog. While Step Functions could be used for orchestration, this adds complexity compared to using an event-driven Lambda function, which provides a simpler and more direct solution for processing

No community discussion yet for this question.