DEA-C01 · Question #246
A data engineer is building a data pipeline. A large data file is uploaded to an Amazon S3 bucket once each day at unpredictable times. An AWS Glue workflow uses hundreds of workers to process the…
The correct answer is B. Create an event-based AWS Glue trigger to start the workflow. Configure Amazon S3 to log. An event-based trigger driven by Amazon EventBridge provides the lowest-latency, lowest- overhead way to kick off your Glue workflow as soon as the file arrives. By enabling S3 data events in CloudTrail and writing a simple EventBridge rule that matches the PutObject event for…
Question
A data engineer is building a data pipeline. A large data file is uploaded to an Amazon S3 bucket once each day at unpredictable times. An AWS Glue workflow uses hundreds of workers to process the file and load the data into Amazon Redshift. The company wants to process the file as quickly as possible. Which solution will meet these requirements?
Options
- ACreate an on-demand AWS Glue trigger to start the workflow. Create an AWS Lambda function
- BCreate an event-based AWS Glue trigger to start the workflow. Configure Amazon S3 to log
- CCreate a scheduled AWS Glue trigger to start the workflow. Create a cron job that runs the AWS
- DCreate an on-demand AWS Glue trigger to start the workflow. Create an AWS Database
How the community answered
(18 responses)- A11% (2)
- B78% (14)
- C6% (1)
- D6% (1)
Explanation
An event-based trigger driven by Amazon EventBridge provides the lowest-latency, lowest- overhead way to kick off your Glue workflow as soon as the file arrives. By enabling S3 data events in CloudTrail and writing a simple EventBridge rule that matches the PutObject event for your bucket, you can target your Glue workflow's trigger directly, eliminating continuous polling or extra Lambda or DMS infrastructure.
Topics
Community Discussion
No community discussion yet for this question.