DEA-C01 · Question #247
A data engineer needs to run a data transformation job whenever a user adds a file to an Amazon S3 bucket. The job will run for less than 1 minute. The job must send the output through an email…
The correct answer is C. Create an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to. An S3 event notification on ObjectCreated can directly invoke a Lambda function as soon as a new file arrives. Lambda can perform your <1 minute transformation and then publish the result to an SNS topic. With the data engineer subscribed to that topic by email, the output is…
Question
A data engineer needs to run a data transformation job whenever a user adds a file to an Amazon S3 bucket. The job will run for less than 1 minute. The job must send the output through an email message to the data engineer. The data engineer expects users to add one file every hour of the day. Which solution will meet these requirements in the MOST operationally efficient way?
Options
- ACreate a small Amazon EC2 instance that polls the S3 bucket for new files. Run transformation
- BRun an Amazon Elastic Container Service (Amazon ECS) task to poll the S3 bucket for new files.
- CCreate an AWS Lambda function to transform the data. Use Amazon S3 Event Notifications to
- DDeploy an Amazon EMR cluster. Use EMR File System (EMRFS) to access the files in the S3
How the community answered
(35 responses)- A3% (1)
- B6% (2)
- C83% (29)
- D9% (3)
Explanation
An S3 event notification on ObjectCreated can directly invoke a Lambda function as soon as a new file arrives. Lambda can perform your <1 minute transformation and then publish the result to an SNS topic. With the data engineer subscribed to that topic by email, the output is delivered - no servers to manage, no polling, and minimal ongoing operations.
Topics
Community Discussion
No community discussion yet for this question.