DAS-C01 · Question #176
A real estate company maintains data about all properties listed in a market. The company receives data about new property listings from vendors who upload the data daily as compressed files into…
The correct answer is C. Use AWS Glue for processing incoming data. Use AWS Step Functions for workflow orchestration. Use. To process daily compressed files uploaded to Amazon S3 immediately upon arrival, automate the workflow, and enable scalable querying for dashboards and reporting, the most cost-effective solution is to use AWS Glue for data processing orchestrated by AWS Step Functions. This…
Question
A real estate company maintains data about all properties listed in a market. The company receives data about new property listings from vendors who upload the data daily as compressed files into Amazon S3. The company’s leadership team wants to see the most up-to-date listings as soon as the data is uploaded to Amazon S3. The data analytics team must automate and orchestrate the data processing workflow of the listings to feed a dashboard. The team also must provide the ability to perform one-time queries and analytical reporting in a scalable manner. Which solution meets these requirements MOST cost-effectively?
Options
- AUse Amazon EMR for processing incoming data. Use AWS Step Functions for workflow orchestration.
- BUse Amazon EMR for processing incoming data. Use AWS Step Functions for workflow orchestration.
- CUse AWS Glue for processing incoming data. Use AWS Step Functions for workflow orchestration. Use
- DUse AWS Glue for processing incoming data. Use AWS Lambda and S3 Event Notifications for
How the community answered
(38 responses)- A3% (1)
- B3% (1)
- C84% (32)
- D11% (4)
Why each option
To process daily compressed files uploaded to Amazon S3 immediately upon arrival, automate the workflow, and enable scalable querying for dashboards and reporting, the most cost-effective solution is to use AWS Glue for data processing orchestrated by AWS Step Functions. This serverless approach, triggered by S3 Event Notifications, ensures efficiency and reduces operational overhead.
Amazon EMR involves managing clusters, even transient ones, which generally incurs higher operational overhead and can be less cost-effective than serverless solutions like AWS Glue for daily batch processing tasks.
Amazon EMR involves managing clusters, even transient ones, which generally incurs higher operational overhead and can be less cost-effective than serverless solutions like AWS Glue for daily batch processing tasks.
AWS Glue provides serverless, cost-effective ETL for processing compressed data, paying only for the duration of the job execution. AWS Step Functions offers robust, serverless workflow orchestration, triggered by S3 Event Notifications, to automate the processing pipeline efficiently. This combination allows for timely data updates, scalable queries, and minimal operational overhead.
While AWS Glue and S3 Event Notifications are good, orchestrating multi-step data processing workflows using AWS Lambda functions can become complex and less manageable or cost-effective than using AWS Step Functions, which is purpose-built for workflow orchestration.
Concept tested: Serverless data lake ingestion and ETL workflow
Source: https://aws.amazon.com/glue/
Topics
Community Discussion
No community discussion yet for this question.