DEA-C01 · Question #94
A company uses Amazon EMR as an extract, transform, and load (ETL) pipeline to transform data that comes from multiple sources. A data engineer must orchestrate the pipeline to maximize performance…
The correct answer is C. AWS Step Functions. AWS Step Functions is highly cost-effective for orchestrating EMR workflows because you pay only per state transition, with no infrastructure to maintain. It natively integrates with EMR (including EMR on EC2 and EMR Serverless) to start clusters, submit steps, and wait for…
Question
A company uses Amazon EMR as an extract, transform, and load (ETL) pipeline to transform data that comes from multiple sources. A data engineer must orchestrate the pipeline to maximize performance. Which AWS service will meet this requirement MOST cost effectively?
Options
- AAmazon EventBridge
- BAmazon Managed Workflows for Apache Airflow (Amazon MWAA)
- CAWS Step Functions
- DAWS Glue Workflows
How the community answered
(50 responses)- A2% (1)
- B4% (2)
- C84% (42)
- D10% (5)
Explanation
AWS Step Functions is highly cost-effective for orchestrating EMR workflows because you pay only per state transition, with no infrastructure to maintain. It natively integrates with EMR (including EMR on EC2 and EMR Serverless) to start clusters, submit steps, and wait for completion - all without polling logic. Option A (EventBridge) is an event bus suitable for triggering workflows but lacks the stateful orchestration needed to manage multi-step ETL pipelines. Option B (Amazon MWAA) is a fully managed Apache Airflow service that is powerful but carries significantly higher base costs due to its always-on infrastructure. Option D (AWS Glue Workflows) is optimized for orchestrating Glue jobs and crawlers, not EMR jobs, making it a poor fit here.
Topics
Community Discussion
No community discussion yet for this question.