nerdexam
Amazon

DEA-C01 · Question #7

A data engineer must orchestrate a series of Amazon Athena queries that will run every day. Each query can run for more than 15 minutes. Which combination of steps will meet these requirements MOST co

The correct answer is A. Use an AWS Lambda function and the Athena Boto3 client start_query_execution API call to B. Create an AWS Step Functions workflow and add two states. Add the first state before the. To cost-effectively orchestrate daily Amazon Athena queries that can run for more than 15 minutes, use an AWS Step Functions workflow to manage the overall process and leverage AWS Lambda functions to initiate the Athena query executions.

Data Operations and Support

Question

A data engineer must orchestrate a series of Amazon Athena queries that will run every day. Each query can run for more than 15 minutes. Which combination of steps will meet these requirements MOST cost-effectively? (Choose two.)

Options

  • AUse an AWS Lambda function and the Athena Boto3 client start_query_execution API call to
  • BCreate an AWS Step Functions workflow and add two states. Add the first state before the
  • CUse an AWS Glue Python shell job and the Athena Boto3 client start_query_execution API call to
  • DUse an AWS Glue Python shell script to run a sleep timer that checks every 5 minutes to
  • EUse Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to orchestrate the Athena

How the community answered

(27 responses)
  • A
    81% (22)
  • C
    4% (1)
  • D
    11% (3)
  • E
    4% (1)

Why each option

To cost-effectively orchestrate daily Amazon Athena queries that can run for more than 15 minutes, use an AWS Step Functions workflow to manage the overall process and leverage AWS Lambda functions to initiate the Athena query executions.

AUse an AWS Lambda function and the Athena Boto3 client start_query_execution API call toCorrect

An AWS Lambda function provides a cost-effective, serverless way to initiate an Amazon Athena query using the Boto3 start_query_execution API call, which can then be orchestrated and monitored by an AWS Step Functions workflow.

BCreate an AWS Step Functions workflow and add two states. Add the first state before theCorrect

AWS Step Functions is a serverless orchestration service that can manage long-running workflows, including starting and polling the status of Amazon Athena queries for durations exceeding 15 minutes, with costs based on state transitions, making it highly cost-effective for waiting.

CUse an AWS Glue Python shell job and the Athena Boto3 client start_query_execution API call to

AWS Glue Python shell jobs involve provisioned compute resources, which can be more expensive than Step Functions' state-transition model for simple orchestration, especially when waiting for external tasks to complete.

DUse an AWS Glue Python shell script to run a sleep timer that checks every 5 minutes to

Using a sleep timer in a Glue Python shell script is inefficient and not cost-effective, as the job consumes compute resources while idly waiting.

EUse Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to orchestrate the Athena

Amazon Managed Workflows for Apache Airflow (MWAA) provides robust orchestration but generally has a higher baseline cost compared to AWS Step Functions for simpler, daily scheduled tasks.

Concept tested: Orchestrating long-running serverless workflows

Source: https://docs.aws.amazon.com/step-functions/latest/dg/connect-athena.html

Topics

#Athena#Workflow Orchestration#AWS Step Functions#AWS Lambda

Community Discussion

No community discussion yet for this question.

Full DEA-C01 Practice