DEA-C01 · Question #297
A company needs to build a data pipeline to process a 1-TB file from an Amazon S3 bucket. The pipeline needs to create three DataFrames based on business logic. The pipeline must save all three…
The correct answer is C. Configure an AWS Glue workflow to run three AWS Glue jobs in parallel to process the file. An AWS Glue workflow can be triggered by an EventBridge rule when a file is uploaded to the source S3 bucket, and it can orchestrate three AWS Glue jobs to run in parallel. This provides a fully managed, serverless approach for Spark-based DataFrame processing at the 1-TB scale…
Question
A company needs to build a data pipeline to process a 1-TB file from an Amazon S3 bucket. The pipeline needs to create three DataFrames based on business logic. The pipeline must save all three DataFrames to a second S3 bucket in parallel. The company needs to set the pipeline to be the target of an Amazon EventBridge rule that matches file uploads to the source S3 bucket. Which solution will meet these requirements with the LEAST maintenance overhead?
Options
- AConfigure an Apache Spark Streaming application on Amazon EMR to process data from the S3
- BConfigure three AWS Lambda functions to process the business logic and to save the
- CConfigure an AWS Glue workflow to run three AWS Glue jobs in parallel to process the file.
- DConfigure an AWS Step Functions state machine to initiate an AWS Glue workflow to run three
How the community answered
(33 responses)- A6% (2)
- B6% (2)
- C73% (24)
- D15% (5)
Explanation
An AWS Glue workflow can be triggered by an EventBridge rule when a file is uploaded to the source S3 bucket, and it can orchestrate three AWS Glue jobs to run in parallel. This provides a fully managed, serverless approach for Spark-based DataFrame processing at the 1-TB scale while keeping orchestration and operational management minimal.
Topics
Community Discussion
No community discussion yet for this question.