DEA-C01 · Question #131
A data engineer is processing and analyzing multiple terabytes of raw data that is in Amazon S3. The data engineer needs to clean and prepare the data. Then the data engineer needs to load the data…
The correct answer is B. Use AWS Glue DataBrew to prepare the data. Use AWS Glue to load the data into Amazon. The data engineer needs a solution to clean, prepare, and load terabytes of raw S3 data into Amazon Redshift for complex analytics, requiring minimal ETL complexity and no infrastructure management.
Question
A data engineer is processing and analyzing multiple terabytes of raw data that is in Amazon S3. The data engineer needs to clean and prepare the data. Then the data engineer needs to load the data into Amazon Redshift for analytics. The data engineer needs a solution that will give data analysts the ability to perform complex queries. The solution must eliminate the need to perform complex extract, transform, and load (ETL) processes or to manage infrastructure. Which solution will meet these requirements with the LEAST operational overhead?
Options
- AUse Amazon EMR to prepare the data. Use AWS Step Functions to load the data into Amazon
- BUse AWS Glue DataBrew to prepare the data. Use AWS Glue to load the data into Amazon
- CUse AWS Lambda to prepare the data. Use Amazon Kinesis Data Firehose to load the data into
- DUse AWS Glue to prepare the data. Use AWS Database Migration Service (AVVS DMS) to load
How the community answered
(42 responses)- A7% (3)
- B86% (36)
- C5% (2)
- D2% (1)
Why each option
The data engineer needs a solution to clean, prepare, and load terabytes of raw S3 data into Amazon Redshift for complex analytics, requiring minimal ETL complexity and no infrastructure management.
Amazon EMR requires managing clusters, which goes against the requirement to 'eliminate the need to manage infrastructure' and is not serverless for data preparation. AWS Step Functions orchestrates workflows but doesn't perform the data preparation or loading itself.
AWS Glue DataBrew is a visual data preparation tool that allows data engineers to clean and normalize data without writing code, eliminating complex ETL scripting. AWS Glue (specifically Glue ETL jobs) is a serverless data integration service that can then extract, transform, and load the prepared data into Amazon Redshift, providing a fully managed, serverless solution with minimal operational overhead.
AWS Lambda is suitable for event-driven, small-scale tasks but is not designed for processing multiple terabytes of data for cleaning and preparation due to its execution limits. Amazon Kinesis Data Firehose is for streaming data delivery, not batch loading of prepared data from S3 to Redshift in this context.
While AWS Glue can prepare data in a serverless manner, AWS Database Migration Service (AWS DMS) is primarily designed for migrating databases and continuous data replication, not for general-purpose batch loading of prepared S3 data into Redshift, making it an inefficient tool choice for this specific ETL step.
Concept tested: Serverless ETL for data warehousing
Source: https://aws.amazon.com/databrew/
Topics
Community Discussion
No community discussion yet for this question.