DEA-C01 · Question #47
A company stores datasets in JSON format and .csv format in an Amazon S3 bucket. The company has Amazon RDS for Microsoft SQL Server databases, Amazon DynamoDB tables that are in provisioned capacity
The correct answer is A. Use AWS Glue to crawl the data sources. Store metadata in the AWS Glue Data Catalog. Use. To enable SQL-like querying across diverse data sources (S3, RDS, DynamoDB, Redshift) with least operational overhead, use AWS Glue to crawl data sources, store metadata in the Glue Data Catalog, and leverage Amazon Athena's federated queries.
Question
A company stores datasets in JSON format and .csv format in an Amazon S3 bucket. The company has Amazon RDS for Microsoft SQL Server databases, Amazon DynamoDB tables that are in provisioned capacity mode, and an Amazon Redshift cluster. A data engineering team must develop a solution that will give data scientists the ability to query all data sources by using syntax similar to SQL. Which solution will meet these requirements with the LEAST operational overhead?
Options
- AUse AWS Glue to crawl the data sources. Store metadata in the AWS Glue Data Catalog. Use
- BUse AWS Glue to crawl the data sources. Store metadata in the AWS Glue Data Catalog. Use
- CUse AWS Glue to crawl the data sources. Store metadata in the AWS Glue Data Catalog. Use
- DUse AWS Lake Formation to create a data lake. Use Lake Formation jobs to transform the data
How the community answered
(21 responses)- A71% (15)
- B10% (2)
- C5% (1)
- D14% (3)
Why each option
To enable SQL-like querying across diverse data sources (S3, RDS, DynamoDB, Redshift) with least operational overhead, use AWS Glue to crawl data sources, store metadata in the Glue Data Catalog, and leverage Amazon Athena's federated queries.
AWS Glue crawlers can automatically discover schema for data in S3, RDS, and Redshift, storing metadata in the AWS Glue Data Catalog. Amazon Athena, being a serverless query engine, can then use this catalog to query S3 data via SQL and can extend to RDS SQL Server and DynamoDB using Federated Queries, providing a unified SQL interface with minimal operational management.
This option describes querying S3 and DynamoDB with Athena but does not fully encompass all specified data sources (RDS, Redshift) in a comprehensive solution for 'least operational overhead' as a unified SQL interface.
This solution focuses on Redshift Spectrum for S3, but it does not address providing a unified SQL query interface for RDS and DynamoDB efficiently, nor does it explicitly state the use of Glue Data Catalog for schema management for all sources.
AWS Lake Formation focuses on building and managing data lakes, and 'Lake Formation jobs to transform the data' implies significant ETL work. The primary requirement is SQL-like querying of *existing* disparate data sources with 'least operational overhead', which Athena with Glue Data Catalog and Federated Queries provides more directly.
Concept tested: AWS Glue Data Catalog, Amazon Athena Federated Query
Source: https://docs.aws.amazon.com/athena/latest/ug/athena-federated-queries.html
Topics
Community Discussion
No community discussion yet for this question.