DAS-C01 · Question #98
An online retail company is migrating its reporting system to AWS. The company's legacy system runs data processing on online transactions using a complex series of nested Apache Hive queries…
The correct answer is A. Create an AWS Glue Data Catalog to manage the Hive metadata. AWS Glue Data Catalog is fully compatible with Apache Hive Metastore. By registering the S3-backed tables in the Glue Data Catalog, existing Hive HQL scripts can run on Amazon EMR (which uses the Glue Data Catalog as its metastore) with little or no modification - satisfying…
Question
An online retail company is migrating its reporting system to AWS. The company's legacy system runs data processing on online transactions using a complex series of nested Apache Hive queries. Transactional data is exported from the online system to the reporting system several times a day. Schemas in the files are stable between updates. A data analyst wants to quickly migrate the data processing to AWS, so any code changes should be minimized. To keep storage costs low, the data analyst decides to store the data in Amazon S3. It is vital that the data from the reports and associated analytics is completely up to date based on the data in Amazon S3. Which solution meets these requirements?
Options
- ACreate an AWS Glue Data Catalog to manage the Hive metadata.
- BCreate an AWS Glue Data Catalog to manage the Hive metadata.
- CCreate an Amazon Athena table with CREATE TABLE AS SELECT (CTAS) to ensure data is
- DUse an S3 Select query to ensure that the data is properly updated.
How the community answered
(58 responses)- A72% (42)
- B9% (5)
- C3% (2)
- D16% (9)
Explanation
AWS Glue Data Catalog is fully compatible with Apache Hive Metastore. By registering the S3-backed tables in the Glue Data Catalog, existing Hive HQL scripts can run on Amazon EMR (which uses the Glue Data Catalog as its metastore) with little or no modification - satisfying the 'minimize code changes' requirement. Queries always read directly from S3, so results are based on whatever data is currently there, satisfying the 'completely up to date' requirement. Amazon Athena (option C with CTAS) materializes query results into new S3 objects, so data would not reflect upstream updates until the CTAS is re-run - violating the freshness requirement. S3 Select (option D) is a single-object row/column filter, not a query engine capable of running nested Hive queries.
Topics
Community Discussion
No community discussion yet for this question.