DAS-C01 · Question #76
A healthcare company uses AWS data and analytics tools to collect, ingest, and store electronic health record (EHR) data about its patients. The raw EHR data is stored in Amazon S3 in JSON format…
The correct answer is C. Use the AWS Glue API CreateTable operation to create a table in the Data Catalog. The two requirements are: (1) choose the exact catalog table name rather than letting a crawler auto-generate it, and (2) keep partitions updated as new data arrives. The AWS Glue API CreateTable operation lets you programmatically define a table with a precise name you…
Question
A healthcare company uses AWS data and analytics tools to collect, ingest, and store electronic health record (EHR) data about its patients. The raw EHR data is stored in Amazon S3 in JSON format partitioned by hour, day, and year and is updated every hour. The company wants to maintain the data catalog and metadata in an AWS Glue Data Catalog to be able to access the data using Amazon Athena or Amazon Redshift Spectrum for analytics. When defining tables in the Data Catalog, the company has the following requirements:
- Choose the catalog table name and do not rely on the catalog table
naming algorithm.
- Keep the table updated with new partitions loaded in the respective
S3 bucket prefixes. Which solution meets these requirements with minimal effort?
Options
- ARun an AWS Glue crawler that connects to one or more data stores, determines the data
- BUse the AWS Glue console to manually create a table in the Data Catalog and schedule an AWS
- CUse the AWS Glue API CreateTable operation to create a table in the Data Catalog.
- DCreate an Apache Hive catalog in Amazon EMR with the table schema definition in Amazon S3,
How the community answered
(42 responses)- A12% (5)
- B2% (1)
- C79% (33)
- D7% (3)
Explanation
The two requirements are: (1) choose the exact catalog table name rather than letting a crawler auto-generate it, and (2) keep partitions updated as new data arrives. The AWS Glue API CreateTable operation lets you programmatically define a table with a precise name you specify, satisfying requirement 1. For requirement 2, you pair it with the Glue BatchCreatePartition API (or MSCK REPAIR TABLE via Athena) on a schedule to register new S3 partitions automatically. A Glue crawler (A) does not allow you to control the table name-it derives names from the data store. Manually creating tables via the console (B) is not minimal effort for ongoing partition updates. An Apache Hive catalog on EMR (D) requires managing EMR infrastructure and does not natively integrate with the Glue Data Catalog without extra configuration.
Topics
Community Discussion
No community discussion yet for this question.