MLS-C01 · Question #222
A data scientist is working on a model to predict a company's required inventory stock levels. All historical data is stored in .csv files in the company's data lake on Amazon S3. The dataset…
The correct answer is B. Use AWS Glue to crawl the S3 bucket and create tables in the AWS Glue Data Catalog. Use. To allow a data scientist to explore 500 GB of CSV data in Amazon S3 using SQL, while minimizing costs and operational overhead, AWS Glue and Amazon Athena provide the optimal serverless solution.
Question
A data scientist is working on a model to predict a company's required inventory stock levels. All historical data is stored in .csv files in the company's data lake on Amazon S3. The dataset consists of approximately 500 GB of data The data scientist wants to use SQL to explore the data before training the model. The company wants to minimize costs. Which option meets these requirements with the LEAST operational overhead?
Options
- ACreate an Amazon EMR cluster. Create external tables in the Apache Hive metastore,
- BUse AWS Glue to crawl the S3 bucket and create tables in the AWS Glue Data Catalog. Use
- CCreate an Amazon Redshift cluster. Use the COPY command to ingest the data from Amazon S3.
- DCreate an Amazon Redshift cluster. Create external tables in an external schema, referencing the
How the community answered
(20 responses)- A5% (1)
- B75% (15)
- C5% (1)
- D15% (3)
Why each option
To allow a data scientist to explore 500 GB of CSV data in Amazon S3 using SQL, while minimizing costs and operational overhead, AWS Glue and Amazon Athena provide the optimal serverless solution.
Creating an Amazon EMR cluster involves provisioning and managing EC2 instances, leading to higher operational overhead and potentially higher costs than serverless options like Athena.
AWS Glue can crawl the S3 bucket to automatically infer schemas and create tables in the AWS Glue Data Catalog. These cataloged tables can then be queried directly using Amazon Athena, a serverless interactive query service that incurs costs only for data scanned, minimizing both operational overhead and cost.
Amazon Redshift is a data warehousing solution that requires provisioning and managing clusters, which incurs significant operational overhead and cost for simple exploratory SQL on S3 data.
While Amazon Redshift Spectrum allows querying external S3 data, an Amazon Redshift cluster still needs to be provisioned and managed, which does not offer the least operational overhead compared to a fully serverless solution.
Concept tested: Querying S3 data with Athena via Glue Data Catalog
Source: https://docs.aws.amazon.com/athena/latest/ug/glue-data-catalog-and-athena.html
Topics
Community Discussion
No community discussion yet for this question.