DEA-C01 · Question #70
A data engineer runs Amazon Athena queries on data that is in an Amazon S3 bucket. The Athena queries use AWS Glue Data Catalog as a metadata table. The data engineer notices that the Athena query…
The correct answer is A. Create an AWS Glue partition index. Enable partition filtering. C. Use Athena partition projection based on the S3 bucket prefix. https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/ Optimizing Partition Processing using partition projection Processing partition information can be a bottleneck for Athena queries when you have a very large number of partitions and aren’t…
Question
A data engineer runs Amazon Athena queries on data that is in an Amazon S3 bucket. The Athena queries use AWS Glue Data Catalog as a metadata table. The data engineer notices that the Athena query plans are experiencing a performance bottleneck. The data engineer determines that the cause of the performance bottleneck is the large number of partitions that are in the S3 bucket. The data engineer must resolve the performance bottleneck and reduce Athena query planning time. Which solutions will meet these requirements? (Choose two.)
Options
- ACreate an AWS Glue partition index. Enable partition filtering.
- BBucket the data based on a column that the data have in common in a WHERE clause of the user
- CUse Athena partition projection based on the S3 bucket prefix.
- DTransform the data that is in the S3 bucket to Apache Parquet format.
- EUse the Amazon EMR S3DistCP utility to combine smaller objects in the S3 bucket into larger
How the community answered
(19 responses)- A58% (11)
- B26% (5)
- D5% (1)
- E11% (2)
Explanation
https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/ Optimizing Partition Processing using partition projection Processing partition information can be a bottleneck for Athena queries when you have a very large number of partitions and aren’t using AWS Glue partition indexing. You can use partition projection in Athena to speed up query processing of highly partitioned tables and automate partition management. Partition projection helps minimize this overhead by allowing you to query partitions by calculating partition information rather than retrieving it from a metastore. It eliminates the need to add partitions’ metadata to the AWS Glue table.
Topics
Community Discussion
No community discussion yet for this question.