DEA-C01 · Question #244
A data engineer is optimizing query performance in Amazon Athena notebooks that use Apache Spark to analyze large datasets that are stored in Amazon S3. The data is partitioned. An AWS Glue crawler…
The correct answer is A. Apply partition filters in the queries. By including predicates on your partition columns in each Athena query (for example, WHERE year = '2025' AND month = '05'), Athena prunes partitions at the optimizer level and reads only the matching S3 folders. This directly minimizes the data scanned and improves query…
Question
A data engineer is optimizing query performance in Amazon Athena notebooks that use Apache Spark to analyze large datasets that are stored in Amazon S3. The data is partitioned. An AWS Glue crawler updates the partitions. The data engineer wants to minimize the amount of data that is scanned to improve efficiency of Athena queries. Which solution will meet these requirements?
Options
- AApply partition filters in the queries.
- BIncrease the frequency of AWS Glue crawler invocations to update the data catalog more often.
- COrganize the data that is in Amazon S3 by using a nested directory structure.
- DConfigure Spark to use in-memory caching for frequently accessed data.
How the community answered
(52 responses)- A79% (41)
- B12% (6)
- C4% (2)
- D6% (3)
Explanation
By including predicates on your partition columns in each Athena query (for example, WHERE year = '2025' AND month = '05'), Athena prunes partitions at the optimizer level and reads only the matching S3 folders. This directly minimizes the data scanned and improves query efficiency with no additional infrastructure changes.
Topics
Community Discussion
No community discussion yet for this question.