A company wants to run analytics on its Elastic Load Balancing logs stored in Amazon S3. A data analyst needs to be able to query all data from a desired year, month, or day. The data analyst should a

Question

Options

• AUse an AWS Glue job nightly to transform new log files into .csv format and partition by year,
• BLaunch a long-running Amazon EMR cluster that continuously transforms new log files from
• CLaunch a transient Amazon EMR cluster nightly to transform new log files into Apache ORC
• DUse an AWS Glue job nightly to transform new log files into Apache Parquet format and partition

• A
  5% (2)
• B
  17% (7)
• C
  7% (3)
• D
  71% (29)

Explanation

The two key requirements are: (1) query by year/month/day partitions, and (2) query a subset of columns - both with minimal operational overhead. Apache Parquet is a columnar storage format, meaning Athena only reads the columns requested, greatly reducing scan costs and improving speed. Partitioning by year, month, and day enables Athena partition pruning, so only relevant partitions are scanned. AWS Glue is serverless, satisfying the minimal operational overhead requirement. Option A uses CSV, which is row-based and inefficient for column subset queries. Options B and C use EMR, which requires cluster management and more operational overhead.

No community discussion yet for this question.