SAP-C02 · Question #731
A company deploys workloads in multiple AWS accounts. Each account has a VPC with VPC flow logs published in text log format to a centralized Amazon S3 bucket. Each log file is compressed with gzip…
The correct answer is C. Update the VPC flow log configuration to store the files in Apache Parquet format. Specify hourly. To achieve the largest performance improvement for Amazon Athena queries on VPC flow logs and reduce storage, the logs should be stored in Apache Parquet format with appropriate partitioning (e.g., hourly). Parquet is a columnar format that enables Athena to scan less data…
Question
A company deploys workloads in multiple AWS accounts. Each account has a VPC with VPC flow logs published in text log format to a centralized Amazon S3 bucket. Each log file is compressed with gzip compression. The company must retain the log files indefinitely. A security engineer occasionally analyzes the logs by using Amazon Athena to query the VPC flow logs. The query performance is degrading over time as the number of ingested logs is growing. A solutions architect must improve the performance of the log analysis and reduce the storage space that the VPC flow logs use. Which solution will meet these requirements with the LARGEST performance improvement?
Options
- ACreate an AWS Lambda function to decompress the gzip files and to compress the files with
- BEnable S3 Transfer Acceleration for the S3 bucket. Create an S3 Lifecycle configuration to move
- CUpdate the VPC flow log configuration to store the files in Apache Parquet format. Specify hourly
- DCreate a new Athena workgroup without data usage control limits. Use Athena engine version 2.
How the community answered
(32 responses)- A13% (4)
- B6% (2)
- C63% (20)
- D19% (6)
Why each option
To achieve the largest performance improvement for Amazon Athena queries on VPC flow logs and reduce storage, the logs should be stored in Apache Parquet format with appropriate partitioning (e.g., hourly). Parquet is a columnar format that enables Athena to scan less data, significantly improving query speed and reducing costs.
Using an AWS Lambda function to decompress and re-compress logs introduces significant operational overhead and processing costs, and a different compression method alone will not provide the same magnitude of query performance improvement as a columnar format.
Enabling S3 Transfer Acceleration only improves data ingestion speed, and moving logs to S3 Glacier/Deep Archive would decrease query accessibility and performance rather than improve it.
Changing the VPC flow log configuration to store files in Apache Parquet format is optimal because Parquet is a columnar storage format that significantly reduces the data scanned by Amazon Athena, leading to faster query performance and reduced storage costs through efficient compression and partitioning.
While Athena workgroups and engine versions can offer minor performance tweaks or better resource management, they do not address the fundamental inefficiency of querying row-oriented, unpartitioned text files, which is the primary bottleneck for performance and cost.
Concept tested: Optimizing Athena query performance with columnar formats and partitioning
Source: https://docs.aws.amazon.com/athena/latest/ug/parquet.html
Community Discussion
No community discussion yet for this question.