DEA-C01 · Question #53
A company is designing a data lake on Amazon S3. To ensure high performance when accessing the data, which best practice should the company adopt in organizing its data in the S3 bucket?
The correct answer is C. Partition data based on commonly accessed attributes and use a consistent naming scheme for. For high-performance data access in an Amazon S3 data lake, the best practice is to partition data based on commonly accessed attributes and use a consistent naming scheme.
Question
A company is designing a data lake on Amazon S3. To ensure high performance when accessing the data, which best practice should the company adopt in organizing its data in the S3 bucket?
Options
- AStore all data files as a single large file and use AWS Lambda to parse required data segments.
- BUse a flat structure by avoiding the creation of any prefix or "folder" hierarchy.
- CPartition data based on commonly accessed attributes and use a consistent naming scheme for
- DEnable S3 Transfer Acceleration to ensure data is quickly accessible from any location.
How the community answered
(30 responses)- A7% (2)
- B3% (1)
- C80% (24)
- D10% (3)
Why each option
For high-performance data access in an Amazon S3 data lake, the best practice is to partition data based on commonly accessed attributes and use a consistent naming scheme.
Storing all data as a single large file is an anti-pattern as it forces query engines to scan the entire file, severely degrading performance, and Lambda is not suitable for parsing large-scale data queries.
Using a flat structure without prefixes makes data organization and filtering inefficient, leading to poor query performance because it inhibits effective partitioning.
Partitioning data in S3 based on frequently queried attributes (e.g., date, region) significantly enhances query performance by reducing the amount of data scanned. A consistent naming scheme for these partitions improves data organization and discoverability for analytical engines like Athena or Spark.
S3 Transfer Acceleration improves data transfer speeds to and from S3, but it does not enhance the performance of querying data that is already stored within the S3 bucket.
Concept tested: Amazon S3 data lake design, data partitioning for performance
Source: https://docs.aws.amazon.com/whitepapers/latest/building-data-lakes/best-practices.html
Topics
Community Discussion
No community discussion yet for this question.