DEA-C01 · Question #23
A company needs to partition the Amazon S3 storage that the company uses for a data lake. The partitioning will use a path of the S3 object keys in the following format: s3://bucket/prefix/year=2023/m
The correct answer is C. Use code that writes data to Amazon S3 to invoke the Boto3 AWS Glue create_partition API call.. Use code that writes data to Amazon S3 to invoke the Boto3 AWS Glue create_partition API call. This approach ensures that the Data Catalog is updated as soon as new data is written to S3, providing the least latency in reflecting new partitions.
Question
A company needs to partition the Amazon S3 storage that the company uses for a data lake. The partitioning will use a path of the S3 object keys in the following format:
s3://bucket/prefix/year=2023/month=01/day=01. A data engineer must ensure that the AWS Glue Data Catalog synchronizes with the S3 storage when the company adds new partitions to the bucket. Which solution will meet these requirements with the LEAST latency?
Options
- ASchedule an AWS Glue crawler to run every morning.
- BManually run the AWS Glue CreatePartition API twice each day.
- CUse code that writes data to Amazon S3 to invoke the Boto3 AWS Glue create_partition API call.
- DRun the MSCK REPAIR TABLE command from the AWS Glue console.
How the community answered
(29 responses)- A3% (1)
- B7% (2)
- C72% (21)
- D17% (5)
Explanation
Use code that writes data to Amazon S3 to invoke the Boto3 AWS Glue create_partition API call. This approach ensures that the Data Catalog is updated as soon as new data is written to S3, providing the least latency in reflecting new partitions.
Topics
Community Discussion
No community discussion yet for this question.