DAS-C01 · Question #188
A company is using an AWS Lambda function to run Amazon Athena queries against a cross- account AWS Glue Data Catalog. A query returns the following error: HIVE_METASTORE_ERROR The error message state
The correct answer is A. Modify the Lambda function to upload the query response payload as an object into the S3 bucket.. This error occurs because Lambda has a hard payload size limit (6 MB for synchronous invocations). When Athena queries a cross-account AWS Glue Data Catalog through a Lambda function, the metadata response - especially for heavily partitioned tables - can exceed this limit. Optio
Question
A company is using an AWS Lambda function to run Amazon Athena queries against a cross- account AWS Glue Data Catalog. A query returns the following error:
HIVE_METASTORE_ERROR The error message states that the response payload size exceeds the maximum allowed size. The queried table is already partitioned, and the data is stored in an Amazon S3 bucket in the Apache Hive partition format. Which solution will resolve this error?
Options
- AModify the Lambda function to upload the query response payload as an object into the S3 bucket.
- BRun the MSCK REPAIR TABLE command on the queried table.
- CCreate a separate folder in the S3 bucket. Move the data files that need to be queried into that folder.
- DCheck the schema of the queried table for any characters that Athena does not support. Replace any
How the community answered
(35 responses)- A63% (22)
- B20% (7)
- C11% (4)
- D6% (2)
Explanation
This error occurs because Lambda has a hard payload size limit (6 MB for synchronous invocations). When Athena queries a cross-account AWS Glue Data Catalog through a Lambda function, the metadata response - especially for heavily partitioned tables - can exceed this limit. Option A resolves this by modifying the Lambda function to write the large response payload to S3 and return only the S3 reference, sidestepping the payload size constraint. Option B (MSCK REPAIR TABLE) re-discovers and repairs partition metadata but does not reduce the size of the Glue response. Option C (moving data files) reorganizes data but does not affect partition metadata volume. Option D (checking schema characters) is unrelated to a payload size issue.
Topics
Community Discussion
No community discussion yet for this question.