DEA-C01 · Question #251
A company uses AWS Glue Apache Spark jobs to handle extract, transform, and load (ETL) workloads. The company has enabled logging and monitoring for all AWS Glue jobs. One of the AWS Glue jobs…
The correct answer is A. Examine the AWS Glue job and stage details in the Spark UI. AWS Glue provides a built-in Spark UI (accessible from the Glue console's job run details) that exposes per-stage metrics, such as task counts, shuffle read/write sizes, and execution times, for each Spark stage. This is the most direct way to drill down into the individual…
Question
A company uses AWS Glue Apache Spark jobs to handle extract, transform, and load (ETL) workloads. The company has enabled logging and monitoring for all AWS Glue jobs. One of the AWS Glue jobs begins to fail. A data engineer investigates the error and wants to examine metrics for all individual stages within the job. How can the data engineer access the stage metrics?
Options
- AExamine the AWS Glue job and stage details in the Spark UI.
- BExamine the AWS Glue job and stage metrics in Amazon CloudWatch.
- CExamine the AWS Glue job and stage logs in AWS CloudTrail logs.
- DExamine the AWS Glue job and stage details by using the run insights feature on the job.
How the community answered
(39 responses)- A79% (31)
- B3% (1)
- C13% (5)
- D5% (2)
Explanation
AWS Glue provides a built-in Spark UI (accessible from the Glue console's job run details) that exposes per-stage metrics, such as task counts, shuffle read/write sizes, and execution times, for each Spark stage. This is the most direct way to drill down into the individual stages when troubleshooting a failing Glue Spark job.
Topics
Community Discussion
No community discussion yet for this question.