DEA-C01 · Question #257
A company runs an Apache Spark application every night in an Amazon EMR cluster. The company uses Amazon EC2 instances to supply compute capacity for the EMR cluster. The company deployed the Spark…
The correct answer is C. The engineer can connect to the Amazon EMR console to see the Amazon EMR step logs that. In EMR cluster mode, the Spark driver runs inside the cluster and its standard error logs are captured as step logs. These logs are automatically archived to Amazon S3 and accessible from the Amazon EMR console under step logs. This is the correct location for investigating Spark
Question
A company runs an Apache Spark application every night in an Amazon EMR cluster. The company uses Amazon EC2 instances to supply compute capacity for the EMR cluster. The company deployed the Spark application in cluster mode. An error occurs in the Spark application. A log for the error is stored in the application's Spark driver standard error logs. A data engineer needs to investigate the error. Where can the data engineer find this error log?
Options
- AThe engineer can connect to the web UI on the live cluster to see the YARN ResourceManager
- BThe engineer can connect to the persistent application UI to see the first YARN container log in
- CThe engineer can connect to the Amazon EMR console to see the Amazon EMR step logs that
- DThe engineer can connect to the primary node of the cluster by using SSH to see the Spark
How the community answered
(31 responses)- A10% (3)
- B3% (1)
- C84% (26)
- D3% (1)
Explanation
In EMR cluster mode, the Spark driver runs inside the cluster and its standard error logs are captured as step logs. These logs are automatically archived to Amazon S3 and accessible from the Amazon EMR console under step logs. This is the correct location for investigating Spark
Topics
Community Discussion
No community discussion yet for this question.