PROFESSIONAL-CLOUD-DEVOPS-ENGINEER · Question #86
You encounter a large number of outages in the production systems you support. You receive alerts for all the outages, the alerts are due to unhealthy systems that are automatically restarted within…
The correct answer is A. Eliminate alerts that are not actionable. To prevent staff burnout while following SRE practices when systems self-recover quickly and generate many alerts, eliminate alerts that are not actionable.
Question
Options
- AEliminate alerts that are not actionable
- BRedefine the related SLO so that the error budget is not exhausted
- CDistribute the alerts to engineers in different time zones
- DCreate an incident report for each of the alerts
How the community answered
(20 responses)- A80% (16)
- B10% (2)
- C5% (1)
- D5% (1)
Why each option
To prevent staff burnout while following SRE practices when systems self-recover quickly and generate many alerts, eliminate alerts that are not actionable.
A fundamental SRE principle for alerting is that alerts should be actionable and require immediate human attention. If systems automatically recover within a minute without requiring intervention, the alerts are generating noise and contributing to alert fatigue, directly causing staff burnout; eliminating such non-actionable alerts allows staff to focus on real problems.
Redefining the related SLO so that the error budget is not exhausted would potentially mask underlying reliability issues rather than solve the problem of excessive, non-actionable alerts causing burnout.
Distributing alerts to engineers in different time zones only shifts the burden of alert management and does not reduce the number of non-actionable alerts or address the root cause of burnout from excessive notifications.
Creating an incident report for each minute-long, self-recovering outage would significantly increase workload and administrative overhead, directly contributing to staff burnout rather than preventing it.
Concept tested: SRE principles for actionable alerting
Source: https://sre.google/sre-book/monitoring-distributed-systems/#alerting
Topics
Community Discussion
No community discussion yet for this question.