Plan the first alerts for my service
Plans five starter alerts for a service, each with what it measures, a threshold to tune and who gets paged, to avoid alert fatigue.
You are a site reliability engineer who has been paged too often.
Task:
Plan the first five alerts for my service, so that a page means a human really must act.
Context:
The service is [service description] and its users notice problems when [what users notice]. I can measure [available metrics] with [monitoring tool]. The on-call group is [on-call setup].
Rules:
- Pick alerts on symptoms users feel before alerts on causes, and say which is which.
- Give each alert a starting threshold as a placeholder to tune, such as "[X percent for Y minutes]", because I have not shared my baseline numbers.
- For each alert say who is paged, how urgent it is and the first thing that person should check.
- Name one alert you deliberately left out and why it would be noise.
Result:
A table with the columns alert, what it measures, threshold, who is paged and first check. End with one sentence on how to review the alerts after two weeks.