SRE, DevOps and platform teams: who gets the report to the customer?
A healthy queue and a submitted job can coexist with a missing report. Follow the handoffs to see which team can make the next decision.
Guides
Walkthroughs for observability, incident response and AI-assisted operations.
Browse guides with article filtersAIOps fundamentals
Start with what AI can do for operations, and where judgment still matters.
Read AIOps fundamentalsObservability for SRE
Make metrics, logs and traces work together.
Read Observability for SREIncident management with AI
Build an incident response practice that learns.
Read Incident management with AIPractical articles
Instructions, examples and implementation notes.
Capacity testing: what happens when one worker stops?
Restoring capacity is only the start of recovery. Follow a small queue calculation to see how new arrivals compete with old work, and how to test the prediction safely.
The incident looks different from each screen
A missing report, a falling chart and a recent release seem to tell one story. Put their times and sources together before deciding which story it is.
Investigate the incident without undoing each other’s work
One responder needs the worker’s connections. Another wants to restart it. Give their investigations a common question and make the next change visible.
What to tell customers while the answer is changing
At 14:15, one region has delayed reports. Eight minutes later, you know about another. Keep people informed without promising a fix you cannot yet see.
Fewer requests arrived. Did the mitigation work?
A submission limit can do exactly what you asked and still leave reports waiting. Compare it with a rollback, then follow the work after the change.
When the first mitigation helps but does not fix the incident
The request graph falls, yet the oldest report still waits. Keep the protection that worked while finding the next question and the person authorized to act.
Hand over the incident, not another mystery
The incoming lead needs the next decision, the controls still active and the changes that never ran. Follow one fictional handoff from Maya to Rowan.
One report arrived. Is the incident over?
New exports can succeed while older customers are still waiting. Check the download path, the backlog and returning demand before deciding what recovery means.
Review the incident without giving responders tomorrow’s knowledge
A deployment looked suspicious until older-version jobs told a different story. Explain decisions using the clues and authority available when people made them.
The new approval route is written. Can someone use it?
Replay the moment the approver went missing, then watch what the revised procedure lets a responder do and where it still leaves them waiting.
An unfamiliar account appears during the incident
It might be a backup job. It might need urgent attention. Bring the observation to a specialist without losing the service investigation or spreading sensitive records.