GuideIncident response
Keeping incident state consistent across PagerDuty, Jira and Slack
A reference design for duplicate deliveries and partial API success, with durable incident identifiers and summaries drawn from the resulting record.
Articles
Guides, news and explanations about AI and reliable systems.
GuideObservability
Grafana dashboards for incident response
Regional failures can disappear in fleet averages. Grafana queries, population labels and linked traces help responders find the affected requests.
GuideAI in production
Prompting an assistant to investigate an incident
A reusable prompt connects observations, competing explanations and the next read-only check, so an incomplete record can still support useful investigation.
GuideIncident response
Coordinating incidents and bot actions in Slack
Incident channels connect responders to decisions and results. Write-capable bots also need precise action identifiers, duplicate handling and a route around a chat outage.
GuideAI in production
Comparing AIOps tools on responder work
Detection, correlation and remediation need different trials. The comparison includes specialist escalations, correction effort and the integrations left to maintain.
GuideIncident response
Finding the response delay AIOps can reduce
Evidence gathering, authorization and restoration can each hold up an incident. The timeline shows which kind of assistance could shorten the wait.
GuideIncident response
Customer reports reveal failures a fleet average can miss
An upload report becomes investigable through its operation, region and time window. That same detail connects the engineering result back to the customer.
GuideObservability
What makes an alert worth paging for
A capacity warning needs enough lead time and a reachable responder who can act. The alert review separates useful urgency from a missing response arrangement.
GuideIncident response
Who can authorize the next incident mitigation?
A rollback debate can be blocked by compatibility evidence or unreachable authority. A decision worksheet separates those problems and records the recovery checks.
ExplainerIncident response
Calculating mean time to detect from incident records
Worked incident records explain the MTTD formula, uncertain onset times and the effect of including failures first reported by customers.
CommentaryIncident response
Why blameless reviews investigate the conditions around a mistake
A mistaken target selection shows how an incident review can explain a responder’s choice and lead to a more useful interface improvement.
GuideIncident response
Writing a post-incident review the next responder can use
A delayed rollback connects the incident timeline to the information responders had, the compatibility question they faced and the follow-up that remains.