SRE KPIs for service quality and the work behind it

Consider a hypothetical quarter in which incident volume stays the same and recovery becomes faster as every difficult case goes to the same specialist. Customers spend less time waiting for recovery, but the specialist’s engineering work stops. The improvement is real, and so is the work displaced to achieve it.

A scorecard should show both faster recovery and the specialist work behind it. These are key performance indicators, or KPIs: measurements chosen to help judge important results. Keep service quality, response time and operating effort distinct so the review can decide whether to change staffing, share expertise or repair the recurring failure.

A number earns its place when it informs such a decision. If the quarter’s report records only speed, borrowing more of one specialist’s time can look like a system that became easier to operate.

Four questions for the scorecard

The table’s context is part of the measurement. Who counted as an eligible request? Where did the recovery clock stop? Leave those answers beside the result so the next reviewer can compare it without reconstructing the query.

Deployment frequency and change failure rate can add delivery context. Read them alongside customer-facing indicators: more delivery activity does not by itself establish that users received an acceptable service. The useful comparison connects what the team changed with what customers experienced and what it cost to operate.

Keep three kinds of reliability evidence visible

Read diagram description

A scorecard map, not a composite score. Service outcomes, response performance and workload describe different aspects of reliability and should remain separately interpretable. Diagram labels: Customer outcome: SLIs and service objectives; Response performance: Defined detection and recovery intervals; Operating workload: Interruptions, follow-up and specialist demand.

A faster average can conceal a changed incident mix

A mean can improve because the mix of incidents changed rather than because the response became better. Preserve incident counts and distributions, separate severities where useful and inspect long cases. Define “MTTR” locally as well: repair, recovery and resolution may end at different moments.

For an illustrative comparison using the same recovery endpoints, suppose eight minor incidents each take ten minutes and two major incidents each take sixty. The ten-incident mean is twenty minutes. In the next period, nine minor incidents and one major incident take the same respective times, producing a fifteen-minute mean. Neither class recovered faster; only their proportions changed. Reporting each class’s count and duration lets the scorecard show that difference before the lower overall mean becomes evidence for a response-process or staffing claim.

Mean time to detect, or MTTD, requires the same care with its starting point. When impact start is unknown, retain the uncertainty instead of substituting a convenient timestamp silently. A limited measurement remains interpretable when the limit is visible; an apparently precise replacement can change what the metric claims.

If specialist escalation shortens recovery, the scorecard should also show the engineering work it displaces. Together, those observations support a staffing or planning choice: change the specialist’s commitments, spread the diagnostic capability or fund work on the recurring failure. The response improvement is valuable, but its continuing cost belongs in the allocation decision.

One memorable incident does not overturn representative data on its own. Use it to test the scope of the claim and look for missing populations, then examine the wider evidence. Context should improve disciplined measurement rather than replace it with the most compelling story.

When classification or suppression moves the target

Reclassifying incidents can lower their count. Suppressing alerts can lower page volume. For each target, retain evidence about the harm those changes might conceal, such as customer-discovered failures or missed coverage. This helps distinguish a genuine improvement from movement outside the measurement.

A trend line cannot announce that its query changed. Put a new denominator or inclusion rule next to the chart. Otherwise yesterday’s number may count failures today’s number no longer sees, and two correct calculations can tell a false improvement story.

Give each row a decision and someone responsible for making it. An increase in customer-reported failures might lead the service owner to investigate a monitoring gap and bring back findings at the next review. The underlying counts and definitions remain available, while the scorecard records what the trend caused the team to do. The feedback-loop guide develops that link between a claimed improvement and its check.

Request success, budget use and detection time

The formulas below describe separate service and response indicators. In the faster-recovery quarter, those results would sit beside the record of interrupted engineering work before anyone used them to reduce staffing or claim greater efficiency. The calculations remain reproducible without turning their favorable values into a finding about workload.

Keep the same population and observation window within each calculation. Request errors cannot be converted directly to downtime minutes. A latency SLI also needs its threshold, since a technically successful request can still be too slow to qualify as good service.

The guide to defining service level objectives helps select the outcome being measured, and the error-budget worksheet records the allowance and policy. Google's SLO implementation chapter explains the relationship among indicator, objective and budget. Together these definitions let another reviewer reproduce the number before debating what it means for the plan.

What faster recovery means for the specialist’s work

The faster-recovery quarter presents two findings the team should be able to examine together: shorter incident durations and more specialist work. The on-call workload record supplies the second view, including off-shift help and follow-up that page counts miss. With comparable incident populations and timing definitions, the review can distinguish a better response process from one that has become more dependent on a single engineer.

If each difficult incident still needs the specialist, a tighter time target may simply claim more of that person’s day. Shared diagnostic knowledge or appropriate access may help; recurring defects may need prevention work. The scorecard should keep the customer’s improvement visible without erasing who paid for it.