How to evaluate incident-response agents
Build repeatable cases and check recovery independently, with a runnable Python grader.
Practical guidance for SRE and platform teams operating reliable services and production AI. Independent analysis, field guides, and working resources.
Selected reading
Build a prompt-caching pilot that counts retries, failed work and unknown charges, with a local calculator and sample ledger.
A practical pilot for separating cache savings from the cost of completed work.
Read the article
Original writing / Newest first
Build repeatable cases and check recovery independently, with a runnable Python grader.
Check agent status, terminal persistence, and recovery before adopting Herdr.
Separate detection from response and find delays that an improving average can conceal.
Define the action, contain its scope, and verify what changed at the destination.
Use instrumentation and context propagation to connect the evidence in an investigation.
Clarify ownership across shared platforms, service reliability, and incident response.
Releases, research & operational impact
Inference infrastructureNews analysis
NVIDIA’s September 21 demonstration accelerates one video-generation workload across eight GPUs. A service decision still needs the same offered load and completed-work count.
Read the analysisAgent evaluationNews analysis
NVIDIA’s new SGLang benchmark finds patches that pass local checks but fail a running server. Its result is a reason to examine what a green test actually covers.
Local AI agentsNews analysis
Google adds local Gemma and LiteRT workflows. Running the model on a workstation still leaves a separate decision about what its tools may read, change or send.
From reading to doing
Incident response
Work through a database connection example, from the first symptom to a verified recovery.
Reliability planning
Define the service objective, calculate the allowance, and agree when a release needs to wait.
Observability
Start with one important request path. Check propagation, sampling, and the gaps in your evidence.