Build the dashboard for the person waiting on a report
Start with what arrived, what finished and what is still waiting. The machine charts can explain the delay once the customer’s work is visible.
Guides, news and explanations about AI and reliable systems.
Not sure where to begin? Choose a short reading path, or browse at your own pace.
Start with what arrived, what finished and what is still waiting. The machine charts can explain the delay once the customer’s work is visible.
Start with work you recognize: one synthetic export. Three terminals show its acceptance, background completion and recorded steps without touching a production service.
Three errors at the same time may belong to three different jobs. Stable fields let you follow the right export without collecting its private contents.
Twenty failed exports can be 0.2% of the fleet and 20% of one region. Follow the counts to see what each metric can tell you and what remains outside its view.
A correct calculation can still end in a quiet inbox. Follow an export alert through its condition, route, message and recovery check.
A receiver, processor and exporter can all be defined without forming a working route. Follow one local trace, break a reference and put the route back together.
A restart resets the lab’s in-memory counters. Follow that reset through a Prometheus query, then build panels that distinguish failed simulated jobs, inactivity and missing measurements.
The slow trace is useful. The command, restart and comparison that explain it are useful too. Keep them together as you follow a local export from acceptance to completion.
A background job can succeed while its connection to the request disappears. Break and restore one local handoff to see the difference.
Send a local service’s telemetry to the wrong port, then compare its recorded job state with missing spans. Repairing telemetry delivery does not bring every old trace back.
A job ID helps you find one export. Put it on every metric and it can create a new storage problem. Keep the count and the individual story in useful places.
Two lookups overlap, so adding their durations counts some waiting twice. Read the shared timeline to find both the overlap and the time still unexplained.