Write logs that support an investigation

Three errors appear at 14:04. One belongs to the export you are investigating; the others belong to unrelated work. Sorting by time puts them next to each other, but it does not make them part of the same story.

A useful log event gives you a way to follow the right story. Its fields identify the operation, the stage it reached and the work it belongs to. In this lesson you will turn an ambiguous message into a small event schema, then deliberately remove a field to see which question becomes harder to answer.

Use this fictional Dispatch event: 14:04 error request failed. The service processes API requests and background exports, so the line does not tell you which operation failed, which component observed it or how to connect it to the customer's report.

Give the event a stable meaning

Rewrite the event as structured data:

{
  "timestamp": "2026-01-01T14:04:00Z",
  "event": "export_write_failed",
  "service": "dispatch-worker",
  "region": "A",
  "job_id": "example-job-7",
  "trace_id": "example-trace-7",
  "error_type": "storage_timeout",
  "attempt": 2
}

These are illustrative identifiers, not a real trace payload. The event name identifies what happened; service and region preserve scope; job and trace identifiers connect related evidence; error type supports grouping; attempt distinguishes a first failure from repeated work.

The fields let you ask two different questions of one record. Match example-job-7 to follow this customer operation, or group storage_timeout to inspect the failure across many jobs. You no longer have to reverse-engineer the English sentence every time. The identifier follows the work; the error type lets you compare it.

Notice what the record still does not say. A failed write attempt does not establish that the final export failed: a later attempt may succeed. Keeping attempt lets you distinguish repeated work from repeated customer operations. A separate completion event gives the story an ending instead of requiring the reader to infer one from the last error.

The timestamp needs a timezone and an understood clock source. A collector's receipt time is not necessarily the event time. Delayed shipping can make an old failure appear beside current events unless the system preserves both.

For example, a worker may emit a failure at 14:04 while the logging pipeline is disconnected, then deliver it at 14:09. Sorting by receipt time tells you when the observation system caught up. Sorting by event time tells you when the worker reported trouble. Both are useful, provided the field names make the distinction visible. The five-minute difference is not five minutes of worker execution.

Correlate without over-collecting

Search for the job identifier and follow acceptance, worker start, failed write and completion. If an event lacks the identifier, record a correlation gap rather than confidently attaching the nearest line in time.

Keep the same operation, region and interval when linking a log search from a dashboard. A global search may return a similar error from another environment and silently change the explanation.

Do not log export contents, authorization headers or secret tokens to make correlation easier. Prefer purpose-specific identifiers with appropriate access and retention. Identifiers themselves can be sensitive, and removing one field does not automatically anonymize the record.

Practice with a three-event packet

Create synthetic lines for export acceptance at 14:03, worker start at 14:03:02 and write failure at 14:04, all with the same job ID. Add an unrelated successful job between them. Your query should retrieve the three related events without the unrelated job.

Now remove the worker-start job ID and compare the query results. Acceptance and failure still connect, but this particular start event no longer does. The unrelated success should remain excluded in both queries. That gap is the result of the experiment: repair the instrumentation rather than attaching the nearest line in time to make the record look complete.

Next add a completion event with the original job identifier. The query can now show acceptance, a failed write attempt and eventual completion while the unidentified start remains unresolved. One missing link does not require abandoning every supported part of the account. It requires stating exactly which part the records cannot establish.

Define what success means

Give the event schema, sample records and query to another reader. They should be able to tell which operation reached which stage without first learning your private log-message grammar. That is the useful result: less reconstruction before the next question can be asked.

A log records what code emitted, not an independent guarantee that reality matched it. Verify consequential states, such as downloadable output, at the boundary the user depends on.

Sources

OpenTelemetry logs data model documents timestamps and trace correlation. The events here are original synthetic examples.