How OpenTelemetry connects a request across services

Three graphs can be right and still leave you guessing. In this illustrative checkout investigation, the API is slow, a queue is growing and the database shows waiting. The missing fact is not another measurement. It is which of those observations belongs to the same request.

Following a checkout through a payment handoff shows how the pieces fit together. A trace links the operations, logs can explain individual events, and metrics show whether the delay is widespread. Their value depends on preserving enough context for a responder to move among those views.

Three signals describe different parts of the same work

OpenTelemetry grew from the merger of OpenTracing and OpenCensus. It provides APIs, SDKs, conventions and tools for generating and exporting telemetry, including traces, metrics and logs. Storage and investigation still need a backend and analysis interface; OpenTelemetry does not itself supply that entire destination.

A trace follows one piece of work through operations called spans. In checkout, one span might cover the incoming request and another the call to the payment service; their timing and relationship help show where that request waited. Logs record individual events, such as a payment attempt returning an error. Metrics summarize measurements across requests and time, such as the share of checkouts that failed. A sampled trace can explain one slow checkout, while the aggregate helps establish how widespread the problem is.

This combination also applies to work without a shopping cart. A document-processing job or an internal API request can cross several services before producing its result. The useful connection is the same: follow a particular operation, inspect the events inside it and compare it with the wider population. Collecting all three signal types earns its cost when those views answer questions that one alone leaves open.

The most useful first instrumentation is often the missing connection that would change an investigation or recovery choice; further collection can follow the questions that remain. For checkout, the application-to-payment connection may be the missing evidence that determines which system needs attention. It is a more specific starting point than a general wish for additional dashboards.

An unfamiliar system can justify broader baseline collection during discovery. Keep that phase bounded and revisit its cost when recurring questions emerge; focusing instrumentation should not mean collecting only evidence that confirms an existing expectation.

How the payment service knows which request called it

A span is a record of one operation: its name, start and end times, identifying context and useful attributes. It might describe the checkout request or the payment call inside that request. A trace groups the related spans so a backend can display the journey rather than a collection of unrelated timings.

The missing ingredient is an identity that crosses the network. In this simplified example, checkout starts trace T and records its outgoing payment call as span P. When it calls payment, instrumentation includes that trace identity and parent span identity in the request. Payment extracts the context and records a new span Q in trace T, with P as its parent. The letters here are teaching labels, not real identifier formats.

Now the payment span has an address in the story: it belongs beneath the outgoing call that started it. The records can arrive separately because the shared trace and parent identities preserve that relationship. Without them, a backend may hold perfectly good timings for checkout and payment yet have no dependable way to join this checkout to this payment. OpenTelemetry’s context-propagation and trace documentation explain how instrumentation carries that identity across the boundary.

In a hypothetical checkout incident, the application trace stops before a payment call. If both services record spans but the context never crosses that call, each side can look healthy as a telemetry producer while the backend cannot connect their records. Propagating the context repairs that relationship. If payment was never instrumented, propagation alone cannot invent the missing span.

Logs can carry the same trace and span context, letting a responder move from an unusually long payment operation to the events recorded inside it. Metrics answer a wider question, such as how many payments are slow; supported exemplars can link a measurement to an example trace. Each view contributes something different. A trace follows this request, a log records an event, and a metric summarizes a population.

The connected trace can strengthen a causal hypothesis by showing that one operation called another. It does not by itself establish that a nearby deployment caused the delay. Nor should nested span durations simply be added: a parent often includes time spent waiting for its child. The useful question is where the request spent its elapsed time, including work that ran in parallel.

How telemetry reaches the investigation backend

The Collector can receive, process and export telemetry, separating application instrumentation from backend delivery. Depending on its components and configuration, it can batch, filter, enrich or route signals. Direct application export is also possible, so a Collector is an architectural choice rather than a prerequisite for every deployment.

That intermediary gives a team useful control and another failure boundary to operate. Refused or dropped data, export failures, queue pressure and memory use help identify where evidence is being interrupted. A test with the backend unavailable also reveals how the configured SDK and Collector balance telemetry loss against interference with application work. The right balance depends on what the service can tolerate.

What travels through that pipeline matters as much as where it goes. Remove sensitive fields before export. Also consider cardinality: the number of distinct combinations a metric’s labels create. A service-name label groups many requests under one name; a unique request-ID label can create a separate series for every request. The identifier that helps locate one trace can therefore make an aggregate metric expensive to store and query. Sampling needs its own purpose too: retaining unusual failures and estimating population rates are different jobs.

A known test event makes the path inspectable at emission, receipt and export. If it vanishes, the comparison narrows the missing connection rather than sending application and telemetry owners to search everywhere. The warning about that collection failure also needs a route to a responder that survives the affected export path. Otherwise the missing-data alarm can disappear with the data.

A telemetry path has its own failure boundaries

Read diagram description

A common architecture sends traces, metrics and logs through a Collector to a backend. The Collector is optional. Check emission, receipt and export; monitor collection failure through a path that can still reach an owner. Diagram labels: Instrumented application: Traces · metrics · logs; stable service identity; SDK export: Generate and propagate appropriate context; Collector (optional): Receive → process → export; expose drops and queues; Telemetry backend: Store and query evidence; check freshness; Responder: Connect the user symptom to supporting records.

Following a slow checkout through the pipeline

Choose a test journey and the uncertainty to resolve, such as whether checkout time is spent waiting for payment authorization or processing retries. Give services stable names, propagate context across the relevant handoffs and create one deliberately slow request in a test environment. That request provides a known event against which the resulting records can be compared.

Follow it through the payment boundary to the investigation backend. Its trace, related logs and aggregate measurements should present a consistent account, with each signal contributing what it can establish. Then interrupt the telemetry path in the test environment and confirm that the absence of evidence becomes visible. A slow operation and a missing observation about that operation need different responses.

The instrumentation guide develops the rollout. The payoff is the moment the slow-checkout graph leads somewhere: to the payment operation, its events and a question you can investigate. Emission, receipt and export checks then help locate a break in that evidence path. If the payment span is missing, the next job is to find out whether the call never happened or its telemetry was lost. OpenTelemetry has made that question easier to ask precisely; it has not answered it by wishing another graph into existence.