OpenTelemetry Kubernetes attributes v1: what to test during migration

In this illustrative upgrade, the engineer finds the failing agent task’s trace but cannot find its Kubernetes workload or responding team. Records are still arriving. Their address information has changed, and the dashboard and paging route no longer know where to take the investigator.

OpenTelemetry’s September 16 announcement says the Kubernetes attributes processor has reached v1.0.0, meeting stable criteria for testing, documentation, benchmarking and telemetry stability. The same announcement warns of breaking changes and points existing users to migration guidance. Those stability criteria cover the processor; existing queries and routing rules may still need changes to understand what it emits.

The processor attaches details such as namespace and workload to the record, rather like an address on a parcel. Those details connect a slow call to the service that produced it. The upgrade has to preserve the meaning used by the people and queries receiving the parcel.

How a recorded call connects to a Kubernetes workload

The processor documentation describes how the Collector associates telemetry with pods and adds Kubernetes metadata. It can establish the association from connection information or resource attributes; configuration and Kubernetes API access determine which metadata it can obtain.

That lets a responder connect a slow tool call with its worker workload and nearby infrastructure signals. The relationship helps distinguish a wait at an external model service from a wait in the service coordinating the request. It adds context to telemetry that already exists. It does not invent missing model-call instrumentation or determine whether the answer was correct.

Follow one namespace field after collection. A dashboard filters on it, an alert groups by it and a paging rule chooses a team with it. A renamed field can break all three while ingestion stays healthy. The upgrade therefore includes those consumers, not just the processor binary.

The migration is complete when the emitted attributes still let investigation queries find a known failing workload and ownership routes reach its responding team. That test reaches the responder’s job, rather than ending at ingestion. Compatibility work can focus on fields that affect investigation or control; preserving unused attributes indefinitely adds maintenance without protecting a decision.

Stability applies to this processor

This is a component milestone in the broader Collector stability effort, tied to stable Kubernetes semantic conventions. It does not announce the same maturity for every Collector component. Keeping that scope precise makes it easier to identify which guarantees and migration requirements apply to the part being changed.

Begin with the intended Collector distribution and version, then compare its configuration with the version-linked migration guidance. An internal query may depend on an incidental attribute as strongly as on a supported one. Documenting that dependency lets the team choose a replacement instead of discovering it through a broken investigation.

The OpenTelemetry overview explains the surrounding pipeline. For this upgrade, the narrower question is which connection inside that pipeline has changed and what reads the result.

Test the metadata join after an upgrade

Read diagram description

Illustrative telemetry association. Receiving a span is not enough: its Kubernetes identity must still lead the responder to the intended workload after migration. Diagram labels: Agent span or log: Telemetry enters the Collector; Pod association: Match the record to the intended pod; Kubernetes attributes: Attach namespace and workload metadata; Dashboard or alert: Confirm filters and ownership links still work.

Tracing a known task through the upgraded pipeline

Choose a staging agent workload with known namespace, deployment and pod identity, and run a recognizable task whose model and tool calls are already instrumented. Capture enriched telemetry before and after the upgrade. Comparing resource attributes shows what changed; running the actual responder queries against both records shows what the change means.

In a hypothetical migration, the namespace attribute used by a paging route is renamed. The trace arrives normally, but the failed task is sent to an unowned queue. A successful exporter check cannot expose that handoff failure. Following the emitted attribute through the route to its expected recipient can.

To see the size of a gap, count records expected from Kubernetes workloads and how many carry the identifiers the queries require. Keep that expected population explicit. Adding unrelated external-service records to the denominator would change the number without revealing a processor failure.

The same test should survive a pod restart, so a query finds the intended workload rather than only the instance present during setup. Include the real gateway path too. A downstream Collector may see the gateway as its network peer rather than the application, which changes what connection information can establish. These checks follow the same association under the conditions in which responders will need it.

Queries and paging routes move with their attributes

The useful migration record combines before-and-after telemetry, consuming queries, expected results and a rollback procedure. Alert grouping belongs in it alongside dashboards because both can consume the same metadata and fail differently when it changes.

For each changed field, identify the consumer that has to migrate with it. Temporary support for both forms can make the transition easier, provided its removal has an owner once those consumers are ready. That avoids turning a compatibility bridge into permanent unexplained complexity.

Run the recognizable task again after a pod restart and through the gateway. Do the queries still find its workload and the page reach its team? Save those attributes, queries and results for the next upgrade. They explain far more than “telemetry kept arriving.”