Measuring retry amplification in AI agent workflows

An AI agent can finish one task after one model response, or it can finish the same task after three model responses, four tool calls, a timeout, and a repair pass. Both runs may appear as one completed job in a dashboard. They are not the same workload.

Retry amplification measures that hidden multiplication. It connects the logical work a user requested with the physical attempts the system made, then keeps accepted outcomes separate from failed and unresolved work. That distinction matters because a retry can consume tokens and tool capacity, repeat a side effect, or move the failure into a downstream system while the final agent status still says completed.

A completed agent task can hide several model and tool attempts, so cost and latency must be measured against accepted outcomes rather than completion notifications. Preserve one logical task ID across all physical attempts, then report attempts, cost, and latency per accepted outcome while keeping failed and unknown work separate.

Agent cost becomes useful operational evidence when an attempted task can be connected to its accepted outcome. Evaluate attribution across retries, rejected results, and correction work before treating a lower token bill as an improvement.

Count logical work and physical attempts separately

Start with two identifiers. A logical task ID follows the user's requested outcome across the whole run. An attempt ID identifies each physical model call, tool call, or resumed turn used to reach it. If the same tool call is retried, the task ID stays fixed while the attempt ID changes.

This is close to a familiar tracing problem. The OpenTelemetry guidance for semantic conventions asks instrumenters to state whether a client span represents a logical call or a physical per-attempt call. Its generative AI attributes also define conversation and tool-call identifiers. The names will evolve, but the measurement boundary is stable: one user request can contain several attempts, and the trace must preserve both levels.

OpenAI's Agents tracing documentation groups model responses and tool calls beneath a turn and records duration, status, arguments, and results when available. That trace can show the physical work. Your application still has to decide which result was accepted by the consumer.

Use two ratios, not one success rate

The first useful ratio is attempts per logical task. Divide all physical attempts by all eligible tasks. A value of 1.00 means every task used one attempt. A value of 1.27 means the workflow generated 27 percent more attempts than the task volume alone would suggest.

The second ratio is attempts per accepted outcome. Divide the same attempt count by outcomes that passed the consumer's acceptance check. This ratio grows when tasks fail, end in an unknown state, or produce results that require correction. It exposes work that a simple completion rate can hide.

Retry amplification divides physical model and tool attempts by logical tasks, while a second ratio divides the same attempts by outcomes accepted at the consumer boundary.

Consider a hypothetical batch of 100 support-triage tasks. Seventy-eight succeed on the first attempt. Fourteen succeed after one retry. Five fail after three attempts, and three stop after two attempts with an unresolved destination state. The workflow made 127 attempts, so retry amplification is 1.27 attempts per task. It produced 92 accepted outcomes, so it required about 1.38 attempts per accepted outcome. The three unknown results do not count as either success or failure until the destination is reconciled.

Keep model generations and external tool attempts available as separate dimensions. A reasoning retry consumes different capacity from a repeated database write. If they are collapsed into one counter, the total may explain a cost increase without explaining the operational risk.

Connect cost and latency to the accepted outcome

Once the task and attempt relationship is intact, cost becomes easier to interpret. Sum input tokens, output tokens, cache reads, tool fees, and execution time across every attempt for the logical task. Then divide by accepted outcomes, not merely completed turns.

A cheaper model call can make the complete workflow more expensive if it causes more tool retries or correction work. A slower first attempt can be the better choice if it produces a result the consumer accepts without another pass. Neither conclusion is universal. The point is to compare the complete task under the same acceptance rule.

Latency needs the same treatment. Report user-visible task latency alongside time spent in generations, tool calls, backoff, queues, and reconciliation. Parallel work should not be added as if it were sequential elapsed time. The trace describes where work occurred; the task clock describes how long the user waited.

Treat unknown outcomes as their own state

A timeout is not proof that an action failed. The destination may have accepted the change before the acknowledgment was lost. OpenAI's agent recovery guidance tells clients to check saved work and completed actions before retrying a failed turn. It also recommends honoring retry limits and stopping when the error changes.

That recovery behavior should appear in the metric. Record an outcome as unknown when the caller cannot establish whether the side effect occurred. Reconcile it through the destination's read path, an idempotency key, or a stable external identifier. Only then move it to accepted or failed.

Unknown work should also have an age. An unresolved write that is ten seconds old may still be inside a normal reconciliation window. The same state after thirty minutes may need an operator. Count and alert on unknown outcomes by age rather than forcing them into a success percentage.

Build the dashboard around decisions

A useful operating view begins with eligible logical tasks, accepted outcomes, explicit failures, and unknown outcomes. Add attempts per task, attempts per accepted outcome, task latency, cost per accepted outcome, and the distribution of attempts by model, tool, and error class. Keep retries that were prevented by idempotency visible because they still consumed work even though they avoided a duplicate side effect.

Segment before blaming the model. A rising amplification ratio may come from a provider overload, a slow MCP server, invalid tool arguments, a destination conflict, or an application acceptance check that rejects otherwise completed turns. Each cause has a different owner and a different repair.

The boundary is equally important. When the agent platform does not expose physical attempts or stable task identifiers, you cannot calculate retry amplification precisely from completion notifications. Instrument the application and tool boundary you control, mark the remaining work unknown, and avoid presenting the partial ratio as complete.

The prompt-cache SLI guide shows why missing usage cannot be treated as zero. The scheduled execution contract explains how to reconcile a consequential run before retrying it. Retry amplification connects those ideas to one operating question: how much work did the system perform for each outcome the consumer could actually accept?