Count the attempts behind an agent’s finished task

The agent finished one job. To do it, it may have made one model request, or three requests, four tool calls and a repair pass. The customer received one result either way. The system did different amounts of work.

Retry amplification makes repeated work visible. It does not mean every extra step is waste: reading a log and then querying a metric are different operations. A retry tries the same intended operation again after failure, rejection or uncertainty.

Give the task, operation and attempt different identities

Imagine an agent handling a support ticket. Reading its log and querying a metric help answer the same ticket, but they are different operations. If the metric query fails and the agent sends it again, that is another attempt at the same operation. The task ID follows the requested outcome; an operation ID follows each intended model request or tool action; a fresh attempt ID marks each execution. We can now tell a necessary next step from a repeated one without treating both as “more agent work.”

Suppose the agent resumes after an interruption. Did it repeat the metric query, or move on to another action? “Resumed” alone cannot answer that. The recorded identities have to show which happened. OpenTelemetry’s semantic-convention guidance distinguishes logical client calls from per-attempt calls, while its generative AI attributes include conversation and tool-call identities.

There is another question hiding behind the trace: did the customer get an acceptable answer? OpenAI’s Agents tracing documentation groups model responses and tool calls under a turn, with recorded duration, status, arguments and results where available. Those records help reconstruct what our agent did. They do not, by themselves, say that its answer satisfied the customer. The application still decides whether the consumer accepted the result.

Separate repeat load from delivery effort

For a task with one intended operation, divide physical attempts by eligible tasks. If every task reached its first attempt, 1.00 means no repeats and 1.27 means 27% more attempts than tasks.

For a multi-operation workflow, attempts per logical operation isolate retries more directly. Keep attempts per task too: it measures the total work the outcome required, including useful distinct steps.

Then divide attempts by accepted outcomes. Failed and unknown tasks consumed work without yet producing an accepted result, so keep their attempts in the total.

Take a fictional batch of 100 one-operation triage tasks, each attempted at least once. Seventy-eight succeed first time; fourteen succeed after one retry; five fail after three attempts; three stop after two attempts with an unknown destination result. That is 78 + 28 + 15 + 6 = 127 attempts and 92 accepted outcomes.

Retry amplification is 1.27 attempts per operation, and here also per task. Effort per accepted outcome is 127 / 92, about 1.38 attempts. The three unknowns remain neither success nor confirmed failure until reconciled.

Keep the operation type too. Another model response consumes time and cost. Another external write may create another effect. The total says work grew; the operation tells you which consequence grew with it.

Put the full cost beside the user’s wait

Look at one of the fourteen triage tasks that needed a second attempt. Its first attempt did not deliver the accepted result, but it still belongs on the bill. So do the attempts on the five failed tasks and the three whose result remains unknown. The batch cost includes every attempt, its tool charges and its monetary execution costs; dividing that total by the 92 accepted outcomes gives the cost per accepted result.

The provider’s bill needs one careful reading here. Its billed token categories and applicable rates tell us how to price each attempt, but those categories must not overlap in our sum. If cached-input tokens are already inside the input total, adding them again charges our calculation twice. The same applies to a reasoning subtotal already included in an input or output total. We are trying to find repeated work, not accidentally invent repeated charges.

Now the cheap-looking first response is less obviously a bargain. If it needs corrections or more tools, the finished task can cost more than one completed by a slower response without another pass. The comparison only holds if both results face the same acceptance rule. Relaxing that rule would make the work look cheaper without showing that the agent became more efficient.

The customer, meanwhile, waits for the task, not for our cost calculation. Its start-to-finish clock measures that wait. The trace lets us look inside it: generation, tools, pauses before another attempt, queues, and time spent finding out what happened to an uncertain result. Two tools running together may both use resources, but the overlapping part of their execution passes only once on the customer’s clock. Simply adding their durations would count that overlap twice; the task clock and the resource bill answer different questions.

Keep unknown outcomes visible until they change

Return to one of the three unknown tasks. Suppose its write timed out. The agent has no answer, but the destination might already have saved the change. Sending the write again without checking could repeat an effect that already happened. OpenAI’s recovery guidance advises checking saved work and completed actions before retrying a failed turn, respecting retry limits and stopping when the error changes.

We need to find that particular write at the destination. An appropriate destination read or a stable external identifier may let us locate it. Where the destination supports an idempotency key, repeated requests carrying the same key can be recognized as the same intended operation rather than treated as new work; the service’s supported behavior matters. These are ways to reconcile the uncertain result, not permission to assume success. Only evidence that supports acceptance or failure moves the task into that state. The timeout itself never gave us that answer.

That task also has an age. At ten seconds, checking the destination may still be ordinary progress; at thirty minutes, an operator may need to step in. A queue showing which unanswered tasks have waited longest tells us something a success percentage cannot: whose result is still missing an explanation.

Follow a rising ratio to the operation

Imagine the next batch arriving with a higher attempt count. Before calling the agent worse, we can place received work beside accepted outcomes, explicit failures and unknowns, then compare attempts per operation, per task and per accepted result with cost and waiting time. Is the same metric query repeating, or is the task doing useful extra work? Following the model, tool and error class gives us somewhere to look: provider overload, a slow MCP server, invalid arguments, conflicts or rejected output produce different trails.

Even a retry that the destination recognizes and prevents from duplicating an effect has used resources. And if the platform hides attempts or stable task identities, part of the trail is missing. Measurements at boundaries we control can still help, provided we say what remains unobserved. A row of completion notifications cannot supply an exact amplification ratio on its own.

The prompt-cache SLI guide and scheduled-work contract follow related questions about cost and completed work. Here, the useful change is already visible in our hundred tasks: 92 accepted results took 127 attempts, five tasks failed, and three still await reconciliation. The next person looking at the timed-out write can ask whether the destination saved it, rather than simply pressing retry. One delivered result is still one result. We can finally see the work, and the unanswered questions, that came with it.