The AI answer is one paragraph. Where did the tokens go?

Imagine asking an assistant to turn ten records into one paragraph about whether a service can be rolled back. The paragraph arrives. It is short, neatly written and missing the exception that decides whether rollback is allowed. Now you are opening the ten records yourself.

This hypothetical task has two bills: the model's work and the work left for you. To understand either, follow the whole attempt, including repeated requests and the time spent checking the result. Counting the words in that tidy paragraph will miss most of the story.

The model reads more than the question you typed

Your question might fit on one line. Behind it, the application can send instructions, descriptions of available tools, passages from the ten records, earlier conversation and results from tools it has already called. That is the input the model receives, not just the words in the question box.

Before processing text, a tokenizer turns it into units called tokens. A token can represent a word, part of a word, punctuation or another sequence. Different tokenizers and languages divide text differently, so a word count is only an estimate. It cannot establish the charge or tell you for certain whether a request fits the model's context limit, the amount of material it can work with at once.

The answer adds output tokens. Reasoning, images and audio can have their own accounting rules, depending on the model interface. Its reported usage fields are the place to find those categories. Give the overall task an ID and keep each call's usage with it, including retries. Then the one-paragraph answer stays connected to everything that happened before it.

Four attempts belong on the same bill

Suppose an attempt sends 12,000 input tokens and produces 1,000 output tokens. At hypothetical rates of $2 per million input tokens and $8 per million output tokens, the input costs $0.024 and the output costs $0.008. Add them and one attempt costs $0.032. These are teaching rates, not vendor prices.

The arithmetic is simply count multiplied by rate, divided by one million. But that describes this uncached text example. Real usage may include saved input that the service can reuse, called cached input. Anthropic's cache fields, for example, distinguish uncached input from tokens written to the cache and tokens read from it. Charge each category at its applicable rate, once. Follow the interface's definitions for reasoning usage too, and add relevant tool or storage charges separately.

Now let the task take four identical attempts. The model spend is $0.128 before other charges, even if you accept only the last answer. The first three attempts do not disappear when the fourth gets it right.

Count every attempt in the task

Read diagram description

Hypothetical rates from the worked example: 12,000 input tokens at $2 per million plus 1,000 output tokens at $8 per million cost $0.032 per attempt. Four identical attempts cost $0.128 before other charges. These are not vendor prices. Diagram labels: Input: 12,000 tokens: 12,000 × $2 / 1,000,000 = $0.024; Output: 1,000 tokens: 1,000 × $8 / 1,000,000 = $0.008; One attempt: $0.032: Input cost + output cost; Four attempts: $0.128: Include failed attempts and retries in the task ledger; Task outcome: Useful accepted result, or unresolved work.

Shortening the input does not necessarily change output, cache pricing or tool costs. Keep those amounts separate while comparing the complete task. Keep failed attempts in the record as well. If an error returns no usage, mark the charge unresolved until it can be reconciled with billing. A blank amount is an unanswered question, not a free request.

The first word arrives before the work is finished

The assistant starts replying quickly. That feels promising, until it pauses to fetch another record. Time to the first token, time to the complete answer and time waiting for tools are different measurements. Queues, document retrieval and throttling can hold the task up too. Removing text from the prompt will not necessarily shorten whichever stage is making you wait.

Long conversations bring another constraint. The application may need to fit the growing history inside the model's context limit. Depending on the interface, an oversized request might fail, lose some text through truncation or trigger a summary. Try a non-sensitive test case near that limit and inspect what is actually sent and what happens next; the behavior should not be guessed from the size of the original conversation.

A rolling summary can let the conversation continue while leaving our rollback exception behind. Keep important conditions and links to their sources outside that summary, then check whether the assistant can recover them when needed. A routine status update may work with little context. An investigation into a difficult rollback decision may need more room. Giving both the same hard cap can remove the very passage that makes the harder answer useful.

Decide how much work this answer deserves

For the ten-record task, set suitable limits on model calls, elapsed time, output and the size of tool results before the assistant begins. If it keeps circling without finding anything new, stop the loop with its attempts and unresolved questions recorded. The next responder can pick up that partial result rather than pay to discover the same dead end.

Then compare the original and shortened context on the same case. Record all attempts and the time spent checking sources, and look for the rollback exception in the result. The prompting guide develops this check of what survives into the answer. Use current documentation for the particular model interface when calculating charges; the Opus 4.6 announcement supplies historical capability and context information, not a substitute for checking today's purchase terms.

If model spend and source-review time both fall while the necessary condition survives, the saving reaches the team. If one falls and the other rises, you can decide whether that exchange is worthwhile. You still want the short paragraph. You also want to close the ten records knowing why the answer applies.