Opens in a new tab
News

Moving a log classifier to Haiku 5.5: where did the answer go?

In brief

A worker times out, a queue grows, and the new classifier returns no usable category. Following that one event reveals what Haiku 5.5 changes beyond the price.

7 min read

Is this the right starting point?
Before you read
Basic model-API request, response-parsing and log-classification concepts.
What you will take away
Plan a Haiku classifier migration that handles request changes, incomplete responses and unsupported diagnoses separately.
Sources
A terracotta gate separates paper record strips into colored paths, with one loose strip outside.
On this page4 sections

Imagine opening the morning queue and finding an event with no category. A worker timed out while contacting storage. Work is piling up behind it. Yesterday the classifier would at least have put a label on the event; after the model change, the label is missing. The first question is surprisingly practical: did the new model ever get a valid request?

Anthropic’s October 7 Haiku 5.5 announcement makes the move tempting for high-volume classification and extraction. It lists $0.10 per million input tokens and $0.50 per million output tokens through 100,000 prompt tokens, rising to $0.50 and $2.50 above that threshold. The company reports average savings of about 75% over Haiku 4.5 after changes in token use. That is Anthropic’s comparison, not a saving measured by AIOpsSRE.

The lower rate is useful if the event still reaches the right person with an accurate account of what happened. A confident but mistaken diagnosis can consume the responder’s afternoon. A blank result can leave the event waiting unnoticed.

Our imagined classifier has three jobs along the way: send the request, find the completed answer, and decide whether its category fits the logs. Haiku 5.5 changes enough of that route that the missing label could belong to any of those steps. Following the event is more revealing than changing the prompt at random.

The worker waited. What was storage doing?

Start with what the two logs actually give us: the worker timed out while contacting storage, and the queue’s oldest work is getting older. There is no observation from the storage endpoint. We can picture the worker waiting; we cannot yet see what was happening at the other end.

“Storage request timed out” fits those records. “Storage was down” goes further than they allow. A useful classification cites both logs, describes the timeout and leaves the responder a specific question about the endpoint at that time. Adding a mandatory cause field to the form does not supply the missing observation.

That distinction needs to survive the model change. A category may be on the allowed list and still be the wrong conclusion. We want the classifier to hand over the gap honestly, not fill it because the response looks tidier that way.

An opening brace can stop the request

Look first at the request the old client sends. It may end with an assistant message containing an opening JSON brace, a small nudge asking the model to continue the answer from there. Haiku 5.5 rejects this technique, called prefill. The Messages API migration guide says to finish with a user message instead. It also requires removing temperature, top_p and top_k, controls for selecting the next token, or piece of generated text. Manually enabled thinking with a budget_tokens allowance must give way to adaptive thinking, which lets the model choose how much intermediate reasoning to use.

The application still needs a predictable answer to read. Where supported, structured output or tool fields with a fixed set of choices can replace the old brace trick. They help constrain the answer’s shape. The application has another job afterward: check that each cited log exists and that it supports the chosen category.

If the request fails, keep that failure attached to our original event. It has not been classified. Supplying a fallback category just to keep the queue moving would hide the very item that needs attention. Leaving it visibly unresolved gives a responder or retry worker somewhere honest to begin.

The answer may be behind a thinking block

Now suppose the request goes through, but the category is still missing. An older parser may read the first content block and stop. With Haiku 5.5, that can mean it has read intermediate reasoning and never reached the answer. The behavior documentation describes adaptive thinking as the default, with thinking blocks able to precede the text response. Read the block types and find the completed answer. Thinking also consumes the max_tokens allowance, so that allowance can run out before the category arrives.

The same prompt also takes more tokens than before. Anthropic reports roughly 30% more for the same text than Haiku 4.5, with the amount depending on the content. Recount the real prompts and leave room for the output. The category we want may be one line; the work before that line still needs room.

Even HTTP 200, the response code that normally signals a successful request, does not settle whether we have a classification. The structured-output documentation permits a safety refusal with that code and a response outside the requested schema. Reaching the token limit can leave an unfinished answer, too. Neither belongs in the completed pile.

Here is how our example application could keep those cases apart:

What came backWhere the event belongs
The request or client failed.Still waiting for classification.
The model refused, or the output stopped early.Unresolved, with a specific handling path.
The answer has the right shape but invents a cause.Rejected by the check of what the answer means.
The answer fits the logs and keeps their uncertainty.Eligible for the application to accept.

These are states we are proposing for the application, not labels built into Haiku. Their value is in the next action. A broken request needs a client repair. An unfinished answer needs different handling. A completed but unsupported diagnosis needs to fail the classification check, even if every field is present.

Four proposed application checks separate request success, complete response, valid shape and evidence support. Client errors, refusal, truncation, invalid structure and unsupported causes remain unclassified, unresolved or rejected.
A conceptual route through the hypothetical classifier: request success, a complete response, valid structure and support in the logs. The proposed states are not built-in Haiku labels or measured outcomes.

Bring back the same event, then change one thing

Return to the worker waiting on storage. Run the comparison with that missing endpoint observation still missing; neither candidate has enough to declare storage down. Then supply a second case with independent endpoint evidence, so the comparison also rewards a conclusion that is justified. This evaluation approach holds the source material and acceptance conditions steady while the model changes.

Follow where each event stops. If the old opening brace causes a request error, more reasoning effort will not fix the client. If the parser reads the thinking block and misses the answer, a poor classification score can send an engineer after the wrong problem. Recording the step that failed saves that detour.

Once that route works, adjust how much work the model does. Anthropic’s prompting guide calls effort the main cost/quality control. It also notes that turning thinking off while asking for structured JSON can skip a required tool call. In our example, the task might require looking up the storage endpoint: a neat category without that lookup has missed part of the job. Check that the lookup occurred. Refusals need their own route as well; the guide says identical retries usually repeat them, so another trip around the same request is unlikely to resolve the event.

Managed Agents users reach the model through a different setting. That integration’s migration guide describes a version change as typically one field in the agent definition, taking effect in the next new session. The Messages API request repairs above should not be mistaken for that separate migration procedure. A shorter configuration change still leaves the team responsible for checking what the new model does.

There is one more decision after the answer is right: how to supply the service with enough capacity. Haiku 5.5 has no Priority Tier. Capacity commitments therefore need separate planning; a successful classification test cannot make that serving arrangement for you.

Back in the queue, the useful result is modest and specific. The responder can see that the worker timed out while contacting storage, that work is growing older, and that the endpoint’s condition is still unknown. The paired case with endpoint evidence checks whether the classifier can go further when the logs justify it. That is the account worth preserving at the lower price: enough to help the next person act, without quietly supplying a cause nobody has observed.

Source context

Source links appear within this article. Read them alongside the author’s analysis and evaluate the guidance against your environment.

Report an error or outdated detail

More on Anthropic

All Anthropic coverage

Keep reading

Where to go next