AMD Helion separates agent execution from model inference

An agent handling a ticket may be waiting for records rather than using a GPU. AMD's October 12 Helion announcement treats those jobs separately: its open rack-scale reference design separates agent execution, model inference, storage, networking and general-purpose compute so they can meet different demands.

A hardware discussion asked what such a rack actually runs. AMD describes two answers: a CPU-only design for agent execution linked to external inference, and a mixed enterprise design with inference and supporting compute in the rack.

An agent alternates among planning, tool calls, API waits, data work, stored state and model requests. It is not one uninterrupted inference job. The slowest stage can move, so token speed alone cannot describe the needed capacity.

For a support request, more GPUs help the model's portion, not the wait for a ticketing API. Helion's separation allows resources to follow these stages, while the operator still needs to follow the entire request.

Helion splits the agent path into compute zones

Follow a ticket through AMD’s description. A CPU runs the agent program and asks the ticket system for its records. When those records arrive, the program sends the model a question. The CPU-only rack leaves that model work to separate infrastructure; the mixed enterprise rack can do it nearby. That rack combines EPYC CPU blades, up to 16 Instinct MI350P GPUs and general-purpose blades for databases, storage and technical computing. A blade is a server module in the rack.

The ticket’s model request reaches a token router, which chooses a cloud frontier model or an open-source model inside the rack. The choice changes who receives the records, how long the answer may take and what happens if that destination is busy. It can also change the answer itself.

After generation, the CPU-side agent still has to use the answer to finish the ticket. A fast model cannot retrieve a record that the ticket system has not returned. Following both waits shows which part of this arrangement needs more capacity.

The token router needs a service contract

Suppose the next program expects one queue label in a fixed JSON field. The normal model supplies it. During an outage, a replacement replies with a helpful paragraph instead. Someone can read that paragraph, but the dispatcher cannot find its label, so the ticket still goes nowhere.

Before enabling that replacement, decide which records it may receive and what response the dispatcher can use. In this hypothetical setup a local model sorts routine tickets and a cloud model handles a smaller complex group. A record for each ticket keeps the chosen model and destination, waiting time, result and applicable policy together. An unavailable cloud model then leads to a deliberate choice: wait, use a tested permitted replacement, or report that the ticket remains unclassified.

The inference compatibility suite can exercise that missing-label case along with required fields and errors. The same test should reject a route forbidden for those records. A fallback only helps the ticket when both the destination and its response fit the job.

Capacity planning follows the stage that is saturated

A CPU core runs instructions for programs such as the ticket agent. AMD lists up to 32,768 Zen 6 cores in the CPU-only rack, and up to 16 MI350P GPUs and 102 terabits per second of networking in the mixed design. Those design maxima describe hardware provided, not tickets completed.

Now let the ticket lookup slow down. Programs wait for records while GPUs have room to generate. Adding GPUs would enlarge the idle part of the path. Count arriving and finished tickets, then time the lookup, router and model stages; the growing wait shows where another resource might help.

The reverse can happen too: records arrive promptly while model requests accumulate. Even an endpoint that answers can have lost useful capacity. Device conditions and workload measurements help the infrastructure owner distinguish a power restriction from a software change or larger requests; latency alone does not identify which occurred.

In the mixed rack, CPU and GPU work also share power and cooling. Giving more of that allowance to agent execution can leave less for local generation. Sending model requests out frees some local demand but adds a network and provider to the ticket’s route.

Measure routing and execution as one task

Try one controlled failure first: make the ticket store slow. A request should either finish within its agreed wait or return an understandable unfinished state. Follow the saved ticket identity to see whether a later retry repeated an action already completed. That gives the trial a particular result to inspect.

Then vary the other stages separately: remove local inference, fill the execution queue or return a rate limit from the remote model. For each, compare what the dispatcher receives with the documented fallback. Keep retries, rejected tickets and their cost attached to the original request so the apparent success does not omit the work it displaced.

Who can investigate depends on the installation. A CPU-only operator may see the agent’s wait without access to the separate inference fleet; an enterprise may operate both. If the deeper stage is hidden, hand its owner the request and observed wait rather than inventing a hardware explanation.

Helion is a blueprint before it is a result

AMD calls Helion an OCP specification-based blueprint, with a planned contribution to the Open Compute Project. Partner configurations were under development for planned November 2026 availability. The announcement names partners but supplies no independent deployed-fleet benchmark.

The design addresses a real split between running agents and running models. The next evidence needs to show that split handling representative tasks, routing choices, failures and physical limits.

For the support request, success means a correctly handled ticket within its deadline. Bigger inference will not retrieve records sooner, and more execution will not drain the model queue. Separate zones help when their measurements let capacity follow the actual wait.