On this page5 sections
An agent answering a ticket may spend its time waiting for records, not using a GPU. AMD’s October 12 Helion announcement separates those jobs. Its open rack-scale reference design divides agent execution, model inference, storage, networking and general-purpose computing so resources can meet different demands.
A hardware discussion asked what such a rack actually runs. AMD describes two arrangements: a CPU-only design running agents with inference elsewhere, and a mixed enterprise design with inference and supporting compute in the rack.
An agent moves between planning, tool calls, API waits, data processing, saved state and model requests. It is not continuously generating tokens. As the bottleneck moves, token speed alone stops being a useful description of required capacity.
More GPUs may shorten the model stage of a support request. They will not make the ticketing API return records sooner. Helion separates resources around those stages; the operator still has to follow the complete request.
Follow the ticket between execution and inference
Follow a ticket through AMD’s description. A CPU executes the agent and requests ticket records. Once they arrive, it sends a question to a model. The CPU-only rack leaves inference to separate infrastructure; the enterprise rack can perform it nearby. That mixed design combines EPYC CPU blades, up to 16 Instinct MI350P GPUs and general-purpose blades for databases, storage and technical computing. A blade is a rack’s server module.
A token router chooses whether the model request goes to a cloud frontier model or an open-source model inside the rack. That choice changes who receives records, expected waiting time, behavior when the destination is busy and potentially the answer itself.
The CPU-side agent must still use the generated answer to finish the ticket. A fast model cannot fetch records the ticket system has not returned. Following both waits identifies which part needs more capacity.
Give a routed answer a contract the dispatcher can use
Suppose the dispatcher expects a queue label in a fixed JSON field. Its usual model supplies one. During an outage, the replacement writes a helpful paragraph instead. A person can understand it, but the dispatcher cannot locate the label. The ticket still goes nowhere.
Before enabling that replacement, determine which records it may receive and which response the dispatcher can consume. In this hypothetical setup, local inference sorts routine tickets while a cloud model handles a smaller complex group. Keep chosen model and destination, wait, outcome and applicable policy with each ticket. Cloud unavailability then calls for a deliberate decision: wait, use a tested permitted alternative or report the ticket unclassified.
The inference compatibility suite can test the missing-label case, other required fields and errors. It should also reject routes forbidden for those records. A fallback helps only when both its destination and answer are suitable.
Find the busy stage before adding hardware
A CPU core executes program instructions, including the agent’s. AMD lists up to 32,768 Zen 6 cores for the CPU-only rack, and up to 16 MI350P GPUs with 102 terabits per second of networking in the mixed design. These are hardware design maxima, not a count of completed tickets.
Let the lookup slow down. Agent programs wait for records while GPU capacity remains free. More GPUs would enlarge the idle part. Count arriving and completed tickets and time the lookup, router and model stages. The growing wait points to the resource worth investigating.
The reverse is possible: records arrive quickly, but model requests accumulate. An answering endpoint may still have lost useful capacity. Device and workload measurements can help distinguish power restrictions, software changes and larger requests. Latency alone cannot identify which occurred.
CPU and GPU work share power and cooling in the mixed rack. More allowance for execution may leave less for local generation. Routing inference outward reduces some local demand while adding a network and external provider to the route.
Keep failure and recovery attached to the ticket
Start a hypothetical controlled trial by slowing the ticket store. The request should finish within its agreed wait or return an understandable unfinished state. Follow its saved identity to see whether a later retry repeats an already-completed action.
Test the other stages separately: remove local inference, fill the execution queue or have the remote model return a rate limit. Compare the dispatcher’s result with the documented fallback. Keep retries, rejected tickets and cost attached to the original request rather than excluding work that apparent success displaced.
Visibility differs by installation. A CPU-only operator may observe the agent’s wait without access to the separate inference fleet; an enterprise may operate both. When the deeper stage is hidden, give its owner the request and measured wait rather than inventing a hardware diagnosis.
Treat the blueprint as a starting point
AMD describes Helion as an OCP specification-based blueprint, with a planned Open Compute Project contribution. Partner configurations were in development for planned November 2026 availability. The announcement names partners but supplies no independent deployed-fleet benchmark.
The separation between running agents and running models is useful. Representative tasks still need to show how the design handles routing, failures and physical constraints.
For the support request, success is a correctly handled ticket within its deadline. More inference does not retrieve the records sooner; more execution does not drain a model queue. Separate zones help when measurements let capacity follow the actual wait.
Source context
Source links appear within this article. Read them alongside the author’s analysis and evaluate the guidance against your environment.
Report an error or outdated detail