AMD's Helios Rack Takes the Fight to NVIDIA at the System Level

AMD announced Helios on July 23, and the headline numbers are hard to ignore. The rackscale system pairs 6th Gen EPYC “Venice” 9006 Series CPUs with 5th Gen Instinct MI455X GPUs and Pensando networking on a UALoE fabric, connecting 72 GPUs in a single scale-up domain with 260 TB/s of bandwidth. Against NVIDIA’s Vera Rubin NVL72 rack, AMD claims up to 15% more AI compute, 50% more HBM capacity and 50% more scale-out bandwidth. The system delivers 2.9 exaflops of dense FP4 compute, 31 TB of HBM4 and 1.7 PB/s of aggregate HBM bandwidth.

The GPU itself is a step change. On DeepSeek-V4-Flash, AMD says the MI455X delivers up to 34X higher token throughput at high interactivity and up to 18X lower token cost versus the previous MI355X. Those are AMD Performance Labs figures, measured on pre-production hardware under specific conditions, and the footnotes carry the usual caveats. Still, the direction is clear: AMD is no longer selling silicon against NVIDIA’s silicon. It is selling a rack against NVIDIA’s rack.

The rack is the product now

For two years, the AI infrastructure conversation has centered on GPU count and memory bandwidth per chip. Helios reframes the unit of competition. The path an AI workload takes through the rack matters more than any single component: work enters through the host layer, moves into the GPU domain, pulls model weights and active data from HBM4, and communicates across the rack through the scale-up fabric. AMD co-designed all of it as one platform rather than bolting together parts from separate roadmaps.

That is the genuinely new thing here. NVIDIA’s NVL72 established the rack-scale template, and AMD is now answering with a comparable form factor plus an open-standards pitch. Helios pairs the MI455X with EPYC Venice CPUs for host-side routing, scheduling and memory management, and Pensando networking for scale-out. The 43 TB/s of scale-out bandwidth is a deliberate answer to the multi-rack reality of production inference, where a single 72-GPU domain rarely holds an entire frontier model’s working set.

The economics matter as much as the raw numbers. Monthly token consumption has increased 158X in two years, according to Exponential View’s State of the AI Economy report cited by AMD. Training compute has grown 5X per year since 2020 per Epoch AI. Inference is now the largest AI workload, and AMD’s own projections put it at 60% of the workload mix in 2026, up from 40% in 2024. Agentic AI compounds the pressure: a single request can trigger multiple reasoning steps, sub-agents, retrieval operations and tool calls, each consuming tokens and fabric bandwidth.

The comparison game has limits

AMD’s claims against Vera Rubin NVL72 deserve scrutiny. On Kimi K2 Thinking with a 32K input and 8K output sequence, modeled Helios throughput per GPU is up to 15% higher at low interactivity, 12% higher at medium and 10% higher at high interactivity versus the NVIDIA rack. Those are AMD Performance Labs calculations from July 2026, based on published NVIDIA specifications and projected hourly cloud pricing. The figures are not measurements on shipping hardware, and the footnote language concedes that configurations and pricing may vary.

The 15% compute advantage is also a peak-theoretical comparison across different data types: AMD used FP4, FP8 and MXFP formats while NVIDIA’s NVL72 was evaluated on dense NVFP4 and FP8/FP6. Real workloads will land somewhere between the theoretical peaks and the modeled throughput. The honest read is that Helios is competitive with Vera Rubin on paper, with a meaningful edge in HBM4 capacity and scale-out bandwidth, and that the gap is narrow enough to force a real fight on price, software maturity and supply.

Software is the unspoken battleground

AMD positions ROCm as the connective tissue: native support for PyTorch, TensorFlow and JAX, plus optimized libraries and tools for deployment, observability and lifecycle management. The pitch is that developers keep familiar frameworks while the rack’s compute, memory and fabric capabilities stay accessible. That is the right message, but it is also the weakest link historically. ROCm has closed much of the gap with CUDA over the past three years, yet production AI stacks still carry NVIDIA-specific assumptions in kernels, serving runtimes and orchestration layers.

The DeepSeek-V4-Flash and Kimi K2 Thinking benchmarks are telling. AMD chose open-weight models for its headline comparisons, which is sensible given that open-weight inference is where AMD’s software story is strongest. The harder test is the closed-weight frontier: GPT-class models, Claude-class models and their serving stacks are tuned for NVIDIA hardware, and no open-standards pitch changes that overnight. AMD’s 34X token-throughput improvement over its own previous GPU generation is impressive, but the comparison that matters for enterprise buyers is against the NVIDIA stack they already run.

What this means for AI builders

For teams building inference infrastructure, Helios changes the procurement calculus. The 50% HBM capacity advantage matters for large-context workloads and long-running agents that need model weights plus state resident in memory. The 260 TB/s scale-up bandwidth matters for tensor-parallel training and for serving models that exceed a single GPU’s memory. The 43 TB/s scale-out bandwidth matters for the multi-rack deployments that production traffic actually requires.

The open-rack architecture is the quieter strategic play. AMD is betting that buyers want choice in networking, management and serviceability rather than a locked platform. That resonates with cloud operators and sovereign AI projects that have grown wary of single-vendor dependence. The adoption list AMD cites spans AI leaders, cloud partners and infrastructure partners, which suggests the supply chain is already lining up.

The honest caveat: Helios is a reference design with projected specifications. AMD says actual power consumption, thermal design and system configuration are available to qualified customers under NDA. The figures are subject to change. Builders should treat the 2.9 exaflops and 34X claims as directional, not contractual.

What is not directional is the strategic shift. AMD has committed to an annual execution cadence that connects successive Instinct GPU generations with progress in memory, interconnect and scale-up. The rack is the product, the GPU is the engine, and the software is the moat AMD is still digging. NVIDIA will not cede the rack-scale market without a fight, and the Vera Rubin line is already in the field. But for the first time since the AI infrastructure boom began, the highest-performing rack is not automatically an NVIDIA rack. That is a change every AI builder should price into their next capacity decision.