OpenAI's Jalapeño chip beats Blackwell on perf/W: what the benchmarks really show

OpenAI has been quietly building “Jalapeño,” an inference ASIC developed with Broadcom, and the first independent benchmarks are in. SemiAnalysis published the results on August 25, claiming the chip beats every Nvidia, AMD, and Google accelerator they have tested on token throughput per megawatt. The headline number: Jalapeño delivers higher output tokens per MW than Nvidia’s GB200 Blackwell, and it does so without multi-token prediction, speculative decoding, or prefill-decode disaggregation.

The chip went from initial team hiring to tape-out in about 16 months, with design work starting in mid-2024. That is an unusually fast ASIC development cycle, and SemiAnalysis credits AI-assisted chip design for part of the acceleration. The B0 stepping, currently in the fab at TSMC’s N3P node, delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized die at a 700W TDP. Rubin’s comparable die on the same node hits 17.5 PFLOPs but draws 900 to 1,150W. The efficiency gap is the story.

The perf/W framing is the real news

OpenAI designed for tokens per megawatt, not raw FLOPs. The reasoning is straightforward: the company is power-limited, not budget-limited or floor-space-limited. As Nvidia’s own Jensen Huang said at Computex 2026, “If you have 1 gigawatt of power, then throughput per watt is revenue.” Nvidia repeated the same point during the Vera Rubin talk at Hot Chips 2026: “The data center is power limited today.”

This framing matters because it changes what a chip win means. Jalapeño is not trying to beat Nvidia on peak compute. It is trying to convert electricity into tokens more efficiently, and on that metric it appears to succeed. SemiAnalysis reports that on Kimi K2.5, the model behind Cursor Composer 2.5, Jalapeño reaches nearly 700 tokens per second per user and more than 9x the next best chip at 100 tokens per second per user. On GPT-OSS, Jalapeño’s iso-interactivity throughput per MW is nearly double GB200’s highest throughput point and more than 50x GB200’s concurrency-1 point.

At concurrency 1 on DeepSeek R1, Jalapeño hits over 700 tokens per second per user. On Kimi K2.5 and GPT-OSS, the numbers reach approximately 1,400 tokens per second per user. SemiAnalysis verified these runs in person at OpenAI’s labs using their InferenceX suite, though they note that OpenAI provided all the numbers and they did not run the full benchmark suite themselves.

The Blackwell comparison is staged

Here is where the skepticism should sharpen. SemiAnalysis itself flags that comparing Jalapeño to Blackwell is “somewhat incomplete and unfair.” Blackwell uses HBM3E. Jalapeño uses HBM4, the same memory generation as Nvidia’s Vera Rubin, which is already shipping to customers. Jalapeño is still at engineering-sample stage. The fair comparison is against Rubin, and SemiAnalysis provides it: on output tokens per dollar, Vera Rubin and Jalapeño are head-to-head, producing almost the same number of output tokens per dollar. Jalapeño’s STP throughput per MW surpasses Vera Rubin’s MTP results that Nvidia and CoreWeave published in July.

The TCO parity carries a critical asterisk. Vera Rubin’s results use speculative decoding, which SemiAnalysis says leads to a 3 to 5x reduction in cost per token. Jalapeño’s results do not use speculative decoding. When OpenAI implements it, the TCO advantage could widen substantially. Part of the cost edge also comes from Broadcom’s lower margins versus Nvidia’s, though SemiAnalysis notes that Meta and Microsoft’s ASIC programs have struggled despite similar cost motivations, suggesting execution matters more than price.

What the benchmarks do not test

The models used in the benchmarks are not on the open frontier. SemiAnalysis points out that Nvidia and AMD have published results on larger, more recent models like DeepSeek V4 Pro and Kimi K3 using their AgentX suite. The larger and newer the model, the harder it is to bring up on a new chip. Jalapeño is running DeepSeek R1, Kimi K2.5, and GPT-OSS, all substantial models but not the newest frontier releases.

More importantly, all the results are on 8k1k workloads: 8,000-token context, single-turn. SemiAnalysis’s own AgentX suite, designed for long-context, multi-turn agentic workloads, has not been run on Jalapeño. The difference is not cosmetic. AgentX workloads stress routers, prefix cache mechanisms, cache management, and offload infrastructure, all of which behave differently under realistic production load. A chip that shines on short single-turn inference may struggle when a production agent workflow involves dozens of turns with growing context. OpenAI has not yet shown those numbers.

Architecture choices reveal strategy

Jalapeño is not the over-specialized chip that media coverage suggested. SemiAnalysis reports that OpenAI made a generalized inference chip, capable of running a range of models and workloads. As a joke, OpenAI demonstrated Doom running on the chip, ported with Codex prompts. The chip does not disaggregate prefill and decode across separate pools. Instead, draft models and main models share the same chips and fabric. This trades theoretical efficiency for operational flexibility: the ratio of input tokens to cache writes to cache reads to output tokens has shifted across the knowledge, reasoning, and agentic eras of models, and a homogeneous pool adapts better to that drift than fixed heterogeneous silicon.

The GSM8k evals on Jalapeño match Nvidia chips, per SemiAnalysis, which addresses the obvious question of whether the speed comes at a quality cost.

What this means for the AI compute market

The most significant implication is that Nvidia’s dominance in inference is no longer a given. OpenAI has demonstrated that a purpose-built inference chip, designed with tight hardware-software codesign, can beat the incumbent on the metric that matters most for power-constrained operators. The B0 stepping’s 25% perf-per-watt improvement over the A0 silicon, achieved only nine months after the A0 results were gathered, suggests the trajectory is still steep.

But the caveats matter. The benchmarks are vendor-provided, on easy workloads, against a chip that is one generation behind on memory. The real test is AgentX, and it has not been run. The real competitor is Rubin, not Blackwell, and Rubin is shipping today while Jalapeño is still in engineering samples. OpenAI’s own engineers ran the benchmarks in the lab with SemiAnalysis present, which is more verification than most chip launches receive, but it is not independent replication.

For AI builders, the practical takeaway is that the compute stack is becoming a competitive variable again. If Jalapeño’s efficiency holds up under agentic workloads and scales to production, the cost of serving tokens drops meaningfully, and that changes the economics of model deployment. OpenAI is not just a model vendor anymore. It is becoming a hardware vendor with a captive demand source, and that vertical integration is the thing Nvidia should be watching. Jalapeño’s perf/W numbers are impressive. The question is whether they survive contact with real agentic traffic.