llama.cpp b10362: HIP's missing CUB exposes the cost of backend sprawl

The latest llama.cpp release, b10362, is a study in unglamorous maintenance. The headline change is a test skip: the new multi_output_sampling_chain test, which exercises top_k sampling across multiple outputs, is now disabled on HIP, AMD’s GPU compute platform. The reason is telling. The backend probs path for top_k requires CUB, NVIDIA’s parallel primitive library, which is unavailable on HIP. So sampled_probs comes back null and the test aborts. The fix is to add it to the existing HIP skip list, alongside the other TOP_K tests. The companion change skips multi_output_cpu for the same reason.

That is the entire functional delta. No new model support, no quantization breakthrough, no performance claim. Just a test that cannot run on a supported backend, and a CI logging fix for a self-hosted gpu-rocm runner that cannot upload logs to Azure blob because of egress firewalling. The commit is co-authored by Jim Wu, an engineer at Xilinx. Released August 11, 2026, it is the kind of patch that appears dozens of times a week in this repository.

But read it as a signal, and b10362 is more interesting than its diff suggests. This is the sound of a project whose ambition has outgrown its architecture. llama.cpp, the project started by Georgi Gerganov in March 2023, now ships binaries for macOS Apple Silicon (arm64), macOS Intel, iOS, Ubuntu on x64, arm64, and s390x, Windows on x64 and arm64, Android arm64, and openEuler. It targets CUDA 12, CUDA 13, ROCm 7.14, Vulkan, OpenVINO, SYCL on FP32 and FP16, OpenCL on Adreno, and KleidiAI on Apple Silicon. That list is from the release notes alone. Each of those is a separate backend with separate dependencies, separate quirks, and separate test coverage.

The HIP skip is not an anomaly. It is the rule. The release notes show the macOS Apple Silicon build with KleidiAI enabled is marked DISABLED, pointing to pull request #23780. The openEuler build is similarly DISABLED. A project that once ran primarily on CPU with a single CUDA path now maintains a matrix of backends that no small team can fully test. The skip list is the honest accounting of that reality.

Here is the take: local AI’s fragmentation problem is not a hardware problem. It is a software problem. The models run fine. The chips are fast enough. The bottleneck is the combinatorial explosion of backend configurations, each with its own dependency tree, its own driver requirements, and its own silent failure modes. CUB is a good example. It is a header-only library from NVIDIA, so it should be portable. But HIP does not ship it, and AMD’s own alternative, rocPRIM, does not provide a drop-in replacement for every top_k path. So a feature that works on CUDA silently degrades on ROCm, and the only way to know is to run the test and watch it abort.

The CI fix in the same release is a smaller version of the same story. The self-hosted gpu-rocm runner cannot upload logs to Azure blob because its egress is firewalled. A run’s logs were being wiped by the next run. The fix writes each run’s logs to $OUT/run-<run_id>-<attempt>/, so an Actions run URL maps to its logs. This is infrastructure duct tape. It is necessary, and it is exactly the kind of work that does not make release notes exciting.

What does this mean for AI builders? Three things.

First, if you are building on llama.cpp for production, treat backend support as a first-class risk. The project’s own test suite now formally excludes HIP from top_k sampling tests. If your workload depends on multi-output sampling with top_k on AMD hardware, you are running untested code. That does not mean it is broken. It means nobody has verified it, and the maintainers have decided that verifying it is not worth the effort. That is a maintenance decision, not a quality judgment, but it changes the risk profile for downstream users.

Second, the economics of local AI favor the dominant backend. CUDA gets the tests, the optimizations, and the new features first. HIP, Vulkan, and SYCL get the skip lists. The release notes list Windows x64 with CUDA 12 and CUDA 13 DLLs, plus a preview build for Windows arm64 with CUDA 13.4. NVIDIA gets three build targets. AMD’s ROCm gets one, shared between Ubuntu and Windows. The asymmetry is not a conspiracy. It is a reflection of where the contributors are, what hardware they own, and what their CI runners can actually exercise. But the effect is real: local AI on non-NVIDIA hardware is a second-class citizen, and the gap compounds over time.

Third, watch the DISABLED flags. The KleidiAI build for Apple Silicon is disabled, pending pull request #23780. KleidiAI is Arm’s library for AI kernel optimization, and its integration into llama.cpp has been a recurring saga. When a backend gets disabled, it usually means the maintainers found a regression they could not fix quickly. The fact that it is still disabled in b10362, months after it was introduced, suggests the integration is harder than expected. For developers on Apple Silicon, the standard arm64 build works fine. But the promise of extra KleidiAI performance remains unfulfilled.

The deeper pattern is that llama.cpp has become a victim of its own success. It is the default runtime for local inference, with 124,000 stars and 21,600 forks. That scale brings a flood of contributions, each adding a backend, a quantization format, or a hardware quirk. The maintainers cannot keep every path green. The skip list is their release valve. The CI log fix is their way of making the failures debuggable. Neither is glamorous. Both are essential.

For the AI industry, the lesson is about the cost of heterogeneity. The cloud AI economy runs on a small number of standardized stacks: NVIDIA GPUs, CUDA, and a handful of inference servers. Local AI is trying to run on everything, and everything is expensive to support. The b10362 release notes list 29 assets across 20-plus platform and backend combinations. Every one of those combinations is a promise that someone must keep.

The honest reading of b10362 is that the promise is being kept unevenly. The tests that matter for CUDA run. The tests that matter for HIP are skipped. The CI logs for ROCm are preserved in run-specific directories, because the alternative was losing them entirely. This is not a failure of the llama.cpp team. It is the natural state of a project that has become the universal adapter for local AI. Universal adapters are always held together with skip lists and log directories.

The question for builders is whether they want to depend on a universal adapter, or on a stack that is tested end to end. For a hobby project, llama.cpp is the right choice. For a product, the skip list is a map of where your support burden will land. The maintainers have told you exactly which paths they do not test. Read the release notes like a risk assessment, because that is what they are.

The outstanding question is whether the fragmentation will resolve or deepen. The trend is toward more backends, not fewer. The openEuler builds target Ascend 310p and 910b chips. The Windows arm64 preview targets CUDA on ARM. Each new target adds a skip list entry somewhere. b10362 is one data point in that trend, and the trend points toward more complexity, not less.

The concrete observation to close on: the release notes for b10362 carry 29 downloadable assets, and the only code change is a test skip. That ratio, 29 assets to one skipped test, is the real headline. The project is spending more effort shipping to every platform than it is spending on new capabilities. That is where local AI is in August 2026, and it is a phase that will not end quickly.