vLLM's README now lists 200+ model architectures and 2,000 contributors. That is the real AI infrastructure story.

The most consequential document in open-source AI inference is a README. The vllm-project/vllm repository now describes a library with “over 2000 contributors” across “many dozens of academic institutions and companies,” support for “200+ model architectures on Hugging Face,” and hardware plugins spanning NVIDIA, AMD, Intel, Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon, and MetaX GPUs. None of that is a product launch. It is a status report from a project that has quietly become the load-bearing wall under a big share of the industry’s inference.

The origin story still matters. vLLM was “originally developed in the Sky Computing Lab at UC Berkeley,” and its founding paper, “Efficient Memory Management for Large Language Model Serving with PagedAttention,” by Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica, appeared at ACM SIGOPS SOSP in 2023. Three years later the citation block is still the same. The project around it is not.

The hardware list is the argument

Most people read vLLM as a speed story. PagedAttention manages attention key and value memory the way an operating system manages virtual memory pages, and the README credits it for “efficient management of attention key and value memory.” Continuous batching, chunked prefill, and prefix caching follow. Those are real, and they are why the library earned its reputation.

The hardware plugin list is the more interesting claim. A serving engine that runs on NVIDIA GPUs and Huawei Ascend chips is not just a performance project. It is a portability layer. The README names Google TPUs, Intel Gaudi, IBM Spyre, Rebellions NPU, Apple Silicon, and MetaX GPU as supported through plugins, alongside x86, ARM, and PowerPC CPUs. Every one of those is a bet that model serving should not be locked to a single vendor’s stack.

That bet has commercial gravity. Export controls have made NVIDIA’s top accelerators scarce and expensive in parts of the world, and Chinese and Korean chipmakers have spent years trying to convince buyers that alternatives are viable. The hardest part of that pitch is never the silicon. It is the software gap: the kernels, the attention implementations, the batching logic that makes a chip usable for transformer inference. vLLM’s plugin architecture narrows that gap by giving each vendor a single integration point instead of a bespoke stack.

What is actually inside

The technical surface is broader than the tagline suggests. The README lists quantization support for FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ/AWQ, GGUF, compressed-tensors, ModelOpt, and TorchAO. Attention kernels include FlashAttention, FlashInfer, TRTLLM-GEN, FlashMLA, and Triton. GEMM and MoE kernels use CUTLASS, TRTLLM-GEN, and CuTeDSL. Speculative decoding covers n-gram, suffix, EAGLE, and DFlash. There is disaggregated prefill, decode, and encode, plus tensor, pipeline, data, expert, and context parallelism for distributed inference.

On the interface side, vLLM ships an OpenAI-compatible API server, “plus Anthropic Messages API and gRPC support.” It handles structured outputs through xgrammar or guidance, tool calling, and reasoning parsers, and it supports multi-LoRA for both dense and MoE layers. The model list runs from decoder-only LLMs (Llama, Qwen, Gemma) through mixture-of-experts architectures (Mixtral, DeepSeek-V3, Qwen-MoE, GPT-OSS) to hybrid attention and state-space models (Mamba, Qwen3.5), multimodal models (LLaVA, Qwen-VL, Pixtral), embedding and retrieval models (E5-Mistral, GTE, ColBERT), and reward and classification models (Qwen-Math).

Read that list as a map of where the field is moving. State-space and hybrid attention models get first-class support. MoE is treated as a default, not an exotic case. Reasoning parsers are a built-in feature. Two years ago none of those were standard.

The neutral-layer problem

Here is the uncomfortable part. A project this central is now infrastructure in the policy sense, not just the engineering sense. When a serving engine supports both NVIDIA and Huawei Ascend, it sits at the intersection of US export controls and Chinese compute strategy. When it supports both the OpenAI API format and the Anthropic Messages API, it becomes the place where competing interface conventions get reconciled, often before the vendors themselves agree.

The neutral serving layer wins by default, and defaults are hard to govern.

That creates a governance question the project has not really answered in public. The README points security disclosures to GitHub’s Security Advisories feature and partnerships to [email protected]. There is no foundation, no stated governance model, no published roadmap for how hardware vendors get their plugins merged or how conflicts between them get resolved. For a library with 2,000 contributors and this much vendor surface, that is a gap. The Linux kernel has a maintainer hierarchy and a foundation. PyTorch has a governance document. vLLM, as described on its own front page, has a Slack.

What it means for builders

For anyone serving models, the practical takeaway is that the abstraction has stabilized. You write against an OpenAI-compatible endpoint, point it at vLLM, and swap the hardware underneath. That is a genuine reduction in switching cost, and it is the reason the project keeps absorbing contributors from chip vendors who want their silicon to be a first-class target.

The second takeaway is less comfortable. If vLLM is the default serving path, then its quantization choices, its kernel priorities, and its model-support order shape what the rest of the industry can deploy cheaply. A new architecture that lands in vLLM on day one gets used. One that waits three months does not. That is real power, exercised by a project with no formal charter and a README that still opens with “Easy, fast, and cheap LLM serving for everyone.”

The next thing to watch is whether the project publishes a governance model before a vendor dispute forces one. The README lists IBM Spyre and Huawei Ascend in the same sentence. That sentence will eventually need a referee.