Kimi K3 in 8.24 GB of RAM: a 2.78T-parameter model on one CPU

A 2.78-trillion-parameter model running inference on a single CPU in 8.24 GB of RAM, with no GPU, no BLAS, and no framework. That is the claim on the tin at FareedKhan-dev/kimi-k3-in-c, and the repository has already pulled 10.8k stars and 1.7k forks. The number that matters is not the 2.78T. It is the 8.24 GB. That is less memory than a mid-range laptop ships with, and it is the figure that should reframe how the open-model crowd thinks about serving.

The mechanism is not magic, and the repo is honest about it in its topic tags: mixture-of-experts (moe), mxfp4 quantization, linear attention, AVX2 SIMD, and a from-scratch C99 implementation. Every one of those is a known lever. What is new is the combination and the packaging. A portable C99 engine with zero dependencies, Apache-2.0 licensed, 98 commits, a CMake build and a Makefile, and a third_party directory that is apparently small enough not to matter. The author, Fareed Khan, has a writeup on Medium titled “Building Kimi K3 in C to run a 2.8T model on consumer hardware,” and the repo’s own description points there.

Why 8.24 GB is the real headline

The AI industry has spent two years arguing about FLOPs. Training compute, inference throughput, tokens per second per dollar. The Kimi K3-in-C project argues about something less fashionable: bytes. A 2.78-trillion-parameter model at 16-bit precision would need roughly 5.5 TB just to hold the weights. At 4-bit, still about 1.4 TB. Neither fits on a consumer machine, which is why the standard answer has been “you need a data center” or “you need a quantized 7B.”

Mixture-of-experts breaks that arithmetic. A model with 2.78T total parameters but a small active-parameter count per token only needs to load the experts a given token routes to, not the whole model. Combined with MXFP4 (a 4-bit floating-point format with a shared exponent per block), the weight footprint drops hard. The 8.24 GB figure is almost certainly the resident set for the active path plus whatever expert cache the engine keeps warm, not the full checkpoint. That distinction matters, and the repo does not spell it out in the README excerpt we can see. {/* TODO: verify whether 8.24 GB is peak RSS, active-path footprint, or on-disk checkpoint size, and clarify expert offload/caching behavior from the Medium writeup or source. */}

Still, the implication is sharp. If a competent C programmer with a laptop can stand up a working inference path for a frontier-scale MoE in a few thousand lines, then the moat around “you need our GPU cluster” is thinner than the capex numbers suggest. Not gone. Thin.

The systems-programming revival

The most interesting thing about this repo is not the model. It is the genre. A decade of ML tooling pushed everyone toward Python, PyTorch, and a CUDA dependency. That stack is wonderful for research and terrible for portability. Kimi K3-in-C is part of a small but growing countercurrent: people reimplementing inference in C, C++, Rust, and Zig to escape the framework tax. llama.cpp is the famous example. This is the same instinct applied to a model two orders of magnitude larger.

The topic list reads like a syllabus for that countercurrent: avx2, simd, c99, zero-dependencies, memory-efficient, linear-attention, systems-programming. Linear attention in particular is worth flagging. If Kimi K3 uses a linear-attention variant rather than full quadratic attention, the KV cache stops being the dominant memory cost at long context, which is exactly the constraint that has kept long-context models on expensive hardware. Pair that with MoE routing and MXFP4, and the 8.24 GB number becomes plausible rather than promotional.

A 2.78-trillion-parameter model in 8.24 GB is not a stunt. It is a proof that the memory wall is a software problem as much as a silicon one.

What this means for the AI economy

The bear case for this repo is easy to write. A from-scratch C engine is not a production serving stack. It almost certainly lacks the batching, scheduling, and kernel coverage of vLLM or TensorRT-LLM. Throughput on a single CPU will be measured in tokens per minute, not tokens per second, and for most interactive applications that is unusable. The 10.8k stars reflect developer curiosity, not deployment.

The bull case is harder to dismiss. Every capability that moves from “data center only” to “laptop” has eventually reshaped the market around it. Quantization did that for 7B models. llama.cpp did it for local chat. If MoE plus aggressive low-bit formats can put a 2.78T model on a CPU, then the same techniques put a 30B or 70B model on a phone, a router, or an air-gapped workstation. That is a different product surface: privacy-preserving inference, offline agents, embedded copilots. The companies selling GPU hours have a real interest in this staying hard.

There is also a policy angle the repo does not touch. Export controls on advanced accelerators assume that frontier capability requires frontier hardware. A portable C99 engine that runs a 2.78T model on commodity silicon is a data point against that assumption, at least at inference time. Training is still a different story, and nothing here changes that. But the line between “frontier model” and “consumer hardware” is being redrawn by whoever writes the tightest C.

What to watch

The next signal is not the star count. It is whether anyone reproduces the 8.24 GB figure independently, and whether the engine handles long context without the KV cache blowing past that budget. The repo has a benchmarks directory and a tests folder; the community will run them. Watch the issue tracker for memory measurements on different CPUs, and watch whether the MXFP4 path holds up against a reference implementation. If the numbers survive scrutiny, this stops being a curiosity and becomes a template. If they do not, it is a very well-marketed weekend project. Either way, the argument it makes, that memory layout and quantization choices matter as much as raw compute, is one the AI software world should take seriously.