Qwenputer puts a working RISC-V CPU inside Qwen3.5 4B's weights

Divyansh Agrawal has published Qwenputer, a stock Qwen 3.5 4B with a functioning 32-bit RISC-V CPU compiled directly into its weights. The core runs inside the model’s forward passes, uses the model’s own Gated DeltaNet layers as RAM, and executes C and MicroPython without any external runtime, tool server, or sandbox. This is not a wrapper around a Python interpreter. The arithmetic happens in the weights.

The mechanism is the interesting part. Qwen 3.5 routes three of every four attention layers through Gated DeltaNet, a linear RNN whose heads each keep a fixed-size state matrix. Agrawal found that setting the forget gate to 1, the write strength beta to 1, and using one-hot key encoding turns each head into exact, addressable memory: one write overwrites exactly one row, one query reads it back. The delta rule as associative memory is not new; Agrawal cites Schlag, Irie, and Schmidhuber’s 2021 ICML paper “Linear Transformers Are Secretly Fast Weight Programmers.” The new claim is exactness inside a production model, reliable enough to host a CPU.

A CPU that costs 11 layers and no training

The build has no training step. A compiler sets every weight directly. Agrawal describes the CPU as a circuit, a long list of small steps like “read this register” or “jump if zero,” and the compiler turns each step into a slice of transformer layer: attention heads that look up, feed-forward units that do the arithmetic, DeltaNet heads that hold memory. The core is RV32IM plus the Zba extension and a small bignum unit, and it occupies 11 layers in front of Qwen’s 32. Qwen’s original weights stay untouched; the computer weights are added on top in extra dimensions and layers.

The validation claims are specific and worth stating plainly. The CPU passes the official riscv-tests, its programs match qemu, and it ran 18 million instructions free-running without a single error, meaning it kept going on its own outputs with no reference machine correcting it between steps. Agrawal replayed about 230,000 of its output positions against a reference machine and found zero mismatches. Base Qwen 3.5 4B does not keep exact values in GDN by default; this modification does.

The model uses the CPU without post-training. Agrawal piggybacks on the tool-call format Qwen already learned, so the model sends instructions and reads results in a familiar shape, but no external tool call fires. A small driver called RvChat loads firmware into the GDN state and keeps the trace tokens out of Qwen’s own layers. Each instruction becomes a few decoded tokens: one names the memory row (ADDR), the next carries the value (VAL), and a STEP token ends it. The generated tokens are the execution trace, so the CPU runs while Qwen decodes.

The numbers, and where they hold

On DeepMind’s arithmetic sets, Qwenputer scores 97.5% on div_big against 95% for Qwen 3.5 4B with a Python tool and 72.5% for base Qwen. It hits 97.5% on mul_big against 100% with Python and 55% for base. It reaches 100% on mul_div_multiple_longer against 97.5% with Python and 90% for base, and 97.5% on add_or_sub_big, matching the Python tool and beating base Qwen’s 87.5%. On NUPA’s long-integer arithmetic, Qwenputer goes from 48.6% to 100%, matching a Python tool. Agrawal also reports that adding the computer does not change how the model talks: normal text stays within noise of base Qwen at KL 6e-4 against a noise floor of 5e-4.

There is a filesystem too, because a CPU alone was not enough. The model exposes internal put, append, get, list, and delete operations. When Qwen saves a file, the compiled layers hash its name into one of 1,024 slots in a directory held in DeltaNet state, store the text in other heads at eight tokens per row, and chain rows together. Collisions check the next slot. It holds up to 512 files, about 53,000 tokens, and the state persists across conversations: save it at the end of a chat, load it into the next, and the files remain. On facts that changed between chats, Qwen gets them right 90% of the time against 30% when old chats are pasted back in, and it edges out Python-run memory tools at 83.3% versus 78.9%. That filesystem code lives on the rv-store branch and is still experimental.

The limits are real. The checkpoint is 18.8 GB, roughly double base Qwen, mostly zero padding. It needs RvChat to boot firmware, so it does not run with a plain generate() call yet. Every CPU instruction costs a few decoded tokens, so a big calculation takes seconds rather than milliseconds; the batched executor used for benchmarks runs about 3,200 CPU tokens a second on an L4. The whole thing fits on a single 24 GB GPU.

Why this matters beyond arithmetic

The arithmetic benchmark is the demo. The safety implication is the story. Exact control over RNN state turns Qwenputer into a testbed, and Agrawal has already written two follow-ups: one on how hybrid models like Qwen 3.5 can learn to hide reasoning in their RNN state and evade chain-of-thought monitoring, and one on compiled weights as monitoring tools the model cannot switch off, or answer keys for interpretability.

If a model can hold exact state in its RNN layers, the chain of thought is no longer the only place reasoning can live, and monitors that read only the visible tokens are reading a fraction of the machine.

That second point deserves attention from anyone building interpretability tooling. The same property that makes GDN usable as RAM, exactness, makes it a place where computation can happen off the visible trace. Agrawal frames compiled weights as a monitor the model cannot tamper with, and that framing cuts both ways: a compiled monitor is unremovable, but so is a compiled program whose internal state the model’s own output does not reveal.

For AI builders, the practical read is narrower. This is not a path to replacing Python sandboxes tomorrow, given the 18.8 GB checkpoint and the seconds-per-calculation cost. It is a demonstration that the boundary between “model” and “runtime” is a design choice, not a law. Qwen 3.5’s Gated DeltaNet layers were built for efficient long-context attention; Agrawal repurposed them as addressable memory with three parameter settings and a compiler.

Agrawal’s stated next steps are to let Qwen write Python and run it on the MicroPython inside its own weights through a python() tool, and to make the CPU faster. Both are engineering problems. The harder question, the one the follow-up papers circle, is what it means for monitoring when a model’s working memory is exact, persistent, and invisible in the token stream. Qwenputer ships on Hugging Face and ModelScope, with code on GitHub, and runs on one 24 GB card. The safety research it enables is the part to watch.