Cactus Compute's 14MB Needle 2 shrinks tool-calling AI to fit on a phone

Cactus Compute has released Needle 2, a 45M-parameter foundation model for tool calling and structured extraction that ships as a single 14MB binary and runs a full session in about 28MB of RAM. It is designed for phones, wearables, smart home devices, and robots. The model trades wins on benchmarks with FunctionGemma 270M, LFM2.5 230M, and Apple FM, at 5x to 70x smaller, and 2 bits against their f16.

The headline number is the size. A 14MB model that can parse a natural-language request, decide which of a large tool catalogue to invoke, fill arguments with values constrained by a compiled grammar, and return a confidence score is a different class of object from the 7B-parameter models that most on-device AI demos have been built around. It changes what hardware can host an agent, and it changes the economics of who gets to run one.

The architecture is the story behind the number. Needle 2 is built on the Simple Attention Network recipe, a dense small-model design that replaces the feed-forward network with a Hadamard MLP, uses grouped-query attention, engram key-value memory, and multi-lane hyper-connections. The details are in the paper on arXiv:2607.18363. The key move is that the Walsh-Hadamard transform is a fixed orthonormal matrix applied in n log n time with no weights to read. That removes a large chunk of the parameter budget that a conventional transformer would spend on linear projections.

The engineering choices around the inference path matter as much as the architecture. Weights are baked into the engine, so there is no separate model file to manage, and inference does no network calls. A byte-level grammar compiled from the user’s JSON schemas constrains every token the model emits. That means the model cannot produce malformed JSON, and it cannot emit an argument value outside the declared range. The grammar is not a post-hoc validator; it is part of the decoding process itself.

This is the most interesting design decision in the release. Most tool-calling systems generate free text and then parse it, or generate JSON and then validate it. Needle 2 instead makes schema conformance a property of the decoder. If you declare a Literal["heat", "cool", "auto"] argument, the model physically cannot emit anything else. If you declare a pattern on a string field, the grammar enforces it. The failure mode shifts from “the model produced a malformed call” to “the model produced a call that does not match the input evidence,” which is a much more tractable problem.

The confidence gating reinforces that shift. Every response carries a calibrated confidence score from a learned head, computed as the minimum of two signals: a post-hoc head that scores the full prompt plus the call, and the decoding probability of the call tokens. A call is accepted only when both agree. The documented contract is explicit: pick a threshold for your product, act at or above it, re-ask or route to a bigger model below it. Off-topic requests return the empty call [], which is the entire refusal mechanism. There is no free-text fallback.

The refusal behavior is worth dwelling on, because it is a philosophical position as much as a technical one. Needle solves every problem as a function call. Performing an action and extracting structured data are the same operation; the only difference is what you declare as the tool. Extraction is tool calling with one tool. That uniformity is elegant, but it also means the model is useless for open-ended conversation. You cannot ask it to summarize a paragraph or write a haiku. It will return [] and stop. That is a feature for the target use case, but it is a hard boundary that any builder needs to understand before adopting it.

The tool retrieval mechanism is another practical detail. Five or fewer declared tools render directly. Above that, a built-in contrastive head embeds every tool schema at init, embeds the query each turn, and renders only the top five tools into context, with the grammar rebuilt over just that subset. An unselected tool is unreachable, not merely unlikely. This is a real solution to the problem that large tool catalogues blow up the prompt and slow decoding. The embeddings persist to disk via tool_index_path, keyed by a fingerprint over the schemas and the model, so a changed schema re-embeds only what changed.

Memory behavior is bounded by design. A 256-token sliding window keeps the tools pinned as KV sinks, so total memory stays near 28MB no matter how long the conversation runs. That is a hard constraint that most transformer-based models cannot offer, because their KV caches grow with context length. For a wearable or a smart home hub with a few hundred megabytes of usable RAM, this is the difference between a model that can run and one that cannot.

Fine-tuning follows the same philosophy of smallness. LoRA on the frozen base, adapter merged at export, and the tuned model is still a single .cact file that runs on the same engine. The data format is a JSONL file where each example is a query, a tool set, and an answer. Off-topic examples have answers: []. The workflow is cheap enough that the playground UI has a “Finetune on these tools” button that runs the pipeline and hands back a downloadable model.

The honest caveats are in the benchmarks. The source says Needle 2 “trades wins” with FunctionGemma 270M, LFM2.5 230M, and Apple FM. That is not a claim of dominance; it is a claim of parity at a fraction of the size. The 2-bit quantization against their f16 is a meaningful compression ratio, but it also means the model is operating at the edge of what numerical precision allows. The confidence head is calibrated, but calibration on the training distribution does not guarantee calibration on your distribution. Any production deployment should test the thresholds against real traffic.

What this means for the AI industry is a shift in the default assumption about where agents live. The past two years of agent frameworks have been built around cloud-hosted models with large context windows and expensive inference. Needle 2 is an argument that a meaningful class of agentic workloads, the ones that call tools and extract structured data, can run entirely on a device with no network access and no cloud bill. The model fetches its engine once from Hugging Face and caches it; after that, inference is local.

The economics are worth stating plainly. A 14MB model running in 28MB of RAM costs nothing per inference. It has no API token pricing, no rate limits, no data egress. For a smart home company shipping a hub, or a wearable maker shipping a watch, that is not a marginal improvement; it changes the product calculus. The privacy story is also real: no user query ever leaves the device.

The open question is how far the approach scales. The Simple Attention Network recipe is designed for dense small models, and the paper documents ablations, but the 45M-parameter scale is a specific operating point. Whether the same techniques hold at 200M or 500M parameters, where the benchmark competitors like FunctionGemma and LFM2.5 live, is not established by this release. Cactus Compute has picked a niche and optimized for it aggressively.

The most useful way to read Needle 2 is as a proof that the small-model frontier is not just about parameter count. It is about the full stack: architecture, quantization, decoding constraints, and memory management. A 45M-parameter model with a grammar-constrained decoder and a confidence gate is a more reliable tool-calling system than a 7B model with none of those controls, for a fraction of the cost. The builders who treat the model as the only variable are missing the point. The model is the smallest part of the system, and that is the lesson worth taking from this release.