Moonshine Voice beats Whisper on latency with 245M parameters, not 1.5B

The open-source speech-to-text market has a new contender that is not just smaller but faster and more accurate than OpenAI’s Whisper Large v3 on live audio. Moonshine Voice, released by the research group Moonshine AI, ships a family of streaming speech models that run entirely on-device, require no API keys, and claim a 5x latency improvement over Whisper for real-time transcription.

The headline numbers are hard to ignore. Moonshine’s Medium Streaming model uses 245 million parameters and achieves a 6.65% word error rate on the OpenASR leaderboard. Whisper Large v3 uses 1.5 billion parameters and scores 7.44%. On a MacBook Pro, Moonshine Medium processes audio in 107 milliseconds. Whisper Large v3 takes 11,286 milliseconds. On a Raspberry Pi 5, Moonshine Tiny runs in 237 milliseconds. Whisper Tiny takes 5,863 milliseconds — and Whisper Large v3 does not run at all.

The gap is not incremental. It is structural.

What Moonshine fixed

OpenAI’s Whisper family, released in 2022, set the standard for open-source speech recognition. It was a breakthrough: a single model that handled 82 languages, offered multiple sizes, and beat most proprietary systems. But Whisper was designed for batch processing of pre-recorded audio, not for live streaming. Its fixed 30-second input window means every transcription call processes a full 30-second chunk, even when the user speaks for only three seconds. That zero-padding wastes compute and adds latency.

Moonshine removes the fixed window entirely. The model accepts any input length and spends compute only on the actual audio. For streaming applications, it caches the encoder state and part of the decoder state between calls, so repeated transcription of the same audio with new additions does not re-process the full input. The result is that Moonshine returns partial transcripts in under 200 milliseconds — the threshold for acceptable responsiveness in voice interfaces.

Moonshine AI also trained language-specific models rather than a single multilingual one. The group argues, in its Flavors of Moonshine paper, that a model focused on one language achieves higher accuracy per parameter than a generalist model. The supported languages for speech-to-text include English, Spanish, Mandarin, Japanese, Korean, Vietnamese, Ukrainian, and Arabic. For text-to-speech, the list extends to 16 languages including German, French, Hindi, Italian, Portuguese, Russian, and Turkish.

The edge deployment problem

Whisper’s ecosystem is mature but fragmented. FasterWhisper, Whisper.cpp, and other ports exist for different platforms, but each has its own interface, optimization level, and maintenance cadence. Building a voice application that runs on iOS, Android, Linux, Windows, and a Raspberry Pi with a single codebase is not straightforward.

Moonshine ships a portable C++ core library that uses OnnxRuntime for cross-platform inference. On top of that, it provides native bindings for Python, Swift, Java, and C++. The same API works on a MacBook, an iPhone, an Android phone, a Windows desktop, a Raspberry Pi, and even microcontrollers and DSPs. The project includes example apps for each platform as downloadable release archives.

This matters for voice agents. A conversational agent that needs to listen, transcribe, recognize intent, and respond must do all of that on-device to keep latency low and privacy intact. Sending audio to a cloud API adds network round-trips, introduces privacy concerns, and breaks when connectivity is poor. Moonshine’s on-device approach means the entire pipeline — speech-to-text, intent recognition, text-to-speech — runs locally.

What Moonshine does not solve

Moonshine is not a replacement for Whisper in every scenario. For batch transcription of long recordings where throughput matters more than latency, Whisper’s ability to process large chunks in parallel on a GPU remains an advantage. Moonshine is optimized for streaming, not for bulk. Nvidia’s Parakeet models also compete in the cloud batch space.

The language coverage is narrower. Whisper supports 82 languages. Moonshine supports eight for speech-to-text and 16 for text-to-speech. That is enough for many applications, but not for global deployment. The tradeoff is explicit: language-specific models are more accurate per parameter, but you need more models to cover the world.

Moonshine also depends on OnnxRuntime for cross-platform inference. OnnxRuntime is mature and well-supported, but it adds a dependency that some embedded deployments may want to avoid. The project does not yet offer a pure C or assembly-optimized backend for the most constrained devices, though the README mentions support for microcontrollers and DSPs.

What this means for AI builders

The voice agent market is heating up. Companies like ElevenLabs, Play.ht, and Hume AI are building proprietary voice models. OpenAI offers its own Whisper API with improved accuracy over the open model. But the open-source stack for on-device voice has been weak — Whisper is powerful but not designed for live interaction, and alternative projects like Silero VAD are limited to voice activity detection.

Moonshine fills that gap with a purpose-built streaming architecture. For a developer building a voice-controlled smart home hub, a real-time transcription app, or a conversational agent that runs on a phone or a Raspberry Pi, Moonshine offers a single library that handles the full pipeline with latency under 200 milliseconds and no cloud dependency.

The parameter efficiency is the most striking technical result. Moonshine Medium Streaming uses one-sixth the parameters of Whisper Large v3 and still beats it on accuracy. That is not just a win for edge deployment. It suggests that the architectural choices — flexible input windows, state caching, language-specific training — matter more than raw scale for the streaming use case. The open-source community now has a reference point for how to build speech models that are fast, small, and accurate for live audio.

The project is early. The documentation is thorough but the ecosystem is new. Moonshine AI has published benchmarks, a paper, and example apps, but the library has not yet seen the same level of community testing and optimization that Whisper has received over years. Developers should expect rough edges, especially on less common platforms.

Still, the numbers are real. A 245-million-parameter model that runs in 107 milliseconds on a laptop and beats a 1.5-billion-parameter model on accuracy is not a marginal improvement. It is a different design philosophy for speech recognition — one built for the latency and privacy constraints of live voice interfaces, not for the throughput demands of batch processing. That philosophy now has an open-source implementation that anyone can run.