Microsoft's VibeVoice: An open-source voice AI framework with a 7.5 Hz tokenizer and a 90-minute TTS limit

Microsoft open-sourced VibeVoice, a family of voice AI models that includes both text-to-speech and automatic speech recognition. The release is notable not for any single model but for the architecture that ties them together: a continuous speech tokenizer operating at an ultra-low frame rate of 7.5 Hz. That number — 7.5 tokens per second of audio — is the key engineering decision that makes everything else possible.

Most speech tokenizers run at 25 Hz or higher. VibeVoice’s 7.5 Hz frame rate means the model processes roughly one-third as many tokens per second of audio. For long-form tasks, that difference compounds dramatically. A 60-minute audio file at 25 Hz produces roughly 90,000 tokens. At 7.5 Hz, it produces roughly 27,000. That fits comfortably inside a 64K token context window, which is exactly what the VibeVoice-ASR model exploits.

The ASR model, VibeVoice-ASR-7B, is the most technically interesting component. It accepts up to 60 minutes of continuous audio input in a single pass. Conventional ASR systems slice audio into short chunks — typically 10 to 30 seconds — and lose global context between chunks. Speaker tracking, topic shifts, and long-range semantic coherence all degrade when the model cannot see the full conversation. VibeVoice-ASR sidesteps that by keeping the entire hour in context. It jointly performs ASR, speaker diarization, and timestamping, producing structured output that identifies who said what and when. The model also supports customized hotwords: users can provide domain-specific terms or names to guide recognition, which improves accuracy on technical or niche content.

The TTS side is equally ambitious. VibeVoice-TTS-1.5B synthesizes speech up to 90 minutes long with up to four distinct speakers in a single pass. That is an order of magnitude longer than most TTS systems. The model maintains speaker consistency across the entire generation, which is hard for any TTS system that resamples or segments long text. The paper was accepted as an Oral at ICLR 2026, a strong signal that the research community views the work as significant.

There is also VibeVoice-Realtime-0.5B, a lightweight streaming TTS model with roughly 300 milliseconds of first-audio latency. It supports streaming text input and generates speech up to about 10 minutes. At 0.5B parameters, it is deployment-friendly and runs on consumer hardware. The model includes experimental multilingual voices in nine languages — German, French, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish — plus 11 distinct English style voices.

The architecture behind all three models follows the same pattern: a large language model handles textual context and dialogue flow, while a diffusion head generates acoustic details. The LLM component is Qwen2.5 1.5B for the TTS models, and a larger 7B model for ASR. The continuous tokenizer at 7.5 Hz is the shared foundation.

Now the hard questions.

Microsoft removed the VibeVoice-TTS code from the repository in September 2025, shortly after its initial release. The repository notes: “After release, we discovered instances where the tool was used in ways inconsistent with the stated intent. Since responsible use of AI is one of Microsoft’s guiding principles, we have removed the VibeVoice-TTS code from this repository.” The ASR model and the real-time TTS model remain available. The TTS model weights are still on Hugging Face, but the inference code is gone. That is an unusual half-measure. If the concern is misuse — deepfakes, impersonation, disinformation — the weights are the dangerous part, not the code. Removing the code while leaving the weights accessible suggests either a legal or policy-driven decision that did not fully account for the practical reality of open-weight models.

The repository’s stated risks and limitations section is blunt. “Potential for Deepfakes and Disinformation: High-quality synthetic speech can be misused to create convincing fake audio content for impersonation, fraud, or spreading disinformation.” It explicitly recommends against using VibeVoice “in commercial or real-world applications without further testing and development.” The model is intended for research and development purposes only.

This creates an awkward tension. Microsoft is releasing frontier voice AI as open-source research, but the technology is clearly production-capable. A 90-minute multi-speaker TTS model with expressive speech and cross-lingual support is not a toy. A 60-minute ASR model with built-in diarization and hotword support is commercially useful today. The research-only framing feels like a liability shield rather than a genuine capability limitation.

A 90-minute multi-speaker TTS model with expressive speech and cross-lingual support is not a toy. A 60-minute ASR model with built-in diarization and hotword support is commercially useful today.

The competitive landscape makes the release more interesting. ElevenLabs, Play.ht, and Respeecher all offer high-quality voice synthesis as commercial products. OpenAI’s Whisper and Deepgram’s Nova dominate speech recognition. VibeVoice enters this space as an open-source alternative with capabilities that match or exceed some commercial offerings, particularly on long-form tasks. The 7B ASR model is large — too large for on-device deployment — but the 0.5B real-time TTS model is small enough to run on a laptop.

What Microsoft does next matters more than the release itself. The company has a pattern of open-sourcing research and then not investing in the product layer. The VibeVoice repository has not been updated since March 2026, when the ASR model was integrated into the Hugging Face Transformers library. That integration is useful — it means any Transformers user can load VibeVoice-ASR with a few lines of code — but it also signals that Microsoft views this as a research project, not a product.

The community will decide the framework’s trajectory. The ASR finetuning code is available. vLLM inference is supported. The continuous tokenizer at 7.5 Hz is a genuinely novel contribution that other researchers can build on. If someone finetunes VibeVoice-ASR on medical or legal or customer-service data, the model becomes immediately useful in production contexts. If someone builds a real-time pipeline around VibeVoice-Realtime, the latency and quality are good enough for voice assistants and interactive applications.

The 7.5 Hz tokenizer is the lasting contribution. It is not the headline — 60-minute ASR and 90-minute TTS are the headlines — but it is the architectural insight that enables both. Every other speech model that wants to handle long-form audio will have to reckon with that frame rate decision. Microsoft showed that you can drop the token rate by two-thirds without losing audio fidelity, as long as you pair the tokenizer with a diffusion head that can reconstruct the acoustic details. That is a research result worth building on.

For now, VibeVoice sits in an ambiguous position: too capable to be dismissed as pure research, too incomplete to be a product, too open to be controlled, too risky to ignore. That is the space where open-source frontier models live.