Nari Labs Open-Sources Low-Latency Qwen3-TTS Implementation

Nari Labs has released an open-source implementation of the Qwen3-TTS 1.7B model designed to eliminate the lag in real-time voice AI. The system targets extreme low latency to enable seamless, human-like conversations without the awkward pauses typical of current open-source text-to-speech engines.

Running on a single NVIDIA H100 SXM, the implementation achieves a p95 time-to-first-audio (TTFA) of 34 ms at a load of 10 requests per second (RPS). According to Nari Labs, the system maintains a p95 TTFA of sub-50 ms up to 10 RPS and keeps latency below 100 ms even when pushed to 20 RPS. The project is now available to the public via GitHub.

The Production Latency Gap

Time-to-first-audio is the primary metric for determining whether a voice AI feels responsive or robotic. While the Qwen3-TTS model provides a strong foundation for open-source speech generation, the infrastructure used to serve these models often creates a bottleneck.

Existing serving engines, such as vLLM-Omni and SGLang-Omni, have struggled to balance low latency with stable playback in production environments. Nari Labs noted via Hacker News that these open-source implementations are often too slow for professional use and can suffer from playback issues when developers attempt to force lower latency.

Implications for Voice AI

By reducing TTFA to sub-50 ms, Nari Labs is bringing open-source TTS closer to the responsiveness levels required for fluid, bidirectional voice interaction. This development reduces the industry's reliance on proprietary, closed-source voice APIs that have historically dominated the low-latency market due to their optimized infrastructure.

For developers, this means the ability to deploy high-quality, responsive voice agents on their own hardware without sacrificing the user experience. It shifts the frontier of what is possible for self-hosted AI, moving from simple asynchronous responses to true real-time engagement.

What to Watch

As the implementation is now open-source, the next phase will likely involve community stress-testing across different hardware configurations beyond the H100 SXM. Developers will be looking to see if these latency gains hold steady across various languages and longer text inputs. While the core benchmarks are confirmed, the broader impact on production-scale voice agents remains to be seen as more teams integrate the engine into their stacks.