Voxtral Mini 4B Realtime 2602: Low-Latency Open-Source Real-Time Speech Transcription Model

Voxtral Mini 4B Realtime 2602 is a multilingual real-time speech-to-text model that achieves an average word error rate (WER) of 8.72% on the FLEURS benchmark at 480ms latency across 13 languages, approaching the 5.90% WER of its offline counterpart. The 4B-parameter model uses a native streaming architecture with causal audio encoder and sliding window attention, supporting configurable delays from 240ms to 2.4s. It runs at over 12.5 tokens/second on a single GPU with ≥16GB VRAM, making it suitable for voice assistants, live subtitling, and on-device deployment under Apache 2.0 license.

Introduction to Voxtral Mini 4B Realtime 2602

Real-time speech transcription has become essential for voice assistants, live subtitling, and multilingual meetings. Voxtral Mini 4B Realtime 2602 stands out as an open-source solution that delivers accuracy comparable to offline systems while maintaining latency under 500ms. It supports 13 languages: Arabic, German, English, Spanish, French, Hindi, Italian, Dutch, Portuguese, Chinese, Japanese, Korean, and Russian.

The model totals approximately 4 billion parameters, consisting of a 3.4B language model and a 0.6B audio encoder. The audio encoder was trained from scratch with causal attention for streaming. Both components use sliding window attention, enabling effectively unlimited streaming without fixed context windows. At the recommended 480ms delay, its performance matches leading offline open-source transcription models while significantly outperforming existing open-source real-time baselines.

Released in BF16 precision under the Apache 2.0 license, the model supports both research and commercial use. Its efficient design allows real-time operation on edge devices, ensuring privacy by keeping sensitive audio processing local.

Key Features

What makes Voxtral Mini 4B Realtime 2602 different from chunk-based approaches? It uses a native streaming architecture that transcribes audio as it arrives, with fully configurable latency.

Key capabilities include:

• High-quality transcription with confidence scores
• Native support for 13 languages
• Real-time streaming ASR designed for live use cases
• Configurable transcription delays from 240ms to 2.4s

The architecture enables users to trade latency for accuracy depending on the application. For example, shorter delays suit interactive voice agents, while longer delays improve accuracy for subtitling.

Benchmark Results

Voxtral Mini 4B Realtime 2602 demonstrates competitive performance against both offline and real-time models. Here are the detailed results.

FLEURS Benchmark (13 languages)

At 480ms, the average WER is 8.72%, with English at 4.90% (vs. offline 3.32%) and Spanish at 3.31% (vs. offline 2.63%). Extending to 2.4s brings the average down to 6.73%.

Word error rate (lower is better) across languages in the FLEURS transcription benchmark.

Long-form English Benchmarks

Short-form English Benchmarks

The results show that at 480ms latency, the model maintains practical accuracy for real-world use, with gaps to offline performance typically under 1% in many English short-form tests.

Use Cases

Have you wondered where low-latency real-time transcription really shines? Here are practical applications:

• Private meeting transcriptions
• Live subtitle creation for broadcasts and events
• Real-time assistants that understand speech instantly
• Voice agents requiring natural conversation flow

The 480ms sweet spot makes interactions feel responsive, while the multilingual capability supports global teams and international content.

Its 4B parameter size and edge-friendly design make it particularly valuable for privacy-sensitive deployments where audio never leaves the device.

Recommended Settings and Best Practices

For optimal results, follow these precise recommendations:

• Set temperature to exactly 0.0 for deterministic output
• One text token corresponds to approximately 80ms of audio. For a 1-hour meeting, set --max-model-len >= 45000. The default 131072 tokens support roughly 3 hours
• Use WebSockets for audio streaming sessions
• Start with 480ms transcription delay in params.json, adjustable via "transcription_delay_ms": 480

These settings balance throughput, memory usage, and latency effectively.

Step-by-Step: How to Deploy and Use Voxtral Mini 4B Realtime 2602

Installation

1. Install the nightly vLLM package:

   uv pip install -U vllm \
       --torch-backend=auto \
       --extra-index-url https://wheels.vllm.ai/nightly
   

2. Verify mistral_common version:

   python -c "import mistral_common; print(mistral_common.__version__)"
   

3. Install audio processing dependencies:

   uv pip install soxr librosa soundfile
   

Docker images are also available for easier deployment.

Serving the Model

The model runs on a single GPU with at least 16GB VRAM. Launch command:

VLLM_DISABLE_COMPILE_CACHE=1 vllm serve mistralai/Voxtral-Mini-4B-Realtime-2602 --compilation_config '{"cudagraph_mode": "PIECEWISE"}'

Optional flags:

• Adjust --max-num-batched-tokens to trade throughput for latency
• Reduce --max-model-length if transcribing shorter sessions to save RoPE memory

After startup, the realtime endpoint appears at /v1/realtime.

Usage Examples

vLLM provides ready examples for:

• Streaming audio files
• Gradio-based live microphone transcription

Try the live demo: https://huggingface.co/spaces/mistralai/Voxtral-Mini-Realtime

WebSocket streaming ensures continuous low-latency audio processing.

License

Voxtral Mini 4B Realtime 2602 is released under the Apache 2.0 License. Usage must not infringe any third-party rights, including intellectual property.

Frequently Asked Questions (FAQ)

Which languages does Voxtral Mini 4B Realtime 2602 support?
It supports 13 languages: Arabic, German, English, Spanish, French, Hindi, Italian, Dutch, Portuguese, Chinese, Japanese, Korean, and Russian. Performance is strong in Romance languages, with Spanish achieving 3.31% WER at 480ms.

How do I balance latency and accuracy?
Use the recommended 480ms delay (average WER 8.72%). Shorter 240ms increases WER to 10.80%, while 2.4s reduces it to 6.73%. Modify "transcription_delay_ms" in params.json.

What hardware is required?
A single GPU with ≥16GB VRAM delivers real-time performance exceeding 12.5 tokens/second. Default max-model-len of 131072 supports approximately 3 hours of audio.

How should I handle long audio recordings?
Calculate tokens needed using 1 token ≈ 80ms. Set --max-model-len accordingly. The default handles ~3-hour sessions reliably.

Why set temperature to 0.0?
It ensures consistent, deterministic transcription results without randomness.

How does the realtime endpoint work?
vLLM’s Realtime API uses WebSockets to handle continuous audio streaming for production-grade sessions.

How does real-time performance compare to offline models?
At 480ms, the FLEURS average gap is 2.82% (8.72% vs 5.90%). In many English tests, the difference is under 1%, making it viable for interactive applications.

Does it support speaker diarization?
The current real-time version focuses on transcription. Batch models in the Voxtral family provide speaker labels and word-level timestamps.

What deployment best practices should I follow?
Always use WebSockets for streaming, install nightly vLLM for compatibility, and consider Docker for platform consistency.

This guide covers the essential details to get you started with Voxtral Mini 4B Realtime 2602 for production real-time transcription.