NVIDIA Releases Nemotron 3.5 Transcribe for Promptable Multilingual

NVIDIA Nemotron 3.5 Transcribe release

NVIDIA just dropped Nemotron 3.5 Transcribe, an open-weight model that crams transcription, speaker attribution, translation, and audio QA into a single checkpoint. It landed on Hugging Face and ModelScope back on October 28, 2026, right alongside Granary v2-a multilingual data pipeline and fine-tuning toolkit built on NeMo Speech. The announcement comes from a solid crew of NVIDIA researchers: Maryam Motamedi, Piotr Żelasko, Kunal Dhawan, Nikolay Karpov, Nune Tadevosyan, Meline Mkrtchyan, Nithin Rao Koluguri, and Francesco.

This thing handles 40 languages for transcription, translates 31 of them to or from English, and picks out speakers in 13. It chews through recordings up to an hour long and can tag up to eight speakers. Instead of wrestling with fixed APIs, you steer it with natural language prompts-so the same model spits out a speaker-labeled transcript, an English translation, or an answer to something like "what decisions were made in the meeting?" Word-level timestamps are in the mix too. The model card is upfront about the limits: audio QA was trained only in English, and language support shifts depending on the task.

Nemotron 3.5 Transcribe benchmark results

On benchmarks, NVIDIA says it tops the Hugging Face Open ASR Leaderboard for open-weight English models with a 5.33% average WER. Multilingual track? 4.62% mean WER across seven languages. Long-form audio sits at 10.98%-both the lowest among evaluated open weights in their comparison. Artificial Analysis independently has it tied for first at 2.8% AA-WER, hitting that mark with roughly 4 billion active parameters against a rival's 24 billion. On the held-out AA-AgentTalk voice-agent subset (half the index), it scores 1.8% WER versus 2.3% for the next best. The architecture pairs a speech encoder with a hybrid Mamba-Transformer MoE language model, so only about 4 billion of the 30 billion total parameters fire per token, planting it firmly on the Pareto frontier for accuracy versus active params.

Speaker attribution looks just as strong. On David AI's DAI-ASR-I18N diarization track across 21 languages, it posts the lowest DER of the 14 systems tested at 18.26%, edging out NVIDIA's own dedicated Nemotron 3 Diarization model at 18.38%. VoiceArena's English full-session benchmark across 14 systems gives it the lowest cpWER at 32.4%, with deletions at 8.32% of reference words compared to a median near 31%. It also grabs first on Ocular AI's Converse-STT eval among open models and shows up on Josh Talks' Voice of India leaderboard.

Sortformer speaker diarization architecture

The Sortformer speaker cues ride alongside acoustic features straight into the language model, letting Transcribe tag each turn during generation instead of needing a separate diarization pass. Quantized versions are on the roadmap for local deployment on RTX Spark, DGX Spark, and Jetson AGX Thor hardware, with Blackwell data-center GPUs ready for scaled workloads.

Granary v2 pushes that prompt-level control into training. Its multilingual dataset and reusable curation pipeline-both on Hugging Face and ModelScope-let developers define the written form they need and adapt the model to the specific speech their apps encounter. In a medical case study, fine-tuning slashed WER on held-out speakers by 45.8% relative to the best prompt-only result. The model card packs recommended presets, a full prompt guide, and a task-by-language matrix. Standard disclaimer applies: spot-check names, speaker changes, and answers against the recording.

Agora classroom integration demo

Agora-whose real-time voice and video platform powers live convos in apps-has already demoed a classroom integration. They use word-level timestamps to align a transcript with a recorded lecture replay so students can see exactly what was said and when. Their blog walks through the implementation.

Frequently asked questions

What languages does Nemotron 3.5 Transcribe support for transcription, translation, and speaker attribution?

Nemotron 3.5 Transcribe transcribes 40 languages, translates 31 of them to or from English, and performs speaker attribution in 13 languages. Language support varies by task, and audio QA was trained only in English according to the model card.

How does Nemotron 3.5 Transcribe perform on speech recognition benchmarks?

It leads the Hugging Face Open ASR Leaderboard for open-weight English models at 5.33% average WER, scores 4.62% mean WER across seven languages multilingually, and achieves 10.98% on long-form audio-the lowest among evaluated open weights. Artificial Analysis reports 2.8% AA-WER tied for first with only ~4 billion active parameters.

What architecture does Nemotron 3.5 Transcribe use and how many parameters are active per token?

The model pairs a speech encoder with a hybrid Mamba-Transformer MoE language model totaling 30 billion parameters, but only about 4 billion activate per token, placing it on the Pareto frontier for accuracy versus active parameter count.

How does Nemotron 3.5 Transcribe handle speaker attribution without a separate diarization pass?

Sortformer speaker cues are fed alongside acoustic features directly into the language model, allowing Transcribe to tag each speaker turn during generation instead of requiring a separate diarization step.

What is Granary v2 and how does it improve Nemotron 3.5 Transcribe for specific domains?

Granary v2 is a multilingual dataset and reusable curation pipeline that bakes prompt-level control into training. In a medical case study, fine-tuning with Granary v2 reduced WER on held-out speakers by 45.8% relative to the best prompt-only result.

Source: Hugging Face