What Is Nemotron 3.5 Transcribe? Specs & Benchmarks

4.3B Active Parameters and 40-Language Diarization: A First Look at Nemotron 3.5 Transcribe

Nemotron 3.5 Transcribe is an open-weight speech-augmented language model developed by NVIDIA that converts recorded audio into verbatim transcripts, speaker labels, translations, and direct analytical answers. Released on October 28, 2026, the model pairs a specialized speech encoder with a 34-billion-parameter mixture-of-experts language backbone that activates roughly 4.3 billion parameters per token. It processes recorded audio files up to one hour in length across 40 languages under the OpenMDW-1.1 license.

Key Takeaways

• Sparsely Activated Backbone: Nemotron 3.5 Transcribe packages a 34B total parameter model (30B MoE base plus audio frontends) that evaluates only 4.3B parameters per forward step, keeping single-GPU inference viable.
• Consolidated Speech Stack: A single model checkpoint simultaneously executes transcription, multi-speaker diarization, translation, entity prompting, and post-recording audio question answering.
• Top Open-Weight Accuracy: The model posted a 5.33% average Word Error Rate (WER) on Hugging Face's English Open ASR Leaderboard and reached a 2.8% score on the Artificial Analysis AA-WER index.
• Speaker-Attributed Transcription: Using an integrated diarization module, the model tracks up to eight distinct speakers across 13 languages, leading VoiceArena's overlapping speech benchmark.
• Batch-Only Scope: The checkpoint requires complete audio files up to 60 minutes long and does not support low-latency streaming for live voice agents.
• Domain Adaptation Pipeline: NVIDIA paired the weights with the 463,000-hour Granary v2 speech dataset and NeMo Speech recipes that demonstrated a 45.8% error reduction in clinical dictation tests.

What Is Nemotron 3.5 Transcribe?

Modern speech pipelines often force engineering teams to glue together disparate microservices: an acoustic model for transcription, an external diarization network for speaker segmentation, a translation model for multilingual routing, and a separate large language model for summarization. Each handoff between components introduces latency, serialization overhead, and cascaded transcription errors.

Nemotron 3.5 Transcribe belongs to an emerging category called a Speech-Augmented Language Model (SALM). Instead of treating automatic speech recognition as a rigid phonetic classification task, the model processes audio tokens directly inside a generative language model decoder. This design allows developers to control transcription output using natural language instructions. You can instruct the model to produce strict verbatim transcripts with conversational disfluencies, output polished business prose, inject domain-specific medical acronyms, or output translations directly into English.

The model is officially released and publicly available. NVIDIA published the model weights on Hugging Face and ModelScope under the OpenMDW-1.1 license, enabling commercial deployment in self-hosted enterprise environments.

Source: @ModelScope2022

Nemotron 3.5 Transcribe at a Glance

The following specifications reflect the official release materials and technical documentation published by NVIDIA:

Specification	Value	Notes
Developer	NVIDIA	Published October 28, 2026
Model Type	Speech-Augmented Language Model (SALM)	Multimodal mixture-of-experts architecture
Total Parameters	34 Billion	Includes Transformer speech encoder
Active Parameters	~4.3 Billion	Per-token forward pass efficiency
Context Window	65,536 Tokens	Accommodates long-form text and prompt history
Maximum Audio Length	60 Minutes (1 Hour)	Non-streaming, offline batch processing
Transcription Languages	40 Languages	Full ASR coverage
Translation Coverage	31 Languages	Bidirectional translation with English
Diarization Capacity	Up to 8 Speakers	Supported across 13 primary languages
License	OpenMDW-1.1	Permissive commercial and research use
Primary Checkpoints	BF16, FP8, NVFP4, GGUF	Quantized variants target distinct hardware tiers
Training Dataset	Granary v2	463K+ hours across 48 languages

How Nemotron 3.5 Transcribe Works

Nemotron 3.5 Transcribe departs from standard encoder-decoder speech models by integrating specialized audio encoders directly into a state-space hybrid language model.

[Raw Audio (<=60 min)] ──> [Transformer Speech Encoder] ──┐
                                                          ├─> [Parallel Expert Encoder]
[Speaker Activity]     ──> [Frozen Diarization Model]   ──┘           │
                                                                       ▼
[Instruction Prompt]   ──> [Hybrid Mamba-2–Transformer MoE Decoder (34B / 4.3B active)]
                                       │
                                       ▼
                       [Generated Text + CTC Word Timestamps]

1. Hybrid Mamba-Transformer MoE Backbone

The text generation engine derives from the Nemotron 3.5 Lightning 30B-A3B architecture. It relies on a hybrid design that interleaves Mamba-2 state-space layers with standard self-attention Transformer layers, backed by a sparse Mixture-of-Experts routing mechanism. Mamba-2 layers process long temporal sequences with linear computational complexity, which prevents memory blowups when ingesting up to 60 minutes of audio representations. By activating only two experts per token alongside shared expert routing, the 34B parameter network limits per-token active compute to approximately 4.3B parameters.

2. Parallel Expert Encoder

In traditional speech systems, speaker diarization runs as an isolated clustering step before or after transcription. Nemotron 3.5 Transcribe implements a Parallel Expert Encoder. The architecture pairs a standard Transformer acoustic encoder with a frozen speaker identification network derived from Nemotron 3 Diarization (utilizing Sortformer speaker cues). Acoustic embeddings and speaker activity vectors are merged before projection into the language model decoder. This design allows the decoder to emit speaker tags sequentially alongside spoken text without requiring custom multi-head decoding architectures.

3. Native Multi-Token Prediction (MTP) and Speculative Decoding

Autoregressive decoding across hour-long transcripts creates significant serving bottlenecks. To accelerate throughput, NVIDIA trained the model with joint Multi-Token Prediction (MTP). The primary network contains auxiliary prediction heads that emit multiple future tokens simultaneously. During inference in engines like vLLM, these native MTP heads act as a self-contained draft model for speculative decoding. For higher inference concurrency, developers can optionally attach an external DFlash2 draft model.

4. CTC Adapter for Word-Level Timestamps

Because causal language models generate text autoregressively without intrinsic alignment to audio frame rates, getting accurate word-level timestamps is notoriously difficult in decoder-only systems. Nemotron 3.5 Transcribe solves this by training a lightweight Connectionist Temporal Classification (CTC) adapter. After the text decoder finishes generating the transcript, the CTC adapter aligns the generated token sequence against the acoustic encoder's frame representations. This step extracts precise word-level timestamps without degrading the generative quality of the transcript.

5. Contextual Entity Prompting

A core problem in automatic speech recognition is the misinterpretation of rare tokens, such as proprietary software packages, enterprise names, or pharmaceutical drugs. The model was trained using supervision-balanced instruction tuning that supports dynamic entity prompting. By supplying an expected vocabulary list within the system prompt, the model cross-references acoustic hints with the provided entity list. In benchmarks on ContextASR-Bench, supplying context reduced domain-specific entity transcription errors six-fold.

What You Can Do With Nemotron 3.5 Transcribe

Nemotron 3.5 Transcribe processes multimodal inputs via standard instruction formats. Developers supply audio alongside text directives specifying the task.

Multi-Speaker Meeting Transcripts

The model tracks up to eight conversational participants simultaneously, maintaining speaker continuity across fast-paced dialogue. During evaluation on overlapping speech scenes from historical films and corporate meetings, the model preserved sentence boundaries across interrupting speakers:

Prompt: "Transcribe this recording and label each speaker."
Output:
Speaker 1: The deployment completed ahead of schedule.
Speaker 2: Did we verify the latency on the Blackwell cluster?
Speaker 1: Yes, round-trip time dropped below our target threshold.

Contextual Domain Adaptation

When transcribing specialized conversations, engineers can guide model behavior using prompt-level constraints rather than immediate fine-tuning. This pattern proves effective in enterprise settings:

Prompt: "Transcribe this technical review. Possible terms: Kubernetes, Istio, etcd. Use these terms only when supported by the audio. Formatting: written form with proper capitalization."

Cross-Lingual Translation

Instead of routing an ASR transcript into a secondary machine translation model, Nemotron 3.5 Transcribe translates audio directly into English from 31 source languages:

Prompt: "Translate this Japanese audio into English text. Output transcript only."

Post-Call Question Answering and PII Redaction

Because the backbone is a full language model, the system can parse spoken content and extract insights directly from raw audio. In legal or medical recording workflows, users can instruct the model to sanitize sensitive data:

Prompt: "Return a full transcript of this patient intake recording with all personally identifiable information (PII) redacted using bracketed placeholders."

For teams designing broader automated voice pipelines, these analytical outputs can be fed directly into downstream voice synthesis engines like ElevenLabs V3 to build complete asynchronous voice-to-voice workflows.

How Nemotron 3.5 Transcribe Compares

Independent evaluations confirm that Nemotron 3.5 Transcribe sets a new performance benchmark for open-weight speech recognition.

Source: @AGTPinsights

Feature / Metric	Nemotron 3.5 Transcribe	Voxtral Small	Whisper Large V3	Nemotron 3.5 ASR Streaming 0.6B
Developer	NVIDIA	Mistral AI	OpenAI	NVIDIA
Architecture	Hybrid MoE (34B / 4.3B active)	Dense Decoder (24B)	Encoder-Decoder (1.5B)	FastConformer RNNT (0.6B)
Operating Mode	Offline batch (<=60 min)	Offline batch	Offline batch	Streaming (80–1120ms chunks)
AA-WER Index	2.8%	2.8%	~4.5%	Not evaluated on AA-WER
English Open ASR WER	5.33%	~6.1%	~6.5%	~8.2%
Active Params per Step	~4.3B	24B	1.5B	0.6B
Native Diarization	Yes (up to 8 speakers)	No	No	No
License	OpenMDW-1.1	Apache 2.0	Apache 2.0	OpenMDW-1.1

On independent benchmarks run by Artificial Analysis, Nemotron 3.5 Transcribe recorded a 2.8% score on the AA-WER index, tying the 24-billion-parameter Voxtral Small while evaluating roughly one-sixth the active parameters. On the held-out AA-AgentTalk voice dataset, Nemotron scored 1.8% WER compared to 2.3% for the next best open competitor.

The model showed an even wider performance margin in multi-speaker attribution. On David AI's DAI-ASR-I18N benchmark—which tests 147 hours of unscripted two-speaker dialogue across 21 languages—Nemotron 3.5 Transcribe placed first overall in speaker diarization accuracy, surpassing both open and proprietary baselines. In VoiceArena's 14-system comparison of speech recognition during rapid interruptions, the model missed only 8.32% of spoken words compared to a 31% median failure rate across competing speech engines.

Engineers comparing architectural paradigms between sparse state-space models and massive dense reasoning systems can reference our earlier look at how modern checkpoint efficiency scales in frontier models.

Availability: How to Access Nemotron 3.5 Transcribe

Nemotron 3.5 Transcribe is distributed across several open repositories:

1. Model Weights: Reference FP16 and BF16 checkpoints are available in the Hugging Face Repository, alongside mirrors on ModelScope.
2. Quantized Checkpoints: NVIDIA hosts FP8, NVFP4, and GGUF quantization formats. The NVFP4 build is optimized specifically for Blackwell server architectures.
3. Interactive Demo: A public testing environment is accessible via NVIDIA's Hugging Face Space.
4. Code and Tooling: Official fine-tuning tools and serving recipes are integrated into the NeMo Speech framework and NeMo-Speech.cpp. Detailed implementation mechanics appear in the NVIDIA Research Technical Report.

Hardware Requirements

Despite its 34-billion total parameter count, the sparse activation profile makes local inference straightforward on workstation and datacenter GPUs:

• NVIDIA H100 80GB: Runs the reference BF16 or FP8 checkpoint comfortably within single-device memory.
• DGX Spark / RTX Spark: Edge workstation setups can host the quantized GGUF or FP8 builds.
• RTX PRO 6000 Blackwell: Running the NVFP4 quantized checkpoint alongside the DFlash2 speculative decoding engine achieves throughput up to 2,238 seconds of audio processed per second.

Developers integrating local ASR capabilities into agent frameworks can review our analysis on structuring autonomous worker loops.

What We Don't Know Yet

While the initial technical release is thoroughly documented, several operational questions remain open:

• Real-World Self-Hosting Costs: While high throughput is documented on Blackwell NVFP4 systems, baseline cost-per-audio-hour metrics on standard Ampere and Hopper cloud instances have not been independently mapped across varying batch sizes.
• Audio Q&A Language Capabilities: The technical report explicitly notes that audio question answering was trained almost exclusively on English audio. Performance on non-English audio reasoning tasks remains largely unverified by independent benchmarks.
• Streaming Roadmaps: NVIDIA has not indicated whether the hybrid Mamba-MoE architecture will be adapted into a low-latency chunked variant to support interactive voice interfaces.

Frequently Asked Questions

Is Nemotron 3.5 Transcribe open source?

Nemotron 3.5 Transcribe is released as an open-weight model under the permissive OpenMDW-1.1 license. This license allows both non-commercial research and production commercial deployments without recurring licensing fees.

Can Nemotron 3.5 Transcribe process real-time streaming audio?

Nemotron 3.5 Transcribe cannot process live streaming audio; it is an offline batch model that requires complete audio recordings up to 60 minutes long. Interactive voice bots requiring sub-second response times should use dedicated streaming systems such as Nemotron 3.5 ASR Streaming 0.6B.

How many parameters does Nemotron 3.5 Transcribe have?

Nemotron 3.5 Transcribe contains 34 billion total parameters across its audio encoder and mixture-of-experts language backbone, but it activates only approximately 4.3 billion parameters per generated token during inference.

What languages does Nemotron 3.5 Transcribe support?

Nemotron 3.5 Transcribe supports automatic speech recognition across 40 languages, speech-to-text translation between English and 31 languages, and speaker attribution across 13 languages.

How does Nemotron 3.5 Transcribe perform speaker diarization?

Nemotron 3.5 Transcribe uses a Parallel Expert Encoder that combines acoustic features with speaker embeddings from a frozen diarization model, allowing the language decoder to label up to eight distinct speakers directly in the transcript.

Can Nemotron 3.5 Transcribe run on a single GPU?

Nemotron 3.5 Transcribe runs on a single enterprise or workstation GPU, including single NVIDIA H100 80GB cards in BF16, workstation DGX Spark systems, and RTX PRO 6000 Blackwell GPUs using NVFP4 precision.

What to watch next: Track whether the community releases optimized llama.cpp or vLLM execution kernels for edge execution, monitor independent multi-speaker benchmarks on accented telephone audio, and watch for third-party medical fine-tuning recipes using the open Granary v2 pipeline.

Try related models on Kie.ai: Gemini 3.8 Flash TTS and Gemini 3.1 Flash TTS.