What Is Nemotron 3.5 Bolt? NVIDIA's 74B Model in 24GB

Inside Nemotron 3.5 Bolt: A 74B Agent Model Squeezed Into 24GB of Memory

Nemotron 3.5 Bolt is an open-weight multimodal mixture-of-experts model from NVIDIA. It fits 74 billion total parameters, 9 billion of them active per token, into a 24GB memory footprint so that coding, web-research and tool-calling agents can run on local hardware. NVIDIA released it on October 15, 2026, under the full name Nemotron 3.5 Bolt 74B-A9B 24GB. It takes text, image and video input, supports up to 256K tokens of context, and reaches about 84 tokens per second on RTX Spark in NVIDIA's testing. A smaller sibling, Bolt E4B 4GB, is scheduled for October 23, 2026.

Key Takeaways

• Nemotron 3.5 Bolt has 74B total parameters and 9B active per token. Its backbone is a hybrid Mamba-2/Transformer MoE with a C-RADIO vision encoder.
• It is not trained from scratch. NVIDIA compressed it from Nemotron 3.5 Super VL (120B-A12B) with Star Elastic, then quantized the experts to 2.0625 bits per element.
• Artificial Analysis scores it 21 on its Intelligence Index, which ranks #4 of 67 comparable open models. The class median is 8.
• NVIDIA reports 50.98 on Terminal-Bench 2.1 and 66.75 on BrowseComp for the 24GB file, and ranks it #1 on PinchBench among models with a similar memory footprint.
• The weights ship under the OpenMDW License v1.1, which allows commercial use. Day-one runtimes include llama.cpp, vLLM, SGLang, Ollama, LM Studio and Unsloth.
• Early community tests agree on speed. Reliability and structured-output support are the main open complaints.

What Is Nemotron 3.5 Bolt?

Nemotron 3.5 Bolt is NVIDIA's local-agent tier in the Nemotron 3.5 family. In NVIDIA's lineup, Lightning covers high-throughput workloads, Ultra covers the hardest reasoning, Super VL covers multimodal enterprise agents, and Bolt covers edge and local machines. The NVIDIA launch blog says Bolt is "designed to run everyday agentic tasks locally on NVIDIA RTX/DGX Spark" and runs alongside other applications.

The design brief was narrow. According to the NVIDIA research page, the model had to fit into 24GB, decode at no less than 80 TPS, and be as capable as possible within those limits. The release timing matched the RTX Spark laptop launch on October 16, 2026.

Nemotron 3.5 Bolt is a 74B-parameter Super VL derivative that NVIDIA compressed and quantized to 2 bits so a full multimodal agent model fits in 24GB of memory.

Nemotron 3.5 Bolt at a Glance

Attribute	Nemotron 3.5 Bolt 74B-A9B 24GB
Developer	NVIDIA
Type	Hybrid Mamba-2/Transformer mixture-of-experts, reasoning + tool use
Parameters	74B total, 9B active per token
Parent model	Nemotron 3.5 Super VL (120B-A12B)
Modality	Input: text, images, video (per NVIDIA); output: text
Context window	Up to 256K tokens (262K per Artificial Analysis; Unsloth's guide lists 128K)
Quantization	IQ2_XXS experts (2.0625 bits), NVFP4 Mamba layers, Q8_0 attention and head
Checkpoint size	22.45 GiB (NVIDIA 24GB pack)
Speed (vendor)	~84 TPS on RTX Spark with LiLiCorr drafter
Pricing	Free weights; no first-party API pricing published
Availability	Hugging Face, ModelScope, Ollama, LM Studio, llama.cpp, vLLM, SGLang
License	OpenMDW License Agreement v1.1
Languages	Model card lists 6 languages; full list not yet confirmed

How Nemotron 3.5 Bolt Works

Bolt rests on four named techniques. Together they form a compress-then-quantize pipeline.

Star Elastic compression. NVIDIA's Star Elastic framework does router-guided architecture search. A budget-conditioned router chooses expert counts and MoE widths layer by layer. For Bolt it cut the parent from 120B total and 12B active down to 74B and 9B, and lowered the expert top-K from 22 to 9. The size target already accounted for the quantization step that came afterward. The parent model is covered in our Nemotron 3.5 Super VL reference page.

Source: @ItsmeAjayKV

Post-training recovery. Compression damaged the vision capability, so NVIDIA added post-training stages to bring it back. After that came end-to-end reinforcement learning in simulated environments for local agent skills. A last RL pass tuned reasoning paths and tool execution for token efficiency.

Quantization-Aware Distillation (QAD). All experts are compressed to IQ2_XXS at 2.0625 bits per element, and QAD recovers the lost accuracy. NVIDIA reports that the 2-bit file keeps most of its BF16 quality: PinchBench goes from 90.35 to 87.82, BrowseComp from 70.0 to 66.75, and GPQA Diamond from 88.83 to 86.43.

Source: @TeksEdge

LiLiCorr drafter. NVIDIA trained a LiLiCorr speculative-decoding drafter. NVIDIA says it gets a higher acceptance rate than DSpark and DFlash2, and it is what produces the 84 TPS figure on the AIPerf SPEED-Bench benchmark on RTX Spark.

Unsloth's documentation calls the backbone a Hybrid LatentMoE: Mamba-2, attention and MoE layers combined. A community DGX Spark recipe counts 88 blocks: 40 Mamba, 40 MoE and 8 attention. It also reports up to 496 experts, with 9 routed per token. That breakdown comes from a single third-party source.

NVIDIA's main research claim is worth quoting: for a fixed memory limit, compressing a large model and quantizing it to 2 bits beats training a smaller model from scratch and quantizing it to 4 bits.

What You Can Do With Nemotron 3.5 Bolt

NVIDIA pitches Bolt for delegating projects to local agent harnesses: web research, writing and debugging code, with none of the work sent to the cloud. Ollama's listing supports launching it inside Claude Code, OpenCode, Hermes Agent and OpenClaw. It lists vision, tool calling and switchable thinking as core features.

• Local coding agents. NVIDIA calls it smart enough to finish most jobs and much faster than the smartest models.
• Web research and browsing. It ranks #2 on WebArena and #2 on BrowseComp among models with a similar memory footprint, according to NVIDIA.
• Visual understanding. It reads screenshots, charts and documents, and video input is supported.
• Multi-user local serving. In Mia's DGX Spark test, it served 16 concurrent streams with 256K context, at 71 tok/s single-stream and 166 tok/s across 8 streams. About 60GB stayed free for other tools such as ComfyUI.

Not every early test was positive. In a 480-trial head-to-head, Bolt ran at about 85 tok/s against 41 tok/s for Qwen3.8 27B on an RTX 3090. Its task pass rate was lower, though: 62.5% versus 74%. Another hands-on agent test scored tool calling at 23/24 and reliability at 13/20. That tester also found that the LiLiCorr drafter breaks prefix-cache reuse in vLLM.

How Nemotron 3.5 Bolt Compares

The figures below come from model cards. Bolt's scores are for the 2-bit file, and the others are typically 16-bit.

Benchmark	Nemotron 3.5 Bolt (2-bit)	Qwen3.6-35B-A3B	Nemotron 3.5 Super VL
Parameters (total / active)	74B / 9B	35B / 3B	120B / 12B
Terminal-Bench 2.1	51.0	44.4	66.70
GPQA Diamond	86.4	83.4	89.52
HLE	25.3	19.6	29.20
BrowseComp	66.8	48.7	Not yet confirmed
AA Intelligence Index	21	Not yet confirmed	26

The Qwen3.6 figures come from NVIDIA's launch table. The Super VL figures come from a LocalLLaMA model-card comparison, and the same thread puts Qwen 3.8 27B at 34 on the AA index. On NVIDIA's numbers, Bolt leads Qwen3.6-35B-A3B by 18.1 points on BrowseComp, but it trails newer Qwen 3.8 models on independent aggregates. The independent score is available on Artificial Analysis. It also flags that Bolt is verbose: 230M output tokens on the index, against a median of 100M.

Availability: How to Access Nemotron 3.5 Bolt

NVIDIA publishes the model in several Hugging Face variants:

• NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB works with Transformers, vLLM and SGLang.
• Official 24GB GGUF works with llama.cpp, LM Studio and Ollama.
• BF16 checkpoint is meant for post-training and custom quantization.
• LiLiCorr drafter handles speculative decoding.

Upstream llama.cpp supports the architecture without a custom fork. Unsloth's GGUF repo offers 1-bit and 2-bit quants between 20.7GB and 23.6GB. NVIDIA also mirrors the release on ModelScope. Deployment instructions are CUDA-only, and Apple Silicon is not officially supported. Full methodology is in the technical report.

No first-party hosted API exists today. Teams that need a hosted endpoint for similar agent work in the meantime can call a comparable chat model such as DeepSeek-V4.1-Flash over an API and keep Bolt for local runs.

What We Don't Know Yet

• Real 24GB-GPU behavior at long context. The 24GB figure covers the weights only. KV cache, images and the drafter push practical needs to roughly 32GB or more.
• Context limit in practice. NVIDIA states 256K, while Unsloth's guide caps it at 128K.
• Quant fidelity. One community quantizer claims a 2-bit EXL3 build matches BF16's top token 84% of the time, against about 80% for NVIDIA's file. This is a single, unconfirmed source.
• Structured output. One tester reports that forced tool_choice and structured output return HTTP 400.
• Bolt E4B specs. NVIDIA has published only its 4GB footprint and its target devices.

Frequently Asked Questions

Is Nemotron 3.5 Bolt open source?

Nemotron 3.5 Bolt is open-weight, released under the OpenMDW License Agreement v1.1, which supports commercial and non-commercial use. The model card references NVIDIA's Nemotron pre-training and post-training datasets. OpenMDW is not Apache or MIT, so review its terms before you ship.

How much memory does Nemotron 3.5 Bolt need?

The model footprint is 24GB, and NVIDIA's packed checkpoint is 22.45 GiB. Unsloth's guide budgets about 32GB of RAM or unified memory for short contexts and 48–64GB for headroom. A full 16-stream DGX Spark setup used 51.1 GB.

How much does Nemotron 3.5 Bolt cost?

The weights are free to download. NVIDIA has not published per-token API pricing, and Artificial Analysis lists no public API for the model. Your cost is the hardware you run it on.

Nemotron 3.5 Bolt vs Qwen3.6-35B-A3B: which is better?

On NVIDIA's own table, Bolt wins on Terminal-Bench 2.1 (51.0 vs 44.4), GPQA Diamond (86.4 vs 83.4), HLE (25.3 vs 19.6) and BrowseComp (66.8 vs 48.7). These are vendor-reported numbers. Against newer Qwen 3.8 models, independent tests show Bolt passing fewer tasks while generating tokens about 2x faster.

What is the difference between Nemotron 3.5 Bolt and Nemotron 3.5 Super VL?

Bolt is a compressed Super VL. Star Elastic cut it from 120B/12B to 74B/9B, and 2-bit QAD then shrank it to 24GB. Super VL needs about 70–82GB of RAM locally and targets server-class agents. For how Super VL compares with frontier closed models, see our Super VL vs Claude Opus 5.5 breakdown.

Can Nemotron 3.5 Bolt run on a Mac?

Not officially. NVIDIA provides CUDA instructions, and Ollama lists Apple Silicon/Metal as unsupported. Community MLX conversions exist, and one Reddit tester reported running IQ2_XXS on a 32GB Mac. These routes are unofficial.

What is Nemotron 3.5 Bolt E4B?

Bolt E4B 4GB is the smaller sibling, scheduled for October 23, 2026. NVIDIA built it for conversational and physical agents on GeForce RTX, Jetson and RTX/DGX Spark. Its full specs and benchmarks are not yet published.

What to Watch Next

Three signals will shape this page's next update. The first is independent Intelligence Index and agent-suite results for 4-bit community quants compared with NVIDIA's 2-bit file. The second is whether llama.cpp and vLLM fix the drafter's prefix-cache conflict and add structured-output support. The third is the Bolt E4B release on October 23, along with its benchmark sheet.

Try related models on Kie.ai: Gemini 3.8 Flash, Claude Haiku 5.5 and Gemini 4 Argon.