Blog
Nemotron 3.5 Changes What a Local AI Workstation Can Do
NVIDIA’s new multimodal agent family can read documents, inspect images and video, use tools, and sustain long workflows. Here is what that means for capable local machines—and the honest reality for a 128GB Apple-silicon workstation.
NVIDIA just moved the local-agent conversation from “Can a model fit?” to “What useful work can the entire system complete?”
Nemotron 3.5 Super VL is a 120-billion-parameter multimodal mixture-of-experts model built for long, tool-using workflows across text, images, webpages, PDFs, charts, tables, code, and video. Only 12 billion parameters are active per token, a design meant to deliver far more capability than a conventional model with the same active compute.
That makes this release genuinely important for people building private AI infrastructure. It does not, however, mean that every 128GB Mac can download NVIDIA’s flagship checkpoint tonight and run it perfectly. The model family includes different sizes and formats, and the official Super VL software path currently targets NVIDIA GPUs on Linux. The opportunity is enormous—but the useful story begins with choosing the right model and runtime.
The short version: Super VL is the 120B flagship, roughly 75GB as published, with 12B active parameters and an advertised context window of up to one million tokens. Nemotron 3.5 Bolt is the more immediate local candidate: 74B total, 9B active, a roughly 24GB GGUF package, and support links for local runtimes such as llama.cpp, Ollama, and LM Studio. On Apple silicon, model size may fit while software support remains the gating factor.
What NVIDIA actually released
The Nemotron 3.5 family is not one model. It is a set of open models aimed at different deployment envelopes:
Nemotron 3.5 Super VL: 120B total parameters, 12B active, multimodal input, tool use, and up to a one-million-token context window. NVIDIA publishes it in an NVFP4 format and provides deployment recipes for DGX Spark, H200, H100, and GB200 systems.
Nemotron 3.5 Bolt 74B-A9B: 74B total parameters with 9B active, offered as an aggressively quantized GGUF package of about 24GB. NVIDIA positions it for local agent harnesses, coding, research, and tool use.
Nemotron 3.5 Bolt E4B: a smaller edge-oriented option around 4GB for lower-resource systems and narrower tasks.
NVIDIA says Super VL was trained on roughly 26 trillion tokens and then post-trained on 3.8 million examples using supervised fine-tuning and reinforcement learning. Its architecture combines Transformer attention with Mamba-2 state-space layers, a mixture-of-experts system, and NVIDIA’s C-RADIO vision encoder. Those choices are designed to keep long-context, multimodal work practical without activating the entire 120B model for every token.
NVIDIA reports that Super VL delivers up to five times the throughput of the next-best open model in its class. The company’s technical post also reports strong scores for tool use and chart reasoning, including 80.8% on CharXiv that rises to 84% when the model can use a Python interpreter. Those are vendor-reported results, not a promise that every local workload will see the same speed or accuracy.
Why a multimodal agent is different from a chatbot
A chatbot waits for a question and returns text. An agent receives a goal, examines information, chooses tools, observes the result, and continues until it reaches an answer or a stopping condition. A multimodal agent can do that while understanding far more than a prompt box.
It can read a 200-page PDF, inspect a chart embedded on page 143, compare that chart to a spreadsheet, search a public website, run Python to verify the math, and produce a cited report. It can watch a product demo, read the interface shown in the recording, inspect an error message, open the relevant code, and suggest a fix. The value is not simply that the model can “see.” The value is that seeing, reasoning, and acting happen inside one workflow.
NVIDIA’s examples include converting receipts into structured JSON, reviewing vendor credit, accelerating merger-and-acquisition due diligence, understanding cybersecurity evidence, and using tools inside coding workflows. For a local business, the same primitives apply to less glamorous but highly valuable work: invoices, contracts, estimates, marketing assets, website screenshots, inventory records, recorded calls, and operating procedures.
What capable local machines can do with it
1. Build a private document operations desk
A local agent can index contracts, proposals, PDFs, scanned invoices, spreadsheets, and policy documents without sending every page to a third-party API. It can extract dates and obligations, flag missing fields, compare versions, reconcile totals, and prepare a human-review queue. Multimodal reasoning matters because real business documents are not clean text: they contain tables, charts, stamps, signatures, photographs, and irregular layouts.
2. Run evidence-based research workflows
An agent can search the web, open results, read pages, follow source links, compare claims, and draft a report that preserves the evidence trail. With a long context window, it can keep more of that source material available during the analysis. The result can be a market brief, a competitor scan, a vendor comparison, or a local-business dossier assembled on infrastructure the operator controls.
3. Operate as a coding and systems assistant
With controlled terminal and repository access, the model can inspect a codebase, reproduce a bug, read logs and screenshots, edit files, run tests, and report what changed. The visual component is useful for browser QA: an agent can inspect the rendered page rather than declaring success because a build command passed.
4. Review image and video at scale
A capable multimodal model can classify product photographs, examine creative variations, summarize recorded training, locate relevant moments in video, compare before-and-after images, and produce searchable descriptions. That turns an archive of otherwise opaque media into usable business memory.
5. Connect business systems through tools
Tool calling allows the model to interact with a CRM, calendar, file store, browser, accounting export, or an MCP-compatible service. A well-designed agent could collect a lead, research the company, draft a personalized follow-up, create a CRM record, and schedule a reminder. High-impact actions should still require approval. The model’s job is to assemble evidence and prepare the work—not silently grant itself authority.
6. Become a specialized local expert
NVIDIA is publishing model weights, datasets, recipes, and customization support through NeMo. That gives organizations a path to evaluate or adapt the model for a domain rather than relying only on a general-purpose cloud assistant. A contractor could build an estimating assistant; a law office could create a privileged document-review system; a marketing agency could build a brand and campaign analyst. Each requires careful evaluation, but the infrastructure is becoming ownable.
Why the long context window matters
Super VL advertises context up to one million tokens. That is enough theoretical space for large document collections, long videos represented as frames and transcripts, substantial codebases, or extended agent histories. NVIDIA also highlights prefix caching and context reuse, which reduce the cost of repeatedly processing the same large body of information.
But maximum context is not free. The key-value cache grows with context length, and concurrency consumes additional memory. A model card’s maximum is not the same as a sensible first deployment. A local installation should begin with a smaller context—perhaps 32K or 64K—measure memory and latency, and expand only when the use case proves it needs more.
The honest 128GB Mac assessment
A 128GB Apple-silicon workstation is an unusually capable local-AI machine because its unified memory can hold large quantized models while the GPU accesses the same memory pool. A 4TB SSD provides room for multiple checkpoints, embeddings, document indexes, caches, datasets, and backups. That is serious infrastructure.
There is one important specification check: Apple’s official specification for the 2021 M1 Max MacBook Pro tops out at 64GB of unified memory, although it could be configured with a 4TB SSD. If “About This Mac” reports 128GB, the machine uses a later Max-generation chip. That distinction matters because the chip generation changes GPU performance and runtime compatibility.
The published Super VL repository is roughly 75GB. That suggests a 128GB unified-memory machine has enough raw capacity to store the weights in memory with room left for the operating system and some runtime overhead. Capacity is not compatibility. NVIDIA’s official Super VL checkpoint, NVFP4 kernels, and deployment recipes target CUDA on NVIDIA Blackwell or Hopper hardware. NVIDIA does not currently publish an official macOS/Metal deployment path for Super VL.
In other words, the 128GB Mac may be big enough in memory and still lack the software path needed to run the flagship efficiently. A future MLX or compatible GGUF conversion could change that. Today, promising it as a plug-and-play Mac model would be inaccurate.
Bolt is the practical local starting point
Nemotron 3.5 Bolt 74B-A9B is the more realistic experiment. Its main GGUF file is about 22GB and its multimodal projector adds roughly 1.7GB, producing a package of approximately 23.75GB before runtime overhead. NVIDIA links to llama.cpp, Ollama, LM Studio, Unsloth, and vLLM integrations from the model page.
Because GGUF is the format commonly used by llama.cpp-based local applications, Bolt is structurally closer to the Apple-silicon ecosystem. Still, a file labeled GGUF is not automatic proof of Mac support. The local runtime must understand this model architecture, its mixture-of-experts routing, its vision projector, and the required operators on Metal. The correct process is to confirm support in a current llama.cpp build, start with text inference, then test images, tool calls, and long context separately.
A practical memory budget would allow roughly 24GB for the published model files plus substantial headroom for context cache, multimodal inputs, speculative decoding, the operating system, and the agent process. NVIDIA’s own local recipes use 28GB and 32GB classes of GPU memory depending on configuration. A 128GB unified-memory Mac therefore has generous capacity for Bolt if the runtime path works.
Three deployment paths that make sense
Apple-silicon pilot: Test Bolt through a current compatible llama.cpp, Ollama, LM Studio, or MLX path. Begin with one user, one sequence, and a moderate context window. Validate text, vision, JSON, and tool calling before building production workflows.
Official NVIDIA local appliance: Run Super VL through NVIDIA’s documented DGX Spark or data-center GPU recipe. This is the cleanest route when the flagship model, NVFP4 acceleration, and vendor-supported stack matter more than using the Mac as the inference device.
Private hybrid system: Use the Mac as the orchestration, storage, indexing, and user-interface machine while an NVIDIA workstation or server on the local network performs inference. Sensitive source files can remain on premises, and each machine does the job its architecture handles best.
What “always-on agent” should mean
An always-on agent is not a model continuously improvising with unrestricted access. It is a controlled service that wakes for events, works inside an explicit boundary, records its actions, and asks for approval when consequences are meaningful.
A responsible local stack should include:
Least-privilege tools: read-only access by default and narrowly scoped write permissions.
Approval gates: human confirmation before sending messages, publishing content, spending money, deleting files, or changing production systems.
Audit logs: a record of sources opened, tools called, files changed, and outputs delivered.
Sandboxing and allowlists: separation between agent experiments and sensitive production data.
Task-specific evaluations: tests for the documents, images, and decisions the business actually handles—not only public benchmark scores.
Backups and recovery: local ownership is valuable only when the local system is resilient.
Models can hallucinate, misread small print, call the wrong tool, or persist in a bad plan. More agentic capability increases the importance of permissions and observability. Local inference improves data control; it does not eliminate operational risk.
What this changes for local businesses
The biggest change is economic and organizational. A business may be able to operate a substantial AI capability as owned infrastructure, use it across many internal workflows, and keep sensitive material closer to home. Once the model is downloaded, the marginal cost of another document review or coding iteration is mainly electricity, hardware time, and human supervision rather than a metered API call.
That creates room for persistent systems that were previously too expensive or privacy-sensitive: an internal knowledge desk, a nightly website and reputation auditor, a contract and invoice reviewer, a marketing research analyst, a code maintenance agent, or a media archive that can finally answer questions about its contents.
The advantage will not come from simply installing a model. It will come from connecting the model to clean data, safe tools, repeatable processes, and measurable business outcomes. The companies that win will know exactly which hour of human work the agent is improving and how they verify the result.
The bottom line
Nemotron 3.5 is a major release because it brings open, multimodal, tool-using intelligence closer to infrastructure an organization can own. Super VL demonstrates the upper end: long context, vision, documents, video, coding, and multi-step tool use in one model. Bolt offers a more practical path toward local experimentation.
For a 128GB Apple-silicon machine with 4TB of storage, the opportunity is real. It is an excellent orchestration, knowledge, and potentially inference platform. The honest near-term plan is to verify the exact chip, test Bolt with a compatible Apple runtime, and use an NVIDIA system for Super VL until a dependable Metal-native conversion exists.
The breakthrough is not that every local computer can run every frontier model today. It is that a serious, private agent stack—one that can see, read, reason, code, and use tools—is becoming something a small organization can realistically own.
Sources and further reading
NVIDIA Newsroom: Nemotron 3.5 Super VL Brings Frontier-Level Intelligence to Specialized AI Agents
NVIDIA Technical Blog: Nemotron 3.5 Super VL Powers Specialized Agents With Multimodal Reasoning
NVIDIA Nemotron 3.5 Super VL model card and files
NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB GGUF model card and files