NVIDIA Releases Four Cosmos 3 Nano Models for World Simulation

NVIDIA Cosmos 3 Nano models released

NVIDIA just dropped four new open-weight checkpoints for its Cosmos 3 family. All of them stem from the 16-billion-parameter Cosmos 3 Nano architecture and land under the OpenMDW 1.1 license, so commercial and non-commercial use are both fair game. The news hit Hugging Face on November 5, 2026, complete with technical reports, inference endpoints, and notebooks for each model. If you're building autonomous vehicles, robotics simulators, or basically any system that needs to predict how the world reacts to actions, these checkpoints give you a shared foundation you can adapt to custom sensor rigs, embodiment types, and operating domains.

The four models cover complementary stages of the simulation pipeline. Cosmos3-Nano-Transfer-Auto runs offline, ingesting per-camera World Scenario Map controls and text prompts to generate synchronized video across up to eleven camera views-fisheye lenses included-plus aligned LiDAR point clouds. You can tweak weather, lighting, and time of day, and anchor the output with an initial RGB frame when you need consistency with a specific scene. Since Transfer-Auto is built as a base for further post-training, teams can fine-tune it on their own camera layouts and LiDAR configs before moving to streaming inference.

Cosmos 3 Nano model variants and use cases

Cosmos3-Nano-Sim-Auto is the causally distilled streaming counterpart. It pumps out successive chunks of synchronized camera and LiDAR data in just a few denoising steps, conditioning each chunk on the generated history and updated driving controls. That design enables long, interactive rollouts where a driving policy can steer the simulation in real time. The recommended workflow? Adapt Transfer-Auto to your target domain first, then distill that adapted checkpoint into a Sim-Auto variant matching the same sensor rig.

For researchers already running a physics simulator, Cosmos3-Nano-Sim-Transfer bridges the gap between ground-truth state and visual observations. The simulator hands over depth maps, segmentation masks, edge maps, or blur cues, while a text prompt guides appearance. The model then renders photorealistic RGB frames that a perception stack or policy can consume. Physics and exact geometry stay the simulator's job; the model only handles the visual translation layer.

Cosmos3-Nano-Sim-Bimanual targets robotic manipulation. Given an initial image and a sequence of bimanual actions-59 control values padded to 64 during training, covering coordinate frames, rotations, gripper calibration, and validity masks-it predicts the next observation. A single control sequence can switch between moving the camera and moving the arms, supporting both YAM and AgiBot embodiments. The release includes a detailed data contract and action schema so developers can format their own control streams correctly. Running the model requires Linux, an NVIDIA GPU, Docker, and the NVIDIA Container Toolkit; the published vLLM-Omni setup was validated on a single GB200, though that's not presented as a minimum spec.

Cosmos 3 Nano MoT backbone architecture

All four checkpoints share the same Mixture-of-Transformers backbone, connecting a text-processing network and a visual-generation network through cross-attention. During streaming generation, the model receives controls and a key-value cache of prior history, predicts a short video segment by denoising compressed representations, decodes the frames, and updates the cache for the next step. Temporal-causal attention ensures the model can read past observations and current controls while blocking access to future targets, a design that keeps each prediction consistent with the unfolding trajectory.

The open weights and post-training recipes are meant to give the community a common starting point for comparing methods, contributing improvements, and measuring where these models faithfully support policy learning versus where they fall short. NVIDIA's stated vision is to help researchers build richer, more responsive simulated worlds, but the company acknowledges that evaluating fidelity for downstream tasks remains an open problem.

How to use Cosmos 3 Nano checkpoints

Developers wanting to experiment should start by pulling the checkpoints from Hugging Face and running the provided notebooks to understand each model's input contracts. Teams with custom sensor suites should plan a two-stage adaptation: fine-tune Transfer-Auto or the relevant base model on domain data, then distill to a streaming variant if interactive rollout is required. Robotics groups can test Sim-Bimanual against their own action schemas, paying close attention to the timing convention that T transitions require T+1 observations. Across all use cases, the immediate work is to benchmark these models in closed-loop policy training and evaluation, document failure modes, and feed those findings back into the shared codebase.

Frequently asked questions

What license covers the Cosmos 3 Nano models and what uses does it permit?

All four Cosmos 3 Nano checkpoints are released under the OpenMDW 1.1 license, which allows both commercial and non-commercial use. The models were published on Hugging Face on November 5, 2026, along with technical reports, inference endpoints, and notebooks for each variant.

How do Cosmos3-Nano-Transfer-Auto and Cosmos3-Nano-Sim-Auto differ in their intended use?

Transfer-Auto runs offline, generating synchronized video across up to eleven camera views plus aligned LiDAR from World Scenario Map controls and text prompts. It serves as a base for fine-tuning on custom sensor rigs. Sim-Auto is the causally distilled streaming counterpart that produces successive chunks of camera and LiDAR data in few denoising steps, enabling real-time interactive rollouts conditioned on generated history and updated driving controls.

What is the role of Cosmos3-Nano-Sim-Transfer in a simulation pipeline?

Sim-Transfer bridges physics simulators and visual observations. It takes depth maps, segmentation masks, edge maps, or blur cues from a simulator along with a text prompt, then renders photorealistic RGB frames for perception stacks or policies. The physics simulator retains responsibility for exact geometry and physics; the model handles only the visual translation layer.

What input does Cosmos3-Nano-Sim-Bimanual require for robotic manipulation tasks?

Sim-Bimanual requires an initial image and a sequence of bimanual actions-59 control values padded to 64 during training, covering coordinate frames, rotations, gripper calibration, and validity masks. It predicts the next observation and supports both YAM and AgiBot embodiments. The release includes a detailed data contract and action schema for formatting control streams.

What hardware and software are needed to run the Cosmos 3 Nano models?

Running the models requires Linux, an NVIDIA GPU, Docker, and the NVIDIA Container Toolkit. The published vLLM-Omni setup was validated on a single GB200, though this is not presented as a minimum specification. Developers should start by pulling checkpoints from Hugging Face and running the provided notebooks to understand each model's input contracts.

Source: Hugging Face