Cosmos, GR00T, Nemotron: the open Physical AI stack, read from the data side

The first half of 2026 left Physical AI with something it did not have a year ago: a complete open stack. NVIDIA now publishes weights for every layer of it. Cosmos 3 generates and reasons about worlds. Isaac GR00T turns perception into robot actions. Nemotron 3 runs the agents that sit around both. We build our pipelines on parts of this stack, so this is less commentary than the map we actually work from. It is worth walking layer by layer, because each one changes what a synthetic data effort looks like, and none of them changes the part that decides whether the effort pays off.

Worlds: Cosmos 3

Cosmos 3 is a world foundation model that reads a scene before it generates one: a vision-reasoning tower works out what is happening, a generation tower produces the frames, the audio, and the raw action data, joint angles, trajectories, gripper states. NVIDIA says it was trained on 20 trillion tokens of multimodal data, including 400 million real and synthetic videos. It ships in three sizes, Nano at 16B, Super at 64B, and since July, Edge at 4B, with open weights on Hugging Face under the Linux Foundation's OpenMDW license. Edge runs in real time on Jetson and RTX-class hardware. Alongside the models NVIDIA released six synthetic data datasets and HUE, a benchmark that grades generated video on physics, geometry, and visual integrity.

For synthetic data work, this layer changes the economics of the generate step. Scenes that used to be hand-built in an engine can be described, and rare events your fleet has seen twice can be varied a thousand ways. The Cosmos Dreams demo at SIGGRAPH made the direction concrete: a closed-loop driving simulator producing photoreal, drivable worlds frame by frame on a single RTX PRO 6000 workstation card. Generation cost keeps falling, and because the weights are open, it falls for everyone at the same time.

Bodies: Isaac GR00T

GR00T N1.7 is the current release: an open vision-language-action model for humanoids under Apache 2.0, cross-embodiment, and commercially deployable, NVIDIA lists LG Electronics and NEURA among the teams already shipping on it. It is trained the way the whole field now trains, imitation learning, reinforcement learning, and video, with most of the hours accumulated in simulation rather than on hardware. The next step is visible too: GR00T N2, previewed for the end of the year, moves to a world-action design based on the DreamZero research and, per NVIDIA, succeeds at new tasks more than twice as often as leading VLA models. There is even a reference body now, a Unitree H2 Plus with tactile five-finger hands and Jetson Thor onboard, available from Unitree late this year.

This is the layer where synthetic data actually lands. A VLA is only as general as the distribution it trained on, and the training recipe, simulate, randomize, transfer, is now packaged and public. When everyone downloads the same policy architecture and the same simulator, the models stop being the differentiator. What you feed them is.

Agents: Nemotron 3

Nemotron 3 is NVIDIA's open family for agentic systems: Nano, Super, and Ultra, built on a hybrid Mamba-Transformer mixture-of-experts design that keeps a fraction of the parameters active per pass. Nano runs 30B total with about 3B active; Super, 120.6B with 12.7B active and a million-token context; Ultra is the frontier model of the family. An Omni variant adds audio and vision understanding. None of this is a robot model, and NVIDIA does not pitch it as one. Its place in a Physical AI shop is quieter: it is the layer that operates the tooling. With the Agent Toolkit's Omniverse libraries, an agent can lay out a simulation-ready scene, prep the physics, and run sensor simulation as tool calls.

In practice that targets the biggest human line items in a synthetic data project: scene assembly, asset wrangling, and the glue between pipeline stages. Those hours do not show up in papers, but they dominate real budgets.

What the stack assumes you bring

All three layers are generic by construction. They were trained on the world's distribution, not on your production line, your fleet's geography, or your sensor suite. The stack quietly assumes somebody supplies the specifics: the geometry and materials of your environment, the calibration of your cameras and lidars, the failure cases that matter to your safety team, and the acceptance criteria for calling any of it good.

That last item is the one no release this year has touched. HUE will tell you a generated clip is physically plausible. It will not tell you whether the distribution you generated covers the conditions your robot will meet, whether a model trained on it transfers, or how you will show an auditor the lineage from deployed behavior back to training frames, the kind of record the EU AI Act's Article 10 already expects. That work is still manual, still domain-specific, and still decides project outcomes.

So the honest summary of mid-2026: the open stack generates worlds, acts in them, and increasingly runs its own tooling. It is good, and we are glad to build on it. What it does not do is know your domain, or prove that data made in it will hold up outside it. datadoo's work sits exactly there, building physically grounded digital twins, generating with the open stack, and validating that the result transfers, with the evidence kept. The teams that treat that half as the project, rather than the afterthought, are the ones we expect to ship.