Cosmos-Transfer2

A multi-controlnet world foundation model that transforms structured simulation data into physics-aware, photorealistic video for physical AI training.

Cosmos-Transfer2.5 is a diffusion transformer model engineered by NVIDIA to bridge the gap between simulation and reality for robotics and autonomous vehicles. By processing multimodal inputs like depth maps, segmentation masks, and RGB video, the 2B-parameter model generates high-fidelity world simulations that maintain strict temporal and physical consistency. It is 3.5 times smaller than its predecessor, Cosmos-Transfer1-7B, yet delivers superior prompt alignment and significantly lower error accumulation in long-horizon video generation. Developers use it to scale synthetic datasets with precise control over environmental variables (lighting, weather, and object placement) ensuring that perception models trained in Sim2Real pipelines translate reliably to real-world hardware.

https://github.com/nvidia-cosmos/cosmos-transfer2.5