NVIDIA Unveils Cosmos World Foundation Models to Accelerate Physical AI Development

NVIDIA has introduced the Cosmos World Foundation Models (WFMs) to support developers in accelerating the development of physical AI. The Cosmos platform allows developers to customize out-of-the-box pretrained models for specialized applications in robotics, autonomous vehicles, and industrial AI. These models are trained using unlabeled datasets and are designed to streamline the creation of physically accurate simulations and digital twins, significantly reducing the time and complexity involved in traditional development processes [1].

A key component of the Cosmos platform is the NVIDIA Cosmos Curator framework, which enables developers to efficiently filter, annotate, and deduplicate massive amounts of sensor data. This capability improves the quality and relevance of training data, leading to more accurate and robust AI models. Developers can use this curated data to create tailored datasets that meet specific needs in physical AI applications such as factory automation, warehouse logistics, and autonomous driving on complex terrains [2].

The Cosmos platform includes three foundational models: Predict, Transfer, and Reason. Predict generates continuous video sequences up to 30 seconds in length, based on multimodal inputs and user prompts. Transfer is a multicontrol model that simulates different environments and lighting conditions, supporting data augmentation for 3D inputs from simulation frameworks like CARLA and Isaac Sim. Reason is a fully customizable Vision Language Model that enables AI agents to understand and interpret operations in industrial and urban settings. These models collectively enhance the ability of AI systems to perceive, reason, and act in real-world environments [4].

NVIDIA also announced the release of the Omniverse libraries, powered by RTX PRO Servers and DGX Cloud. These libraries allow developers to build physically accurate digital twins, which are essential for training AI agents and physical AI models. The libraries support data interoperability between OpenUSD and MJCF, enabling cross-platform simulation of robots and industrial AI systems. The RTX ray-traced 3D Gaussian splatting technique further enhances the realism of these simulations by reconstructing real-world environments from sensor data [3].

According to Rev Lebaredian, Vice President of Omniverse and Simulation Technologies at NVIDIA, the company is committed to enabling the next generation of robotics and autonomous systems. He emphasized the convergence of AI and computer graphics as a transformative force in robotics, with potential to reshape industries worth trillions of dollars [6]. The Omniverse libraries and SDKs are now available for developers to build and deploy robotics and industrial AI applications, with notable adopters including Lightwheel, Moon Surgical, and Skild AI [5].

The Cosmos WFMs and associated tools are now available through NVIDIA’s developer resources, offering a structured learning path for developers. The company has also introduced new educational content to guide users in generating and curating synthetic data, further supporting the development of high-quality AI models [7].

The rollout of Cosmos WFMs marks a significant advancement in the field of physical AI, enabling developers to build more accurate, efficient, and scalable AI systems. As the industry continues to evolve, NVIDIA’s contributions underscore the company’s leadership in advancing AI-driven robotics and simulation technologies [8].