Revolutionizing Interactive Simulation with NVIDIA Omniverse NuRec and 3DGUT

NVIDIA Omniverse NuRec and 3DGUT enable the reconstruction of photorealistic 3D scenes from simple sensor data, which can be deployed instantly in NVIDIA Isaac Sim or CARLA Simulator. The process involves capturing real-world data, training a reconstruction, and loading the results into Isaac Sim. This streamlines neural reconstruction and rendering, improving sim-to-real transfer for robot training in realistic simulations.

NVIDIA has made significant strides in advancing robotics and autonomous vehicle development with the introduction of new libraries and models under the NVIDIA Omniverse and NVIDIA Cosmos platforms. The company’s latest innovations, NVIDIA Omniverse NuRec and 3DGUT, enable the reconstruction of photorealistic 3D scenes from simple sensor data, which can be deployed instantly in NVIDIA Isaac Sim or CARLA Simulator. This technology streamlines neural reconstruction and rendering, enhancing the sim-to-real transfer for robot training in realistic simulations.

The NVIDIA Omniverse NuRec libraries leverage 3D Gaussian splatting, a rendering technique that allows developers to capture, reconstruct, and simulate the real world in 3D using sensor data. This technology is integrated into CARLA, a leading open-source simulator used by over 150,000 developers, and is being adopted by companies such as Boston Dynamics, Figure AI, and Hexagon to accelerate their AI robotics development.

In addition, NVIDIA Cosmos models, downloaded over 2 million times, offer advanced synthetic data generation capabilities. The Cosmos Transfer model simplifies prompting and accelerates photorealistic synthetic data generation, while Cosmos Reason introduces a 7-billion-parameter reasoning VLM for physical AI and robotics. These models enhance the ability of robots and AI agents to understand and act in the real world, improving data curation, annotation, and planning.

To support these advanced technologies, NVIDIA has also announced AI infrastructure designed for the most demanding workloads. The NVIDIA RTX PRO Blackwell Servers and NVIDIA DGX Cloud offer a single architecture for every robot development workload across training, simulation, and deployment. These servers are available on Microsoft Azure Marketplace and are being adopted by industry leaders such as Accenture and Hexagon.

NVIDIA's advancements in 3D scene reconstruction and AI infrastructure are set to significantly impact the robotics and autonomous vehicle industries. By enabling more realistic and efficient simulations, these technologies are poised to transform how robots and autonomous vehicles are developed and deployed, potentially leading to trillions of dollars in industry transformations.

References:
[1] https://finance.yahoo.com/news/nvidia-opens-portals-world-robotics-150100737.html