Today at SIGGRAPH, NVIDIA set its sights firmly on the future of robotics, simulation, and large scale world reconstruction and radiance fields are at the epicenter. The company announced a wave of new Omniverse libraries, including NuRec 3D Gaussian Splatting, alongside updates to its Cosmos world foundation models (WFMs), all designed to push the limits of physically accurate digital twins, real world capture, and synthetic data generation.
Among the biggest announcements is Omniverse NuRec, a suite of libraries and AI models that introduce ray traced 3D Gaussian Splatting through NVIDIA’s 3D Gaussian Unscented Transform. This technology lets developers capture, reconstruct, and simulate the real world from sensor data with remarkable fidelity. By bringing neural rendering directly into Omniverse workflows, NVIDIA is making it easier to bridge the gap between physical reality and simulation. NVIDIA has also released a tutorial showing how to import 3DGUT captures into Isaac Sim.
NuRec isn’t staying in the lab either. It’s already integrated into CARLA, the open source simulator used by over 150,000 developers for autonomous vehicle testing. Industry leaders like Foretellix are using it in combination with Omniverse Sensor RTX and Cosmos Transfer to generate physically accurate driving scenarios at scale. Voxel51’s FiftyOne, a popular data engine for visual AI used by Ford and Porsche, now supports NuRec to streamline preparing datasets for 3D reconstructions.
While much of the announcement is framed in the context of robotics and simulation, the inclusion of 3D Gaussian Splatting in Omniverse NuRec is a significant development for the radiance field community. It signals a push toward large scale, photorealistic world reconstruction in environments where accuracy and interoperability are non-negotiable. As these tools make their way into open-source frameworks, industrial robotics, and AV simulation, the techniques pioneered in radiance field research are poised to become part of the standard toolkit for building the next generation of physically aware AI.
Aaron Lefohn, vice president of graphics research and head of the Real-Time Graphics Research group at NVIDIA puts the advantages of radiance fields bluntly.
“We’re now at a point where we can take pictures and videos — an accessible form of media that anyone can capture — and rapidly reconstruct them into virtual 3D environments.”
The barrier to entry for radiance fields is incredibly low, but the potential impact they have on businesses is significant. The advances on lifelike 3D have not gone unnoticed at some of the world’s largest industries and companies.
New to Gaussian Splatting? Start here
