CARLA, the open source simulator for autonomous driving research, prized for its ability to serve as a testbed for everything from perception systems to full autonomy stacks has introduced formal support for NVIDIA's NuRec with Version 0.9.16.
First announced at SIGGRAPH 2025, NVIDIA’s NuRec is a neural reconstruction pipeline built on radiance field methods like Gaussian Splatting and 3DGUT.
Instead of relying solely on synthetic geometry, developers can now ingest real world sensor data from cameras, LiDAR, or recorded logs and re-render those scenes with neural graphics. With the lifelike 3D that comes out of NuRec, it becomes possible to render entirely new views for simulation. This showcases one of the most powerful emergent benefits of Gaussian Splatting: the ability to simulate the physical world around us. For perception training, it’s a major step toward datasets that feel truly real while still offering the controllability of simulation.
By using NuRec inside CARLA, researchers can generate hybrid environments, rooted in real world capture data, but flexible enough to randomize lighting, add objects, or stress test edge cases. That means a researcher can take a real urban intersection, replay it in CARLA, then layer in random pedestrians or even simulate rare “long-tail” scenarios like a child chasing a ball into the street.
Setting up NuRec requires a CARLA 0.9.16 installation, CUDA 12.8 or higher, and a Hugging Face account for accessing the sample datasets. NVIDIA provides a Docker image (carla/nvidia-nurec-grpc) along with Python replay scripts. Once installed, you can load pre trained scenarios from the NVIDIA Physical AI Dataset for Autonomous Vehicles and replay them directly inside CARLA, complete with multi-camera visualization and neural rendering from arbitrary viewpoints.
Full setup instructions are available in the official CARLA documentation.
This release marks the latest example of radiance fields moving from research into real-world tools, making a tangible difference for product teams. Expect deeper integrations between CARLA and NuRec as neural reconstruction continues to evolve and build CARLA here.
