Interview with Nick Schneider, Neural Reconstruction Manager at NVIDIA

I sit down with NVIDIA's Neural Reconstruction Manager, Nick Schneider.

I sit down with NVIDIA's Neural Reconstruction Manager, Nick Schneider.

Michael Rubloff

Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

Tags

NVIDIA Logo

At SIGGRAPH 2025, NVIDIA unveiled NuRec, its latest leap in neural reconstruction technology. Building on a rapid succession of breakthroughs, Instant-NGP in 2022, 3D Gaussian Splatting in 2023, and subsequent extensions like Gaussian ray tracing, NuRec signals another step toward merging real-world environments with simulation and robotics.

To better understand this moment, I sat down with Nick Schneider, NVIDIA’s Neural Reconstruction Manager, who joined the company after a decade in the automotive industry and a stint leading autonomous driving validation at TuSimple Robotics. Our conversation ranged from his first hands on work with radiance field representations to the role of standardization, robotics, and simulation in shaping what he calls “a future where everything that moves should be autonomous.”

Interview

Michael: Nick, thank you for joining me today at SIGGRAPH. NVIDIA just announced NuRec during the special address, and I’m excited to dive into it with you.

Nick: Thanks, Michael. It’s great to be here. 

Michael: SIGGRAPH has always been central to the development of radiance field representations, and it’s exciting to see how much has emerged here. Just a couple of years ago, Instant-NGP won Best Paper at SIGGRAPH, then the following year 3D Gaussian Splatting won. Now we’re here with the announcement of NuRec. Do you remember when you first encountered radiance field representations? Was it NeRF, Instant-NGP, or something else?

Nick: My background is a bit different. Before joining NVIDIA, I spent over ten years in the automotive industry. In 2023, I was at TORC Robotics, leading the team responsible for validating our autonomous driving stack.

That’s a difficult challenge—when you’re running 40-ton trucks, mistakes can be scary. We needed better simulation, but most conventional simulators weren’t good enough. Our AI-based software didn’t behave the same way in simulation as it did in the real world.

So, we explored new methods. We began with NeRFs but quickly moved to Gaussian Splatting because we needed speed in our pipeline and the ability to composite different assets. That was my first hands-on experience developing with NeRFs and Gaussian Splatting for simulation.

Michael: The pace of progress has been remarkable. After 3D Gaussian Splatting, NVIDIA introduced Gaussian ray tracing and then the Gaussian Unscented Transform. Why is something like 3DGUT so valuable for simulation?

Nick: Great question. With 3D Gaussian Ray Tracing, we could support more camera models, like fisheye lenses and rolling shutters—critical for autonomous vehicles with diverse sensors. But the challenge was speed. It wasn’t fast enough to be directly applicable.

That’s where 3D Gaussian Unscented Transform came in. It made the system as fast, while retaining the accuracy of ray tracing. It opened up practical applications for robotics and simulation.

Michael: One of the critiques of early Gaussian Splatting was that it required pinhole cameras. Extensions like 3DGUT have opened the door to robotics. We also saw USDZ conversion added recently. Why is that so important for developers?

Nick: It’s key for making Gaussian Splatting truly usable in Omniverse. Now, you can train environments with your cellphone camera, record your living room, for example, and bring that into simulation.

Once you have it in USD, you can render it in Omniverse with NuRec and then layer applications on top—say, robotics with Isaac Sim. That means anyone can capture their environment and immediately test agents inside it.

On a personal level, I think of kids recording their room and then dropping an R2-D2 in it. It democratizes simulation in a powerful way.

Michael: And it’s Apache 2 licensed, right?

Nick: Yes, fully open and commercially usable. That flexibility is important. Developers can use it for robotics, but also for creative applications—games, experiences, even playful ideas like a “roller coaster tycoon” in your living room.

Michael: We’ve also seen companies like Voxel51 adopt NuRec. How are they using it?

Nick: Voxel51 provides data quality tools. Good data is essential for reconstruction—the better the input, the better the output. They’ve integrated NuRec to assess and improve input data fidelity, which makes reconstructions more robust. It’s a critical piece of the ecosystem.

Michael: Another feature I wanted to touch on is Fixer. What does that add?

Nick: Fixer addresses gaps in data. Say you trained from one lane of a road but now want to simulate driving two lanes over. The original data may not cover that.

Fixer leverages Cosmos,a model trained on millions of scenes, to fill those gaps. It transfers prior knowledge into new reconstructions, enabling novel views and higher fidelity where data was missing.

Michael: NuRec also includes nRender. Can you explain its role?

Nick: nRender is a highly optimized rendering library within NuRec. If you want to do reinforcement learning, you need millions of scene variations—and that requires extremely fast rendering. nRender accelerates Gaussian Splatting to make that feasible.

Michael: Beyond autonomous vehicles, where else do you see robotics benefiting from these representations?

Nick: Honestly, I think everything that moves should eventually be autonomous. To train those agents, you need accurate simulations of the real world across industries. Radiance field methods bring that possibility closer.

Michael: Standardization has been a big topic. With so much rapid research, when is the right time to standardize Gaussian Splatting?

Nick: That’s a tough one. Every year, new papers push the field forward—MLPs moving Gaussians in space or time, for example. That complexity makes it hard to lock down a standard too soon.

But it’s important. Industry adoption is already happening. A standard must be flexible and extensible, yet not so loose that it fails as a standard. Finding that balance will be a challenge, but it’s necessary.

Michael: I also wanted to touch on CARLA. How is NuRec being used there?

Nick: We’ve reconstructed over 80 scenarios now and contributed them to the CARLA community. Researchers can use those reconstructions to drive different routes, create novel views, and expand autonomous driving research. It’s about making roads safer by bringing real-world complexity into simulation.

Michael: That’s exciting. And NuRec also includes the Asset Harvester, right?

Nick: Yes. Asset Harvester allows you to extract assets from real-world data and build libraries. It includes a generative component that can infer unseen views, enabling novel scene composition. That’s where radiance fields combine with generative models to open entirely new workflows.

Michael: Incredible. Nick, thank you so much for joining me today. I really appreciate the conversation.

Nick: Thanks for having me.

Thumbnail for Gaussian Splatting News episode 13

13:12

Watch · New every Monday

The Biggest Week Yet for Gaussian Splatting: Blender + Unreal

Episode 13 · Nov 16, 2026

Blender 5.3 is out, and Gaussian splats are now a native part of it: PLY and SPZ files import as their own point cloud type and render in EEVEE, Cycles and Workbench, with no add-on.

Featured

Recents

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Platforms

ReactVision Advanced VPS Turns Phone Scans Into Splats

ReactVision Advanced VPS in ViroReact 3.1.0 turns a one-minute phone scan into a map, mesh, GLB model, object map and Gaussian splat in one coordinate frame.

Michael Rubloff

Nov 16, 2026

Platforms

ReactVision Advanced VPS Turns Phone Scans Into Splats

ReactVision Advanced VPS in ViroReact 3.1.0 turns a one-minute phone scan into a map, mesh, GLB model, object map and Gaussian splat in one coordinate frame.

Michael Rubloff

Trending Articles
  1. TRENDING
    Loading...
  2. TRENDING
    Loading...
  3. TRENDING
    Loading...
Michael Rubloff

Written by Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

More from Michael Rubloff

More from Michael Rubloff

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

CARLA

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

Michael Rubloff

Nov 17, 2026

CARLA

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

CARLA

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

New to Gaussian Splatting? Start here

Radiance fields, explained from the ground up: what a Gaussian splat is, how to capture one, and which tools and gear to start with.

Newsletter

Keep up with the field.

New research, tools and captures from the Radiance Fields newsroom, in your inbox. Free, no spam.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.