Nerfstudio Integrates 3DGUT into gsplat

It’s a very exciting day in the radiance fields world.

Ruilong Li, lead author of gsplat 1.0 and core maintainer of Nerfstudio has delivered once again with a native implementation of 3DGUT (3D Gaussian Unscented Transform) directly inside gsplat. 

For those unfamiliar, 3DGUT addresses two major limitations of traditional Gaussian Splatting:

• Camera Assumptions: Gaussian Splatting assumes a perfect pinhole camera model, which limits its applicability in domains like robotics, where fisheye or non-standard camera setups are common.

• Lighting Effects: Rasterization-based splatting can’t capture complex secondary lighting phenomena such as reflections and refractions—scenarios where ray tracing typically shines.

3DGUT solves both issues while preserving the performance advantages of rasterization. Incredibly, you still get well above real-time framerates, and the output remains a standard .ply file. This should result in clean compatibility with the burgeoning ecosystem.

3DGUT takes the same COLMAP or other SfM as the other radiance field representations. In order to take advantage of the new addition, you must pip install e . for the latest version of gsplat. 

NVIDIA has also just published a blog post highlighting the inclusion of 3DGUT—part of a broader recognition of radiance field methods by the Santa Clara company. Additionally, they have also creted a 3DGUT contest, giving away TWO 4090s!

At GTC 2025, NVIDIA announced the open-sourcing of both 3D Gaussian Unscented Transform (3DGUT) and 3D Gaussian Ray Tracing (3DGRT).

I had the chance to sit down with Sanja Fidler, VP of AI Research at NVIDIA, to talk through how 3DGUT works and why it’s such a powerful upgrade to the radiance field toolbox.

For people on Linux or wanting to run 3D Gaussian Ray Tracing, the original repository can be found here. I believe that it will take a bit longer to see 3DGRT inside of Nerfstudio. 

Both gsplat and Nerfstudio remain open source under the Apache 2.0 license. If you’re excited about this work and want to get involved, consider contributing to the repos.