Nerfstudio Integrates 3DGS MCMC

Michael Rubloff

Michael Rubloff

Jul 3, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
3DGS MCMC
3DGS MCMC

In its latest merge, Nerfstudio has incorporated an exciting update to its Gaussian Splatting method. It now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

The Nerfstudio team are currently refactoring the codebase to have splatfacto flag with MCMC, but for now people can train with it using the command:

 simple_trainer_mcmc.py

MCMC reinterprets the placement and adjustment of 3D Gaussians as a process of sampling from a probability distribution. This perspective aligns the updates of 3D Gaussians with the principles of Markov Chain Monte Carlo (MCMC) sampling. Specifically, they adopt Stochastic Gradient Langevin Dynamics (SGLD), a variant of MCMC, to introduce a noise term into the Gaussian updates. This reformulation naturally promotes the exploration of the scene and improves the effectiveness of the Gaussians in faithfully reproducing the scene.

Instead of using heuristics for cloning and splitting Gaussians, MCMC employs a relocalization scheme. This scheme moves 'dead' Gaussians (those with low opacity) to the locations of 'live' Gaussians (those with high opacity). The parameters of these relocated Gaussians are adjusted to minimally impact the rendering quality, ensuring that the probability distribution of the Gaussians remains stable.

To further optimize the system, the method introduces L1 regularization on the opacity and scale of Gaussians. This encourages unused Gaussians to disappear, thus managing the number of Gaussians effectively and reducing computational overhead.

Some users have noticed a strange haze effect on a percentage of captures, but they are currently investigating a fix. Nerfstudio and gsplat can be installed from their Github page

Featured

Recents

Featured

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff

Oct 9, 2024

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff

Oct 9, 2024

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff

News

Radiance Fields win at IBC 2024

Back at IBC 2024, An overview of Radiance Fields wins best Technical Paper.

Michael Rubloff

Oct 8, 2024

News

Radiance Fields win at IBC 2024

Back at IBC 2024, An overview of Radiance Fields wins best Technical Paper.

Michael Rubloff

Oct 8, 2024

News

Radiance Fields win at IBC 2024

Back at IBC 2024, An overview of Radiance Fields wins best Technical Paper.

Michael Rubloff

Platforms

GSOPS 2.0 Announced

Two exciting Radiance Field projects are coming together in Houdini.

Michael Rubloff

Oct 7, 2024

Platforms

GSOPS 2.0 Announced

Two exciting Radiance Field projects are coming together in Houdini.

Michael Rubloff

Oct 7, 2024

Platforms

GSOPS 2.0 Announced

Two exciting Radiance Field projects are coming together in Houdini.

Michael Rubloff

Research

How EVER (Exact Volumetric Ellipsoid Rendering) Does This Work?

Another Ray Tracing Radiance Field emerges, this time from Google.

Michael Rubloff

Oct 3, 2024

Research

How EVER (Exact Volumetric Ellipsoid Rendering) Does This Work?

Another Ray Tracing Radiance Field emerges, this time from Google.

Michael Rubloff

Oct 3, 2024

Research

How EVER (Exact Volumetric Ellipsoid Rendering) Does This Work?

Another Ray Tracing Radiance Field emerges, this time from Google.

Michael Rubloff