Nerfstudio Integrates 3DGS MCMC

Michael Rubloff

Michael Rubloff

Jul 3, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
3DGS MCMC
3DGS MCMC

In its latest merge, Nerfstudio has incorporated an exciting update to its Gaussian Splatting method. It now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

The Nerfstudio team are currently refactoring the codebase to have splatfacto flag with MCMC, but for now people can train with it using the command:

 simple_trainer_mcmc.py

MCMC reinterprets the placement and adjustment of 3D Gaussians as a process of sampling from a probability distribution. This perspective aligns the updates of 3D Gaussians with the principles of Markov Chain Monte Carlo (MCMC) sampling. Specifically, they adopt Stochastic Gradient Langevin Dynamics (SGLD), a variant of MCMC, to introduce a noise term into the Gaussian updates. This reformulation naturally promotes the exploration of the scene and improves the effectiveness of the Gaussians in faithfully reproducing the scene.

Instead of using heuristics for cloning and splitting Gaussians, MCMC employs a relocalization scheme. This scheme moves 'dead' Gaussians (those with low opacity) to the locations of 'live' Gaussians (those with high opacity). The parameters of these relocated Gaussians are adjusted to minimally impact the rendering quality, ensuring that the probability distribution of the Gaussians remains stable.

To further optimize the system, the method introduces L1 regularization on the opacity and scale of Gaussians. This encourages unused Gaussians to disappear, thus managing the number of Gaussians effectively and reducing computational overhead.

Some users have noticed a strange haze effect on a percentage of captures, but they are currently investigating a fix. Nerfstudio and gsplat can be installed from their Github page

Featured

Recents

Featured

Platforms

GSOPs 2.0: Now Commercially Viable with Houdini Commercial License

The 2.0 release for GSOPs is here with a commercial license!

Michael Rubloff

Dec 20, 2024

Platforms

GSOPs 2.0: Now Commercially Viable with Houdini Commercial License

The 2.0 release for GSOPs is here with a commercial license!

Michael Rubloff

Dec 20, 2024

Platforms

GSOPs 2.0: Now Commercially Viable with Houdini Commercial License

The 2.0 release for GSOPs is here with a commercial license!

Michael Rubloff

Platforms

Odyssey Announces Generative World Model, Explorer

Odyssey shows off their photo real world generator, powered by Radiance Fields.

Michael Rubloff

Dec 18, 2024

Platforms

Odyssey Announces Generative World Model, Explorer

Odyssey shows off their photo real world generator, powered by Radiance Fields.

Michael Rubloff

Dec 18, 2024

Platforms

Odyssey Announces Generative World Model, Explorer

Odyssey shows off their photo real world generator, powered by Radiance Fields.

Michael Rubloff

Platforms

PICO Splat for Unreal Engine Plugin

The Unreal Engine plugin for Pico headsets has been released in beta.

Michael Rubloff

Dec 13, 2024

Platforms

PICO Splat for Unreal Engine Plugin

The Unreal Engine plugin for Pico headsets has been released in beta.

Michael Rubloff

Dec 13, 2024

Platforms

PICO Splat for Unreal Engine Plugin

The Unreal Engine plugin for Pico headsets has been released in beta.

Michael Rubloff

Research

HLOC + GLOMAP Repo

A GitHub repo from Pablo Vela has integrated GLOMAP with HLOC.

Michael Rubloff

Dec 10, 2024

Research

HLOC + GLOMAP Repo

A GitHub repo from Pablo Vela has integrated GLOMAP with HLOC.

Michael Rubloff

Dec 10, 2024

Research

HLOC + GLOMAP Repo

A GitHub repo from Pablo Vela has integrated GLOMAP with HLOC.

Michael Rubloff