Platforms

Nerfstudio Integrates 3DGS MCMC

Michael Rubloff

Michael Rubloff

Jul 3, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
3DGS MCMC
3DGS MCMC

In its latest merge, Nerfstudio has incorporated an exciting update to its Gaussian Splatting method. It now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

The Nerfstudio team are currently refactoring the codebase to have splatfacto flag with MCMC, but for now people can train with it using the command:

 simple_trainer_mcmc.py

MCMC reinterprets the placement and adjustment of 3D Gaussians as a process of sampling from a probability distribution. This perspective aligns the updates of 3D Gaussians with the principles of Markov Chain Monte Carlo (MCMC) sampling. Specifically, they adopt Stochastic Gradient Langevin Dynamics (SGLD), a variant of MCMC, to introduce a noise term into the Gaussian updates. This reformulation naturally promotes the exploration of the scene and improves the effectiveness of the Gaussians in faithfully reproducing the scene.

Instead of using heuristics for cloning and splitting Gaussians, MCMC employs a relocalization scheme. This scheme moves 'dead' Gaussians (those with low opacity) to the locations of 'live' Gaussians (those with high opacity). The parameters of these relocated Gaussians are adjusted to minimally impact the rendering quality, ensuring that the probability distribution of the Gaussians remains stable.

To further optimize the system, the method introduces L1 regularization on the opacity and scale of Gaussians. This encourages unused Gaussians to disappear, thus managing the number of Gaussians effectively and reducing computational overhead.

Some users have noticed a strange haze effect on a percentage of captures, but they are currently investigating a fix. Nerfstudio and gsplat can be installed from their Github page

Featured

Featured

Featured

Platforms

splatviz: Interactive 3DGS Viewer and Python Editor

splatviz, an innovative interactive viewer available on GitHub offers real-time editing and visualization capabilities using a native Python GUI library.

Michael Rubloff

Jul 17, 2024

Platforms

splatviz: Interactive 3DGS Viewer and Python Editor

splatviz, an innovative interactive viewer available on GitHub offers real-time editing and visualization capabilities using a native Python GUI library.

Michael Rubloff

Jul 17, 2024

Platforms

splatviz: Interactive 3DGS Viewer and Python Editor

splatviz, an innovative interactive viewer available on GitHub offers real-time editing and visualization capabilities using a native Python GUI library.

Michael Rubloff

News

Amazon Prime Day Sales

Amazon Prime Day is here and with it are some deals that will help you train more NeRFs/3D Gaussian Splatting captures!

Michael Rubloff

Jul 16, 2024

News

Amazon Prime Day Sales

Amazon Prime Day is here and with it are some deals that will help you train more NeRFs/3D Gaussian Splatting captures!

Michael Rubloff

Jul 16, 2024

News

Amazon Prime Day Sales

Amazon Prime Day is here and with it are some deals that will help you train more NeRFs/3D Gaussian Splatting captures!

Michael Rubloff

Platforms

Three.js-based implemetation, GaussianSplats3D adds 2DGS Support

The Three.js-based implementation of Gaussian Splatting has officially added 2DGS support in v0.4.3.

Michael Rubloff

Jul 16, 2024

Platforms

Three.js-based implemetation, GaussianSplats3D adds 2DGS Support

The Three.js-based implementation of Gaussian Splatting has officially added 2DGS support in v0.4.3.

Michael Rubloff

Jul 16, 2024

Platforms

Three.js-based implemetation, GaussianSplats3D adds 2DGS Support

The Three.js-based implementation of Gaussian Splatting has officially added 2DGS support in v0.4.3.

Michael Rubloff

Research

3D Gaussian Ray Tracing

Today, things are taking an exciting step forward with the introduction of 3D Gaussian Ray Tracing (3DGRT).

Michael Rubloff

Jul 10, 2024

Research

3D Gaussian Ray Tracing

Today, things are taking an exciting step forward with the introduction of 3D Gaussian Ray Tracing (3DGRT).

Michael Rubloff

Jul 10, 2024

Research

3D Gaussian Ray Tracing

Today, things are taking an exciting step forward with the introduction of 3D Gaussian Ray Tracing (3DGRT).

Michael Rubloff