Research

Pop Goes the 3DGS Flicker: Stop the Pop

Michael Rubloff

Michael Rubloff

May 28, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Stop the Pop
Stop the Pop

If you've interacted with Gaussian Splatting, you might have noticed an issue described as "popping," where details appear to flicker as you move through a scene. This can be quite annoying, and a new method called Stop the Pop promises to eliminate this flickering and enhance visual fidelity.

The Stop the Pop method begins with the calculation of an optimal depth value for each Gaussian along every view ray. This depth is determined as the point of maximum contribution of the Gaussian along the ray, ensuring that the most significant elements are rendered first, thereby minimizing visual inconsistencies.

To efficiently manage the vast number of Gaussians typically involved in rendering complex scenes, Stop the Pop employs a hierarchical sorting and culling strategy. This approach divides the rendering process into multiple levels, each handling progressively finer details.

The first level involves dividing the image into tiles, typically 16x16 pixels in size, similar to how 3DGS functions. For each tile, the method evaluates the maximum within the tile (in 2D) and then reconstructs the corresponding ray (from the camera to this specific point on the image plane) - subsequently, we compute the depth for this ray. This significantly reduces the number of Gaussians that need to be considered at finer levels.

During this culling process, the bounding rectangle for each Gaussian is examined by evaluating its 2D projection onto the image plane. This projection uses the largest eigenvalue of the 2D covariance matrix to establish a conservative bound. Gaussians that do not intersect with the tile are discarded.

Once the relevant Gaussians for each tile are identified, a representative depth value for each Gaussian is calculated within the tile. This depth is used to sort the Gaussians, ensuring that those closer to the camera are rendered first. 

Because they have the maximum within the tile and know that Gaussians with alpha < 1/255.f are not considered during rendering, they can compute the maximum alpha value within the tile and discard Gaussians under the threshold, which increases performance.

For 4x4 subtiles, they repeat the exact same procedure (compute the maximum in 2D, reconstruct the ray, compute depth, culling) - this drastically reduces the number of Gaussians to consider as a large number of Gaussians only cover a single pixel.

After the initial sorting at the tile level, the sort order is further refined by performing local resorting within smaller sub-tiles (e.g., 2x2 pixels). They take the center ray of the tile, as computing the maximum is costly, and can't amortize it with culling for such small tiles.

This step ensures that the depth order is accurate down to the individual pixel level, effectively eliminating popping artifacts.

The hierarchical approach necessitates an optimized rendering pipeline that balances computational efficiency with high accuracy. To handle the variable workload associated with different Gaussians, a load balancing scheme is used. Gaussians that cover a large portion of the screen are processed cooperatively by multiple threads, distributing the computational load and minimizing bottlenecks.

Memory access patterns are also optimized to maximize performance, ensuring that data is efficiently loaded and shared among threads. This involves preloading commonly used values and using shared memory to minimize global memory accesses.

StopThePop requires a change to the backward pass to bypass the requirement of storing per-ray sorted Gaussians: This is achieved by performing the backward pass equivalently to the forward pass, and computing gradients in addition to blending.

To quantify the results, the developers uses FLIP from NVIDIA. Despite all the updates made to 3DGS, Stop the Pop only results in a 4% increase in render times. This slight tradeoff is worthwhile for the increased visual fidelity. Additionally, Stop the Pop works well on consumer-grade GPUs, with tests completed on an NVIDIA RTX 4090.

Stop the Pop’s code is available on GitHub, and while the original Gaussian Splatting license remains for the 3DGS portion, Stop the Pop’s code is MIT Licensed, making it permissibly licensed for people to expand upon. Additionally, they’ve also provided a real time viewer through SIBR, just like traditional 3DGS. The team behind Stop the Pop will also be giving a talk at SIGGRAPH in a couple of months.

With the recent release of Gracia's VR 1.0, it would be exciting to see Stop the Pop implemented into the pipeline, making captures look even more lifelike! Hopefully we will start to see the method be adapted into other platforms, such as Nerfstudio’s Splatfacto and Polycam. 

Featured

Featured

Featured

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Featured

Featured

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio