Research

Mip-Splatting: Anti-Aliasing for Gaussian Splatting

Michael Rubloff

Michael Rubloff

Nov 28, 2023

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Mip-Splatting
Mip-Splatting

If you've been trying out Gaussian Splatting, you may have noticed that the scene tends to degrade quickly, especially when you pull outside of the capture path or change the focal length, compared to NeRF. While it is a problem, like almost all problems, there are solutions and this solution is Mip-Splatting. The researchers evidently noticed this too and have attributed it to a lack of 3D frequency constraints and the usage of a 2D dilation filter. Mip-Splatting, with its innovative approach, was developed to overcome these limitations.

The story of Mip-Splatting begins with a critical challenge faced by 3DGS – the presence of high-frequency artifacts in zoomed-in images. Here enters the first part of Mip-Splatting, the 3D smoothing filter. This innovative filter, embarks on a mission to regulate the size of 3D Gaussian primitives. Its strategy is simple yet effective: applying a Gaussian low-pass filter to each primitive before its projection onto the screen space. This approach ensures that the primitives do not exceed the maximal frequency allowed by the sampling rate of the input views. In essence, the 3D smoothing filter becomes a guardian, preventing the creation of unfaithful high-frequency details that lead to visual artifacts when zooming in.

While the 3D smoothing filter tackles one aspect of the problem, another challenge lurks in the realm of scaling – the issue of aliasing and dilation artifacts. This is where the 2D Mip filter comes into play, complementing its counterpart in this journey of innovation. Replacing the 2D dilation filter used in 3DGS, the 2D Mip filter approximates the box filter used in physical imaging processes. This approximation is done through a 2D Gaussian filter, which, while computationally efficient, adeptly simulates the integration of photons over a pixel area on a camera sensor. The result is a more accurate and artifact-free rendering of images, particularly when zoomed out, thereby preserving the integrity and consistency of the visual output across different scales.

The true prowess of Mip-Splatting lies in the harmonious interplay between the 3D smoothing filter and the 2D Mip filter. These components do not function in isolation; instead, they perform a delicate dance of technological symbiosis. The 3D smoothing filter ensures fidelity to the input view constraints by regularizing the frequency content, while the 2D Mip filter takes charge of rendering this regulated representation accurately across various scales. This synergy allows Mip-Splatting to render images with unparalleled quality, free from the distortions and inconsistencies that plagued earlier methods.

Despite its advancements, Mip-Splatting is not without its challenges. The method's reliance on a Gaussian filter as an approximation to a box filter, while efficient, introduces errors, especially noticeable when the Gaussian is small in screen space. Additionally, the requirement to compute the sampling rate for each 3D Gaussian periodically adds a computational overhead, potentially limiting its efficiency.

Furthermore, while Mip-Splatting handles a range of scaling scenarios with aplomb, it may encounter difficulties in extreme zoom-out situations due to the inherent limitations of the Gaussian approximation.

Mip-Splatting marks a significant leap in the field, offering robust solutions. Its innovative approach in rendering artifact-free scenes across different scales unlocks the possibility for higher fidelity representations. While current limitations exist, they provide ground for future research and development. As the technology evolves, Mip-Splatting is poised to play an increasingly crucial role, potentially transforming a variety of applications in computer graphics and beyond.

The code for Mip-Splatting is available right now and comes with the same license information as the original 3DGS paper. This means if you would like to use this method for commercial purposes, you must license it properly through Inria. It can be used for for research and/or evaluation purposes only.

Featured

Featured

Featured

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV