Research

Shrinking 3DGS File Size

Michael Rubloff

Michael Rubloff

Apr 11, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
3dgs compress
3dgs compress

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

The original authors are back and are addressing some of those criticisms, which they have distilled into three observations: the number of 3D Gaussian primitives used to represent a scene, the number of coefficients for the spherical harmonics, and the precision required to store Gaussian primitive attributes. This is by no means the only project that has tackled the memory issue since the release of 3DGS, but it does provide a great view into what makes up the memory footprint behind Gaussian Splatting.

The first observation was that the original 3DGS method generated an excessive number of 3D Gaussian primitives, many of which were redundant, leading to an unnecessary increase in memory usage. This redundancy was particularly prevalent in areas of the scene where detailed reconstruction was not required, yet a dense cluster of primitives existed.

To address this, the team developed a resolution-aware pruning method that dynamically adjusts the number of primitives based on their necessity for accurate image representation. They introduced a two-step algorithm that first calculated a redundancy score for each primitive based on its contribution to the rendered view and then culled primitives that fell below a certain utility threshold. This approach effectively reduced the total count of primitives by about 60%, significantly decreasing the memory footprint. This might sound a little familiar conceptually to how RadSplat evaluates their Gaussians.

Interestingly, this comes right on the heels of a paper that revisits the densification stage of Gaussian Splatting. I'd be curious to see how the two papers could be applied to one another.

The second observation related to the use of SH bands, which are employed to model directional radiance. The original method used three SH bands for all primitives, regardless of whether the scene's visual complexity required that level of detail. This one-size-fits-all approach led to an over allocation of memory resources.

The researchers tackled this by implementing an adaptive adjustment method for SH bands during the training phase. By analyzing the contribution of each SH band to the visual appearance of the primitives and assessing multi-view consistency, they were able to reduce the number of SH bands dynamically. This revised new method ensures that only essential data is retained, thereby reducing the memory required for storing these elements.

The final observation was that many of the attributes associated with the primitives, such as opacity and color, did not require high dynamic range or precision, yet they were stored with high precision which led to wasteful memory utilization.

To optimize memory usage for these attributes, they introduce a codebook-based quantization approach. This involved clustering similar values and replacing them with indices to a shared codebook, thus storing only the index rather than the full precision data. Additionally, they applied half-float quantization to further compress the data size without significantly affecting the visual output. This approach drastically reduced the memory requirements for storing primitive attributes.

These updates resulted in reducing the file size by 27 times and 1.7X speed boost for rendering.

They have created a viewer that still retains spherical harmonics to include view dependent effects. They have two different modes: a light one and a heavy. The light version runs smoothly on moderate devices like an M2 laptop, while the heavy version requires more powerful GPU support. Check out their Project Page here!

Featured

Featured

Featured

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

To embed a website or widget, add it to the properties panel.

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV