Research

Gaussian Surfels

Michael Rubloff

Michael Rubloff

Apr 30, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Gaussian Surfels
Gaussian Surfels

As SIGGRAPH 2024 approaches, some of the early papers have started to surface. Among these, the Gaussian Surfels paper, announced last week, is gaining attention. This paper isn't vastly different from the 2DGS paper, also slated for SIGGRAPH, but it brings unique elements worth discussing.

Surfels have been around for a long time, but what's new here is a way to combine the strengths of 3DGS and the precision of surfels for surface alignment.

The Gaussian Surfels technique starts by taking a set of posed RGB images of an object. The primary goal is to reconstruct the object's surface with high fidelity by optimizing a set of parameters that define the Gaussian surfels. These parameters include the position, orientation, scale, and color properties of each surfel.

Each surfel is defined as a Gaussian kernel, flattened into a 2D ellipse. This is achieved by setting the z-scale of its scale matrix to zero, simplifying the three-dimensional Gaussian distribution into a more manageable two-dimensional ellipse. This adjustment allows for precise alignment with the surface details of the object.

A major part of the method is the optimization process, where the surfels are adjusted to minimize the photometric differences between the rendered images and the actual images. This optimization is guided by several loss functions:

- Photometric Loss: It measures the difference in pixel values between the rendered and actual images, pushing the surfels towards a configuration that visually matches the captured images.

- Normal-Depth Consistency Loss: This loss ensures that the surfels accurately replicate not only the appearance but also the geometric structure of the object’s surface. It promotes consistency between the depth and normal information that the surfels render.

- Additional Losses: Other losses, including those for opacity, help fine-tune the surfels to achieve a clear and detailed rendering of the object’s surface.

Once optimized, the surfels are used to render depth maps and normal maps from multiple views. These maps are then integrated through a screened Poisson reconstruction process to create a high-quality surface mesh. A technique called volumetric cutting refines this process by removing erroneous depth values at the boundaries, which significantly enhances the depth map quality.

The entire process is able to be run on consumer grade GPUs, but there is no code yet. If you want to get started right away, Gaustudio has already implemented the adjacent paper, 2DGS into their pipeline. They also added pieces of Gaussian Surfels into their method, by including a monocular prior.

Despite the buzz on social media predicting significant influences on Gaussian Splatting technologies, it may be premature to draw any definitive conclusions.

All of this said, there does appear to be great promise for utilizing surfels and it will be interesting to watch how this unfolds. I believe there is still a lot more to be done, prior to any large declarations to be made.

Featured

Featured

Featured

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV