Gaussian Surfels

As SIGGRAPH 2024 approaches, some of the early papers have started to surface. Among these, the Gaussian Surfels paper, announced last week, is gaining attention. This paper isn't vastly different from the 2DGS paper, also slated for SIGGRAPH, but it brings unique elements worth discussing.

Surfels have been around for a long time, but what's new here is a way to combine the strengths of 3DGS and the precision of surfels for surface alignment.

The Gaussian Surfels technique starts by taking a set of posed RGB images of an object. The primary goal is to reconstruct the object's surface with high fidelity by optimizing a set of parameters that define the Gaussian surfels. These parameters include the position, orientation, scale, and color properties of each surfel.

Each surfel is defined as a Gaussian kernel, flattened into a 2D ellipse. This is achieved by setting the z-scale of its scale matrix to zero, simplifying the three-dimensional Gaussian distribution into a more manageable two-dimensional ellipse. This adjustment allows for precise alignment with the surface details of the object.

A major part of the method is the optimization process, where the surfels are adjusted to minimize the photometric differences between the rendered images and the actual images. This optimization is guided by several loss functions:

- Photometric Loss: It measures the difference in pixel values between the rendered and actual images, pushing the surfels towards a configuration that visually matches the captured images.

- Normal-Depth Consistency Loss: This loss ensures that the surfels accurately replicate not only the appearance but also the geometric structure of the object’s surface. It promotes consistency between the depth and normal information that the surfels render.

- Additional Losses: Other losses, including those for opacity, help fine-tune the surfels to achieve a clear and detailed rendering of the object’s surface.

Once optimized, the surfels are used to render depth maps and normal maps from multiple views. These maps are then integrated through a screened Poisson reconstruction process to create a high-quality surface mesh. A technique called volumetric cutting refines this process by removing erroneous depth values at the boundaries, which significantly enhances the depth map quality.

The entire process is able to be run on consumer grade GPUs, but there is no code yet. If you want to get started right away, Gaustudio has already implemented the adjacent paper, 2DGS into their pipeline. They also added pieces of Gaussian Surfels into their method, by including a monocular prior.

Despite the buzz on social media predicting significant influences on Gaussian Splatting technologies, it may be premature to draw any definitive conclusions.

All of this said, there does appear to be great promise for utilizing surfels and it will be interesting to watch how this unfolds. I believe there is still a lot more to be done, prior to any large declarations to be made.