3D Convex Splatting Radiance Field Rendering

Michael Rubloff

Michael Rubloff

Dec 6, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
3D Convex Splatting
3D Convex Splatting

Over the last year, Gaussian Splatting has dominated the world of Radiance Field rendering. However, the past few months have introduced additional ways to represent and render a Radiance Field, such as Gaussian Ray Tracing and EVER, but today we’re looking at another departure from Gaussian based representations with 3D Convex Splatting.

3D Convex Splatting replaces Gaussian primitives with 3D smooth convexes— flexible shapes capable of capturing both hard edges and dense volumes. These primitives can represent geometrically meaningful elements of a scene with fewer shapes, reducing memory overhead while maintaining visual fidelity. Defined by a set of points forming a 3D convex hull, these shapes are governed by parameters that control their smoothness and sharpness. This flexibility allows convexes to approximate flat surfaces and angular edges without requiring additional primitives, as Gaussian methods often do.

To render a scene, 3DCS projects the points of each convex onto a 2D image plane using a differentiable pinhole camera model. This projection is critical, as it enables the system to compute gradients and optimize during training. Once projected, the convex hull of the points is calculated, defining the 2D boundary of the shape, which is then rendered using a custom CUDA-based rasterizer for real-time performance.

Starting with a sparse set of primitives, 3DCS adaptively densifies convexes in regions requiring higher detail. By splitting shapes into multiple smaller convexes, the method captures finer scene details without bloating memory requirements. While offering higher quality, 3DCS reduced memory usage by up to 30% compared to 3DGS. The tradeoff here is that the training time is a bit longer compared to 3DGS. Rendering frame rates also take a hit, but still hover above real time rates.

One of the most compelling benefits of 3DCS is its ability to produce geometrically meaningful decompositions of scenes. For example, where Gaussian primitives might produce a blurred representation of complex objects, convex primitives can delineate distinct surfaces and edges. This capability makes 3DCS particularly well-suited for structured environments, such as indoor scenes, where sharp boundaries and flat surfaces dominate. Convex primitives can approximate surfaces like walls or angular edges without resorting to additional primitives, as Gaussian methods would require.

Despite its strengths, 3DCS isn't without limitations. In outdoor scenes, its advantages over Gaussian-based methods become less pronounced. Future research could explore hybrid approaches, combining the strengths of convex and Gaussian primitives to handle diverse environments seamlessly.

3D Convex Splatting represents an exciting consideration in radiance field rendering. It is important to remember we have not yet arrived at the pinnacle of inverse rendering and 3D reconstruction.

For more details and implementation resources, visit the project page at convexsplatting.github.io. The code is also available here with an Apache 2.0 license.

Featured

Recents

Featured

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

Jan 11, 2025

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

Jan 11, 2025

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Jan 9, 2025

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Jan 9, 2025

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

Jan 9, 2025

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

Jan 9, 2025

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff

Jan 6, 2025

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff

Jan 6, 2025

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff