Revolutionizing 3D Segmentation with SA3D and NeRFs: Unlocking New Possibilities in Real-world Applications

Imagine a world where 3D objects could be seamlessly segmented and manipulated, regardless of their complexity or composition.

Imagine a world where 3D objects could be seamlessly segmented and manipulated, regardless of their complexity or composition.

Michael Rubloff

Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

SA3D NeRF

Imagine a world where 3D objects could be seamlessly segmented and manipulated, regardless of their complexity or composition. Thanks to a group of researchers, this is now a reality with the development of an innovative framework called SA3D. By combining the Segment Anything Model (SAM) from Meta with Neural Radiance Fields (NeRFs), this groundbreaking technology unlocks endless possibilities for a wide range of real-world applications, revolutionizing industries like product photography, robotics, virtual reality, 3D printing, and entertainment.

NeRFs create a continuous function that maps 3D coordinates and viewing directions to colors and densities. This allows for generating high-quality 3D images from sparse 2D images, making NeRFs an ideal tool for creating realistic 3D representations of objects and scenes.

The powerful SAM model, capable of segmenting any object in 2D images, was limited to just that until researchers behind SA3D bridged the gap between 2D and 3D segmentation by leveraging NeRFs. This innovative combination made SAM even more powerful and applicable to a broader range of scenarios.

SA3D works by allowing users to obtain 3D segmentation results for any target object with just one-shot manual prompting in a single rendered view. The 2D segmentation mask generated by SAM is projected onto 3D mask grids using density-guided inverse rendering. The framework then employs a cross-view self-prompting technique, where 2D masks from other views are used as prompts to improve the segmentation results iteratively.

This groundbreaking process results in accurate 3D segmentation, without the need to redesign or retrain the NeRF model.

The potential applications of this cutting-edge technology are vast, as it can help in various fields such as product photography, virtual and augmented reality, 3D printing, and entertainment. Robotics engineers can use SA3D to create accurate 3D maps of environments, enabling robots to navigate and interact with objects more effectively. In virtual and augmented reality, SA3D could empower developers to create immersive experiences by generating detailed 3D models of real-world objects

Moreover, 3D printing can benefit from SA3D by producing accurate and detailed 3D models, streamlining the design process and reducing manual adjustments. In the entertainment industry, SA3D can be employed to create realistic 3D characters and environments for movies and video games, elevating the overall quality and visual appeal of these productions.

I'm most excited for its application with product NeRFs. As we recently saw with Google, SA3D can help revolutionize how product NeRFs can be created and then isolated efficiently for ecommerce.

In what's seemingly becoming increasing more standard:

The entire segmentation process can be completed in approximately 2 minutes without any engineering optimization.

The entire segmentation process in SA3D takes only about two minutes, making it a highly efficient tool for various applications. Researchers have demonstrated the effectiveness of SA3D in different scenes and conditions, showcasing the potential of SAM in 3D scene perception.

For those that were curious, SA3D also works with multi object segmentation

The code is available right now from Github for anyone that would like to try it out.

I am very curious how SA3D could function with other NeRF papers such as NeRFShop and LERF in the future. Quietly, NeRF editing capabilities have sprung up and SA3D is an exciting development that bridges the gap between 2D and 3D segmentation by harnessing the power of NeRFs. The vast potential for real-world applications is immense and I'm excited to see it be adapted!

Thumbnail for Gaussian Splatting News episode 13

13:12

Watch · New every Monday

The Biggest Week Yet for Gaussian Splatting: Blender + Unreal

Episode 13 · Nov 16, 2026

Blender 5.3 is out, and Gaussian splats are now a native part of it: PLY and SPZ files import as their own point cloud type and render in EEVEE, Cycles and Workbench, with no add-on.

Featured

Recents

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Platforms

ReactVision Advanced VPS Turns Phone Scans Into Splats

ReactVision Advanced VPS in ViroReact 3.1.0 turns a one-minute phone scan into a map, mesh, GLB model, object map and Gaussian splat in one coordinate frame.

Michael Rubloff

Nov 16, 2026

Platforms

ReactVision Advanced VPS Turns Phone Scans Into Splats

ReactVision Advanced VPS in ViroReact 3.1.0 turns a one-minute phone scan into a map, mesh, GLB model, object map and Gaussian splat in one coordinate frame.

Michael Rubloff

Trending Articles
  1. TRENDING
    Loading...
  2. TRENDING
    Loading...
  3. TRENDING
    Loading...
Michael Rubloff

Written by Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

More from Michael Rubloff

More from Michael Rubloff

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

SuperSplat 3.9 Imports GSAF 4D Splats as Timeline Sequences

SuperSplat v3.9.0 imports GSAF 4D Gaussian splat files as timeline sequences, plays several sequences at once and adds a review table for multi-file imports.

Michael Rubloff

Nov 17, 2026

The SuperSplat editor in a browser window, editing a Gaussian splat of a snow-covered cabin in a pine forest

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

CARLA

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

Michael Rubloff

Nov 17, 2026

CARLA

Platforms

CARLA 1.0 Adds NVIDIA NuRec 26.04 Support

CARLA 1.0 adds NVIDIA NuRec 26.04 support, turning recorded real-world drives into simulated scenes with neural lidar, asset swapping and actor compositing.

Michael Rubloff

Nov 17, 2026

CARLA

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

News

Volumetric Media Town Hall Turns to 4D Gaussian Splats

The Metaverse Standards Forum holds its third Volumetric Media Town Hall on November 24, with 4D Gaussian splatting talks by Dymensium, Chronospace and Gracia.

Michael Rubloff

Nov 17, 2026

Metaverse Standards Forum

New to Gaussian Splatting? Start here

Radiance fields, explained from the ground up: what a Gaussian splat is, how to capture one, and which tools and gear to start with.

Newsletter

Keep up with the field.

New research, tools and captures from the Radiance Fields newsroom, in your inbox. Free, no spam.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.