Paper

Wentao Sun

Hanqing Xu

Quanyun Wu

Dedong Zhang

Yiping Chen

Lingfei Ma

John S. Zelek

Jonathan Li

We introduce PointGauss, a novel point cloud-guided framework for real-time multi-object segmentation in Gaussian Splatting representations. Unlike existing methods that suffer from prolonged initialization and limited multi-view consistency, our approach achieves efficient 3D segmentation by directly parsing Gaussian primitives through a point cloud segmentation-driven pipeline. The key innovation lies in two aspects: (1) a point cloud-based Gaussian primitive decoder that generates 3D instance masks within 1 minute, and (2) a GPU-accelerated 2D mask rendering system that ensures multi-view consistency. Extensive experiments demonstrate significant improvements over previous state-of-the-art methods, achieving performance gains of 1.89 to 31.78% in multi-view mIoU, while maintaining superior computational efficiency. To address the limitations of current benchmarks (single-object focus, inconsistent 3D evaluation, small scale, and partial coverage), we present DesktopObjects-360, a novel comprehensive dataset for 3D segmentation in radiance fields, featuring: (1) complex multi-object scenes, (2) globally consistent 2D annotations, (3) large-scale training data (over 27 thousand 2D masks), (4) full 360{\deg} coverage, and (5) 3D evaluation masks.

PDF URL

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.