Research

VastGaussian: Setting New Standards for Large-Scale 3D

Michael Rubloff

Feb 28, 2024

VastGaussian
VastGaussian

The newly announced VastGaussian project introduces a new approach to high-quality reconstruction and real-time rendering of large scenes, showcasing some of the most extensive radiance fields to date.

Developed by a collaborative team from Tsinghua University, Huawei Noah’s Ark Lab, and the Chinese Academy of Sciences, VastGaussian significantly enhances visual quality and rendering speed for large-scale scene reconstructions, presenting a novel technique rooted in 3D Gaussian Splatting (3DGS).

VastGaussian introduces an approach to managing large-scale scenes by dividing them into smaller, manageable cells. This division is not arbitrary; it employs a progressive partitioning strategy that ensures each cell contains an optimal amount of data for processing. This method allows for the parallel optimization of each cell, significantly reducing the overall computational burden and memory requirements. After optimization, these cells are seamlessly merged to form a cohesive and detailed large scene. This strategy addresses the critical challenge of video memory limitations and long optimization times that plague existing methods.

A standout feature of VastGaussian is its decoupled appearance modeling technique. This method acknowledges that appearance variations due to lighting conditions, camera settings, and other environmental factors can significantly impact the quality of the rendered images. By decoupling appearance modeling from the main reconstruction process, VastGaussian can independently optimize for these variations, leading to consistent and realistic renderings across different views. This approach significantly enhances the visual fidelity of the reconstructed scenes, ensuring high-quality outputs that are free from the common artifacts seen in other methods.

My favorite part of this method is how seamless the merge is between cells, to create these large scenes. The process for merging optimized cells is both straightforward and effective. Excess Gaussians outside each cell are removed, and the remaining Gaussians are combined without overlapping, ensuring a seamless and accurate large scene reconstruction. It seems simple in theory, but it's impressive how well they've executed the boundaries, so that there are no gaps or locations that seem out of place.

The project's results are further exciting, with VastGaussian outperforming existing state-of-the-art methods in terms of both quality and efficiency. With large scenes and high fidelity, you're probably thinking this must take a long time to train, be exceptionally computationally heavy, or both. Well to be honest, it's kind of neither. It takes roughly three hours to train VastGaussian and in the paper examples, they do not exceed 12GB. This is a notable use of efficiency, given the size of the capture. Amazingly the examples were all trained on a single 3090, opening the door including for broader use, including by those with consumer-grade hardware. Along with this, you also are getting the 100+ fps from Gaussian Splatting.

With these larger capture zones, it's not difficult to imagine a plethora of exciting use cases. The most logical of which is the AEC (Architecture, Engineering, and Construction) industry. With hyper realistic large scale representations, VastGaussian can revolutionize the pre-construction phase by providing architects and engineers with highly accurate and detailed 3D reconstructions of proposed construction sites. This can aid in better planning, site analysis, and integration of new structures within existing landscapes, ensuring that potential issues are identified and addressed early in the design process.

Additionally, the ability to create photorealistic renderings of future projects can significantly enhance stakeholder engagement. Clients and investors can embark on virtual walkthroughs of proposed designs, offering a clear vision of the project outcome. This immersive experience can facilitate better decision-making and increase stakeholder confidence in the project.

For existing structures, VastGaussian can be used to create detailed 3D models that assist in facility management and planning of renovations. These models can help in understanding the spatial dynamics of a building, planning for space utilization, and conducting energy efficiency analyses.

The construction industry can further leverage VastGaussian's realistic renderings to create safety and training simulations for workers. By simulating construction environments, workers can be trained on safety protocols and procedures in a controlled, virtual space, potentially reducing accidents and improving overall safety on construction sites.

The project's success is a testament to the collaborative effort of the team, highlighting the potential of combining advanced partitioning strategies and appearance modeling techniques in the field of 3D scene reconstruction. As VastGaussian continues to evolve, it promises to set new benchmarks for the quality and speed of large-scale scene reconstructions, opening up new possibilities for realistic and immersive digital environments.

VastGaussian was accepted into CVPR 2024, so it might be a little bit before we're seeing a code release, but this is just the latest indicator of the potential power of radiance fields. For more information and to explore the VastGaussian project further, visit the project page at VastGaussian.

Featured

Featured

Featured

News

An Update to Radiancefields.com

You may have noticed a small change to the website over the weekend. Well, not so small actually.

Michael Rubloff

Apr 16, 2024

News

An Update to Radiancefields.com

You may have noticed a small change to the website over the weekend. Well, not so small actually.

Michael Rubloff

Apr 16, 2024

News

An Update to Radiancefields.com

You may have noticed a small change to the website over the weekend. Well, not so small actually.

Michael Rubloff

Platforms

OpenSplat adds Mac GPU Acceleration

OpenSplat, which brought Mac training to 3DGS has received a big update, now allowing users to train with MPS backend with GPU acceleration.

Michael Rubloff

Apr 15, 2024

Platforms

OpenSplat adds Mac GPU Acceleration

OpenSplat, which brought Mac training to 3DGS has received a big update, now allowing users to train with MPS backend with GPU acceleration.

Michael Rubloff

Apr 15, 2024

Platforms

OpenSplat adds Mac GPU Acceleration

OpenSplat, which brought Mac training to 3DGS has received a big update, now allowing users to train with MPS backend with GPU acceleration.

Michael Rubloff

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Trending articles

Trending articles

Trending articles

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

Live NeRF Video Calls

Catching up with my sister has been an exercise in bridging distances. She recently moved to Copenhagen, trading the familiar landscapes of our shared childhood for the charming streets of the Danish capital.

Michael Rubloff

Oct 5, 2023

Research

Live NeRF Video Calls

Catching up with my sister has been an exercise in bridging distances. She recently moved to Copenhagen, trading the familiar landscapes of our shared childhood for the charming streets of the Danish capital.

Michael Rubloff

Oct 5, 2023

Research

Live NeRF Video Calls

Catching up with my sister has been an exercise in bridging distances. She recently moved to Copenhagen, trading the familiar landscapes of our shared childhood for the charming streets of the Danish capital.

Michael Rubloff

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Lightfield and hologram capture started with a big theoretical idea 115 years ago and we have struggled to make them viable ever since. Neural Radiance fields aka NeRF along with gaming computers now for the first time provide a promising easy and low cost way for everybody to capture and display lightfields.

Katrin Schmid

Mar 2, 2023

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Lightfield and hologram capture started with a big theoretical idea 115 years ago and we have struggled to make them viable ever since. Neural Radiance fields aka NeRF along with gaming computers now for the first time provide a promising easy and low cost way for everybody to capture and display lightfields.

Katrin Schmid

Mar 2, 2023

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Lightfield and hologram capture started with a big theoretical idea 115 years ago and we have struggled to make them viable ever since. Neural Radiance fields aka NeRF along with gaming computers now for the first time provide a promising easy and low cost way for everybody to capture and display lightfields.

Katrin Schmid

Featured

Featured

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV

Research

Live NeRF Video Calls

Catching up with my sister has been an exercise in bridging distances. She recently moved to Copenhagen, trading the familiar landscapes of our shared childhood for the charming streets of the Danish capital.

Michael Rubloff

Oct 5, 2023

Radiance Field Video Call

Research

Live NeRF Video Calls

Catching up with my sister has been an exercise in bridging distances. She recently moved to Copenhagen, trading the familiar landscapes of our shared childhood for the charming streets of the Danish capital.

Michael Rubloff

Oct 5, 2023

Radiance Field Video Call

Research

Live NeRF Video Calls

Michael Rubloff

Oct 5, 2023

Radiance Field Video Call

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Lightfield and hologram capture started with a big theoretical idea 115 years ago and we have struggled to make them viable ever since. Neural Radiance fields aka NeRF along with gaming computers now for the first time provide a promising easy and low cost way for everybody to capture and display lightfields.

Katrin Schmid

Mar 2, 2023

History of Neural Radiance Fields

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Lightfield and hologram capture started with a big theoretical idea 115 years ago and we have struggled to make them viable ever since. Neural Radiance fields aka NeRF along with gaming computers now for the first time provide a promising easy and low cost way for everybody to capture and display lightfields.

Katrin Schmid

Mar 2, 2023

History of Neural Radiance Fields

Guest Article

A short 170 year history of Neural Radiance Fields (NeRF), Holograms, and Light Fields

Katrin Schmid

Mar 2, 2023

History of Neural Radiance Fields

Recent articles

Recent articles