Research

VastGaussian: Setting New Standards for Large-Scale 3D

Michael Rubloff

Michael Rubloff

Feb 28, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
VastGaussian
VastGaussian

The newly announced VastGaussian project introduces a new approach to high-quality reconstruction and real-time rendering of large scenes, showcasing some of the most extensive radiance fields to date.

Developed by a collaborative team from Tsinghua University, Huawei Noah’s Ark Lab, and the Chinese Academy of Sciences, VastGaussian significantly enhances visual quality and rendering speed for large-scale scene reconstructions, presenting a novel technique rooted in 3D Gaussian Splatting (3DGS).

VastGaussian introduces an approach to managing large-scale scenes by dividing them into smaller, manageable cells. This division is not arbitrary; it employs a progressive partitioning strategy that ensures each cell contains an optimal amount of data for processing. This method allows for the parallel optimization of each cell, significantly reducing the overall computational burden and memory requirements. After optimization, these cells are seamlessly merged to form a cohesive and detailed large scene. This strategy addresses the critical challenge of video memory limitations and long optimization times that plague existing methods.

A standout feature of VastGaussian is its decoupled appearance modeling technique. This method acknowledges that appearance variations due to lighting conditions, camera settings, and other environmental factors can significantly impact the quality of the rendered images. By decoupling appearance modeling from the main reconstruction process, VastGaussian can independently optimize for these variations, leading to consistent and realistic renderings across different views. This approach significantly enhances the visual fidelity of the reconstructed scenes, ensuring high-quality outputs that are free from the common artifacts seen in other methods.

My favorite part of this method is how seamless the merge is between cells, to create these large scenes. The process for merging optimized cells is both straightforward and effective. Excess Gaussians outside each cell are removed, and the remaining Gaussians are combined without overlapping, ensuring a seamless and accurate large scene reconstruction. It seems simple in theory, but it's impressive how well they've executed the boundaries, so that there are no gaps or locations that seem out of place.

The project's results are further exciting, with VastGaussian outperforming existing state-of-the-art methods in terms of both quality and efficiency. With large scenes and high fidelity, you're probably thinking this must take a long time to train, be exceptionally computationally heavy, or both. Well to be honest, it's kind of neither. It takes roughly three hours to train VastGaussian and in the paper examples, they do not exceed 12GB. This is a notable use of efficiency, given the size of the capture. Amazingly the examples were all trained on a single 3090, opening the door including for broader use, including by those with consumer-grade hardware. Along with this, you also are getting the 100+ fps from Gaussian Splatting.

With these larger capture zones, it's not difficult to imagine a plethora of exciting use cases. The most logical of which is the AEC (Architecture, Engineering, and Construction) industry. With hyper realistic large scale representations, VastGaussian can revolutionize the pre-construction phase by providing architects and engineers with highly accurate and detailed 3D reconstructions of proposed construction sites. This can aid in better planning, site analysis, and integration of new structures within existing landscapes, ensuring that potential issues are identified and addressed early in the design process.

Additionally, the ability to create photorealistic renderings of future projects can significantly enhance stakeholder engagement. Clients and investors can embark on virtual walkthroughs of proposed designs, offering a clear vision of the project outcome. This immersive experience can facilitate better decision-making and increase stakeholder confidence in the project.

For existing structures, VastGaussian can be used to create detailed 3D models that assist in facility management and planning of renovations. These models can help in understanding the spatial dynamics of a building, planning for space utilization, and conducting energy efficiency analyses.

The construction industry can further leverage VastGaussian's realistic renderings to create safety and training simulations for workers. By simulating construction environments, workers can be trained on safety protocols and procedures in a controlled, virtual space, potentially reducing accidents and improving overall safety on construction sites.

The project's success is a testament to the collaborative effort of the team, highlighting the potential of combining advanced partitioning strategies and appearance modeling techniques in the field of 3D scene reconstruction. As VastGaussian continues to evolve, it promises to set new benchmarks for the quality and speed of large-scale scene reconstructions, opening up new possibilities for realistic and immersive digital environments.

VastGaussian was accepted into CVPR 2024, so it might be a little bit before we're seeing a code release, but this is just the latest indicator of the potential power of radiance fields. For more information and to explore the VastGaussian project further, visit the project page at VastGaussian.

Featured

Featured

Featured

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

To embed a website or widget, add it to the properties panel.

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV