Grid-NeRF: Combining NeRF and Grid-Based Methods for Large Scale Reconstruction

Michael Rubloff

Michael Rubloff

Mar 28, 2023

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Grid-NeRF
Grid-NeRF

From Google to Preimage, there has been a lot of recent hype surrounding large scale NeRFs. Each have their own method and today another one has been introduced.

A recent research paper highlights Grid-NeRF, a breakthrough in the rendering of large urban scenes, a crucial step for real-world applications like virtual reality, video games, and urban planning simulations. The study, conducted by a team of researchers from The Chinese University of Hong Kong, Shanghai AI Laboratory, Max Planck Institute for Informatics, Zhejiang University, and Adobe Research.

They realized that combining NeRFs with a feature grid-based method allowed for cleaner outputs and to overcome the limitations of current state-of-the-art techniques when applied to large-scale scenes, resulting in high visual fidelity rendering. Prior, they found that NeRFs would generate blurry and overly smoothed outputs, while grid-based methods struggled with artifacts (floaters).

In the study's ablation experiments, the team verified the impact of different model configurations. For the grid branch, they showed that adopting a single resolution feature grid leads to inferior performance. The NeRF branch greatly helps feature grids in recovering more details when grid resolution is inadequate. On high-resolution grids, combining NeRF can raise PSNR (Peak Signal-to-Noise Ratio) by approximately roughly 9-14% on the scenes.

The paper also discusses the limitations of small renderers in translating grid features, producing inaccurate geometry and scene contents. However, given enough modeling capacity, NeRF picks up more details with the help of increasingly higher PE (Positional Encoding).

However, the model also inherits some limitations of NeRF-based methods, such as slow training and Another critical issue is dealing with a large amount of high-resolution images, as the current batch sampling of shuffled rays is highly ineffective without distributed training. From the example video, over 5,000 drone images were used to generate the 2.7km^2 ground area.

Grid-NeRF presents a groundbreaking framework that integrates the MLP-based NeRF with an explicitly constructed feature grid to effectively take the best parts of each. Grid-NeRF is able to demonstrate high visual fidelity rendering for large urban scenes. Although the model still faces some limitations, this innovative approach opens up new possibilities for real-world applications in urban planning and large digital twin creation. This paper is another step forwards to helping companies simulate autonomous driving networks without cars on the road, aerial surveying, urban planning, and other geological industries.

Featured

Recents

Featured

Platforms

Gracia Launches Dynamic Gaussian Splatting

Gracia launches dynamic captures of Radiance Fields, using Gaussian Splatting.

Michael Rubloff

Oct 17, 2024

Platforms

Gracia Launches Dynamic Gaussian Splatting

Gracia launches dynamic captures of Radiance Fields, using Gaussian Splatting.

Michael Rubloff

Oct 17, 2024

Platforms

Gracia Launches Dynamic Gaussian Splatting

Gracia launches dynamic captures of Radiance Fields, using Gaussian Splatting.

Michael Rubloff

Research

Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency

Rendering speeds are getting a big boost, with some pesky features addressed.

Michael Rubloff

Oct 16, 2024

Research

Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency

Rendering speeds are getting a big boost, with some pesky features addressed.

Michael Rubloff

Oct 16, 2024

Research

Efficient Perspective-Correct 3D Gaussian Splatting Using Hybrid Transparency

Rendering speeds are getting a big boost, with some pesky features addressed.

Michael Rubloff

Platforms

Postshot Announces Unreal Engine Beta

Postshot adds a UE5 plugin in its latest pre-release build.

Michael Rubloff

Oct 14, 2024

Platforms

Postshot Announces Unreal Engine Beta

Postshot adds a UE5 plugin in its latest pre-release build.

Michael Rubloff

Oct 14, 2024

Platforms

Postshot Announces Unreal Engine Beta

Postshot adds a UE5 plugin in its latest pre-release build.

Michael Rubloff

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff

Oct 9, 2024

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff

Oct 9, 2024

News

ITS Launches with Radiance Fields

The International Talent Support 2025 launch video arrives full of Radiance Fields.

Michael Rubloff