Grid-NeRF: Combining NeRF and Grid-Based Methods for Large Scale Reconstruction

From Google to Preimage, there has been a lot of recent hype surrounding large scale NeRFs. Each have their own method and today another one has been introduced.

A recent research paper highlights Grid-NeRF, a breakthrough in the rendering of large urban scenes, a crucial step for real-world applications like virtual reality, video games, and urban planning simulations. The study, conducted by a team of researchers from The Chinese University of Hong Kong, Shanghai AI Laboratory, Max Planck Institute for Informatics, Zhejiang University, and Adobe Research.

They realized that combining NeRFs with a feature grid-based method allowed for cleaner outputs and to overcome the limitations of current state-of-the-art techniques when applied to large-scale scenes, resulting in high visual fidelity rendering. Prior, they found that NeRFs would generate blurry and overly smoothed outputs, while grid-based methods struggled with artifacts (floaters).

In the study's ablation experiments, the team verified the impact of different model configurations. For the grid branch, they showed that adopting a single resolution feature grid leads to inferior performance. The NeRF branch greatly helps feature grids in recovering more details when grid resolution is inadequate. On high-resolution grids, combining NeRF can raise PSNR (Peak Signal-to-Noise Ratio) by approximately roughly 9-14% on the scenes.

The paper also discusses the limitations of small renderers in translating grid features, producing inaccurate geometry and scene contents. However, given enough modeling capacity, NeRF picks up more details with the help of increasingly higher PE (Positional Encoding).

However, the model also inherits some limitations of NeRF-based methods, such as slow training and Another critical issue is dealing with a large amount of high-resolution images, as the current batch sampling of shuffled rays is highly ineffective without distributed training. From the example video, over 5,000 drone images were used to generate the 2.7km^2 ground area.

Grid-NeRF presents a groundbreaking framework that integrates the MLP-based NeRF with an explicitly constructed feature grid to effectively take the best parts of each. Grid-NeRF is able to demonstrate high visual fidelity rendering for large urban scenes. Although the model still faces some limitations, this innovative approach opens up new possibilities for real-world applications in urban planning and large digital twin creation. This paper is another step forwards to helping companies simulate autonomous driving networks without cars on the road, aerial surveying, urban planning, and other geological industries.