For the last year+, Instant NGP has been seen as the fastest local training NeRF method that would run on a consumer grade GPU. Yesterday, a paper unleashing MixNeRF was released, which trains faster and uses less parameters than Instant-NGP.
NeRFs have made significant advancements in quality and efficiency, but storing features in dense grids can lead to memory bottlenecks and lengthy training times.
One of the common issues with NeRF thus far has been the amount of memory or VRAM that it takes to run a NeRF. MixNeRF attempts to solve this problem by using a mixed up hash table that improves memory efficiency, while also achieving outstanding training time and maintains the quality of reconstruction.
This is primarily done in two steps. The first, as mentioned above is the inclusion of the mixed up hash table to map it into a single hash table. Inherently, that also decreases the number of encoding parameters. However, it may also lead to a decline in learning performance due to limited memory space, so it’s critical that the number of hash tables that multiple windows is optimized.
Afterwards, they design an index transformation to get the correct index of a grid point.
Researchers benchmarked MixNeRF against state-of-the-art methods such as Instant-NGP, TensoRF, and DVGO. Results showed that MixNeRF achieved the fastest training time on the same GPU hardware while maintaining similar or higher rendering quality. For example, compared to Instant-NGP, the MixNeRF-8(222) setup achieved around a 12% training time reduction on the same GPU with 0.2 higher average PSNR in the synthetic NeRF dataset.
With its memory-efficient approach, MixNeRF is poised to become the go-to framework for fast and high-quality 3D scene representation.
“Compared to Instant-NGP, we can see that MixNeRF-8 achieved a parameter reduction of 47.00%/47.44%/47.23%/45.94%, and training time reductions of 4s/38s/175s/290s, with a 0.09/0.2/0.24/0.18 PSNR improvement at all four different T settings, respectively.”
MixNeRF improves memory efficiency, reduce training and rendering time, while maintaining reconstruction quality.
It seems that the NeRF training times are only going to get faster and faster, while still keeping the same reconstruction quality. I’m excited to see how this all progresses and how our definition of “instant” evolves.
