Today's been a really busy day of news!
While there was the exciting news on the platform side, with Polycam implementing Gaussian Splatting, we have an equally exciting research paper, NeuRBF, that greatly increases the quality of NeRFs. We are beginning to see NeRFs hit the mid 30s for their PSNR score. NeurRBF has all different kinds of applications, but among them are neural radiance fields. Let's explore how they do this and what their net effects are.
NeuRBF introduces advancements in the representation of neural fields, reshaping (no pun intended) approaches to NeRFs and beyond. One of the groundbreaking innovations is the concept of spatial adaptivity, something that Gaussian Splatting also leverages, though it was not known until both papers were published. This adaptivity is achieved through the implementation of adaptive radial bases endowed with general anisotropic kernel functions, allowing the model to exhibit a higher level of spatial adaptivity and align more closely with target signals. In practical terms, spatial adaptivity means the model can more accurately map and reconstruct intricate 3D spaces and scenes, capturing high-frequency components and intricate details with more effectiveness, an advancement pivotal to applications in neural radiance fields where precision and detail are paramount.
However, NeuRBF doesn’t solely rely on adaptive components to enhance representation. It amalgamates these with grid-based radial bases utilizing N-dimensional linear kernel functions. These grid-based components are integral for maintaining interpolation smoothness within the model. This helps achieve coherent and smooth representations and avoiding distortions in the reconstructed fields. This amalgamation of adaptive and grid-based components results in a model that not only excels in adapting to represent complex patterns and details but also ensures smooth transitions and interpolations, balancing adaptivity and smoothness, and preventing abrupt changes in the representations.
In essence, the innovations brought forward by NeuRBF in spatial adaptivity and enhanced representation through multi-frequency sinusoidal composition are not just incremental advancements, but big steps forward in the field of neural representation. They push the boundaries of accuracy and detail representation in NeRFs, opening up new possibilities and setting the stage for more sophisticated and nuanced applications in 3D scene reconstruction and rendering.
It seems like a lot of the advancements are built around generating flexibility for what it's working on. In a lot of ways, that flexibility and additional options is exciting to me, because it offers the practicality of real world data sets. Additionally, while it's a smaller note, NeuRBF does not require Structure from Motion (SfM), such as COLMAP. Overall, by utilizing spatial adaptivity and frequency extension of radial basis functions, the outputs that we are getting from NeRFs and other neural fields are going to increase dramatically, while not increasing the file size.
NeuRBF's methodology is not confined to NeRFs but also extends possibilities to affect various realms such as signal compression, 3D reconstruction, neural rendering, medical imaging, acoustic synthesis, and even climate prediction, due to its ability to represent both 2D images and 3D shapes effectively.
The paper is being released as part of ICCV 2023, which takes place next week in Paris. The code is now live for those that want to try it out for themself.
