Memory-Efficient Radiance Fields (MERF) Announced by Google Research

Almost coincidentally, just hours after Luma AI announced in web browser realtime rendering, a new method of radiance fields has been published, Memory-Efficient Radiance Fields or MERF. The paper appears to be a collaboration between Google Research and the University of Tübingen AI Center in Germany, though on refresh of the page, the authors have been removed.

MERF reduces the memory consumption of prior sparse volumetric radiance fields using a combination of a sparse feature grid and high-resolution 2D feature planes. To support large-scale unbounded scenes, we introduce a novel contraction function that maps scene coordinates into a bounded volume while still allowing for efficient ray-box intersection.

Abstract

In other words, MERF aims to solve the problem of how much memory strain NeRF's place on a device and improves it to the point of enabling realtime rendering in a web browser. In the demo video, a 3090 GPU is able to display the MERF at 60fps without issue.

The team has published a collection of interactable demos here. Give them all a try; each one is astounding in its own right. Additionally, see below for a detail comparison of the different NeRF methods, including Instant-NGP.

Whether or not this technology is powering the new announcement by Luma AI or not, it represents a positive step in the direction of bringing down the barrier to entry for NeRFs. The rate at which NeRF advancements have been occurring are nothing short of stunning and seemingly what was thought to be a fanciful request becomes more and more real every day.