For the last couple of weeks, the internet has been abuzz with Gaussian Splatting and how NeRFs will be affected by them. Today Luma unveils the newest iteration of the seminal method, Luma Interactive Scenes.
I can confirm that these are in fact Gaussian Splats that are being run and it’s a proprietary iteration of the original Inria paper. They hybridize the performance gain of realtime rendering with Gaussian Splatting with robust cloud based rendering that’s already widely being used in commercial applications. This has been in the works for a while over at Luma and I had the opportunity to try out some of my datasets on their new method.
As a reminder, Luma allows for full commercial use of its platform.
The resulting files are crisper, with additional interactivity running on mobile. Luma also allows you to reprocess any of the datasets you’ve already taken, though there is a cap of 20 a month right now. I imagine that their servers are about to be slammed.
Prior to this, Luma would output a preview of a structured light field, which was used as a proxy for the full quality NeRF. Now we get a fully interactive Gaussian Splat.
3D was either pretty, or fast. Now it’s BOTH! Meet Interactive Scenes built with Gaussian Splatting:
🔥Browser & Phone-Friendly: Hyperefficient and fast rendering everywhere
👌Embed Anywhere: 8-20MB streaming files (even smaller soon!)
✨Ultra High Quality offline NeRF renders &… pic.twitter.com/nAbqr9jqOp — Luma AI (@LumaLabsAI) October 3, 2023
All datasets uploaded through the mobile app, web, and API are now being run through this pipeline automatically. They are the first to offer Gaussian Splatting through an API. The outputs are both viewable and interactive on mobile already, which is an impressive feat. The files continue to be able to be exported to Unreal Engine through the .Luma file. You also have the ability to export the original Gaussian Splat file, that is compatible with other interactive viewers, such as the SIBR viewer and nerfstudio.
One thing about Luma that I’ve always wanted is the ability to watch the NeRF train itself and form into the final scene. Now with this update, you can watch the scene be rapidly generated in front of your eyes.
Luma continues to be available on both iOS and web. Stay tuned for additional updates.
