Lifecast.ai officially announced its groundbreaking platform on Monday, May 20th, introducing incredible applications that bring Neural Radiance Fields (NeRFs) to Mac, XR, and browser environments.
Lifecast emerged from Y Combinator in 2021, with founder Forrest Briggs, Ph.D., at the helm. Briggs brings an impressive background with over 40 patents and experience at JPL, Facebook, Lyft, and Google X. Lifecast is dedicated to pushing the limits of immersive media while creating practical tools for creators to unleash their creativity.
No, your eyes do not deceive you upon visiting lifecast.ai. That is a NeRF video running in a browser and is able to be accessed in VR too. Lifecast exports their NeRFs into a layered depth image, otherwise known as a ldi3 format, which enables real-time WebXR viewing and integration with Unreal Engine 5.1 or Unity.
Underneath Lifecast are three main branches:
lifecast.ai - The home of Lifecast’s open source projects, including a NeRF video engine in C++ that compiles on Mac, Windows, and Linux, and video players for the ldi3 format in JavaScript, Unreal Engine, and Unity. Lifecast published a paper on how to compress NeRFs into layered depth images (ldi3 format), the details of their NeRF implementation. Lifecast’s Github comes with an MIT License for people to build on top, and their structure from motion solver (which can be used instead of COLMAP).
Volurama - Volurama is a GUI for Mac and Windows that runs Lifecast’s NeRF engine for static scenes (not dynamic NeRFs yet), and can train with GPU acceleration on Mac. We’ve seen a little bit of radiance fields appear on Mac through OpenSplat, but never with NeRFs before. Volurama can render videos where a virtual camera flies through the scene, and it is flexible about the properties of the virtual camera. For example, it can simulate a VR180 camera, export to Looking Glass’s Quilt format (for holographic displays), or render stereoscopic output which can be converted into spatial video. It can also export ldi3, which bakes the NeRF into a format that can be rendered in real time on the web. With Volurama’s 1.5 update, it adds an option to export OBJ triangle meshes which are compatible with blender. Most recently with V1.6, now uses tiny cuda nn on Windows, giving it another boost of about 1.5 times.
Right now I've found that videos underneath 30 seconds work best to reconstruct on my Macbook Pro. Once you've gotten your NeRF, you can set camera keyframes to render out. It's still a bit slow compared to running on Windows, but this is a massive bump forwards from anything that's come before it for Mac. Be aware that in order to render out a capture, you will need a license from Volurama, which runs $99.
Volumetric Video Editor - VVE is a tool for Windows and Mac which converts VR180 (stereo) video into ldi3 (volumetric) video using machine-learning and inpainting. It is not based on NeRF (it’s their previous generation of tech), but exports into the same ldi3 format.
Creators have begun embracing Lifecast’s Volurama, with inclusions in short films like Hugh Hou’s Dolce Mia.
Additional features and improvements are on the way from the California based team. To get in touch with the Lifecast team, they can be reached either by email or the Lifecast Facebook Group.
