Platforms

NeRFs in Unreal Engine 5 Alpha and Luma Fields Announced by Luma AI

Michael Rubloff

Michael Rubloff

Apr 3, 2023

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Luma AI Unreal Engine 5
Luma AI Unreal Engine 5

Luma AI has started April with a bang, announcing a integration of NeRFs in Unreal Engine 5. This continues to open the floodgates for professional uses and further developments into the gaming and VFX world. This groundbreaking product feature is set to revolutionize the way creators, developers, and gamers interact with virtual environments, offering unparalleled photo-realistic rendering capabilities through Luma Fields.

Previously, we have seen integrations to Nerfstudio with Volinga AI and Instant NGP with Velox. Luma had previously been quiet on any plans to expand into Unreal Engine, but the move, while inevitable— was unexpected so soon.

https://twitter.com/LumaLabsAI/status/1642883558938411008

Luma AI in Unreal Engine 5 Plugin

Getting Started with Luma AI in Unreal Engine

Download the plugin here and it is compatible with Windows.

Luma also suggests downloading their cinematic and film starter pack.

  1. Download the sample project and extract it somewhere

  2. Open it in Unreal Engine 5 by double-clicking the .uproject file

    • (This might initially take a while for Unreal to compile shaders if you have never opened Unreal Engine before or didn't compile some of Unreal's shaders before.)

  3. Download a Luma Field file from one of your captures

  4. Drag the downloaded file into the Content Browser in the Unreal Editor

  5. After a couple of seconds, Blueprints will be generated for your NeRF captures, which you can simply drag in and use inside Unreal Engine!

    • Depending on your system, it might take a couple of seconds to compile all the Blueprints – you can already drag them into the scene while they are compiling and they will start to render once the compilation is done.

  6. There are multiple Blueprints generated for each Luma Field

    • Baked: These Blueprints contain the lighting as you captured them, so they will look as close to your capture as possible. They are not affected by shadows, or lighting, but can still be composited with other Luma Fields and Unreal Objects.

    • Dynamic: These Blueprints are affected by Unreal's lighting and shadows.

    • Cropped: These Blueprints contain an automatically cropped version of the center object of your capture, if one exists.

    • With Environment: These Blueprints additionally render a fitting sky environment for each capture for any region that was unseen during capture.

If you are planning on using the plugin in to view a Luma Field in VR, Luma AI recommends:

we recommend using the DLSS plugin on NVidia GPUs with a ScreenPercentage of 50% to ensure smooth rendering and prevent motion sickness.

Luma AI

NeRFs, are a cutting-edge technology that harnesses the power of artificial intelligence and deep learning, have gained immense popularity for their ability to generate incredibly lifelike scenes. The technology works by capturing a scene's 3D structure and appearance from a collection of 2D images, rendering the scene with stunning accuracy and detail. With Luma AI's latest innovation, users can now access NeRFs directly within Unreal Engine, opening up new creative possibilities and transforming the way digital content is developed and consumed.

The introduction of NeRFs in Unreal Engine is expected to have a significant impact on industries such as gaming, architecture, film, and virtual reality, where high-quality 3D rendering is crucial for delivering immersive and realistic experiences. By offering a more efficient and visually impressive method of generating 3D content, Luma AI's latest feature has the potential to redefine industry standards and streamline the creative process for professionals in these fields.

Luma AI's NeRFs-Unreal Engine integration has already garnered considerable interest from the global tech community, with many eager to explore the new horizons it will open up. As developers and creators continue to push the boundaries of what is possible in the digital world, the incorporation of NeRFs in Unreal Engine is poised to become a major milestone in the evolution of 3D rendering technology. We are entering a new, exciting chapter of NeRF and I am excited to see what is created from it.

To learn more about Luma AI's groundbreaking NeRFs integration in Unreal Engine and its implications for the future of 3D rendering, visit their website at https://captures.lumalabs.ai/

About Luma AI:

Luma's mission is to enable everyone to capture and experience the world in lifelike 3D. To bring about the next step function change in how we share memories, explore products, and spaces on the internet. To propel photos and videos into our mixed-reality 3D future. Some of the brightest minds in computer vision and graphics have dedicated their careers to solving this problem of photorealistic 3D capture.

Now, advances in neural rendering, deep learning, and compute have finally brought this dream within our reach. If thinking about this future and this monumental engineering challenge gets you excited, and wide-eyed with ideas, come make it real with us. Join our founding team with the autonomy to experiment, to grow, and to help form Luma's technical, product, and business roadmap.

Featured

Featured

Featured

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

May 24, 2024

Platforms

OpenNeRF added to Nerfstudio

OpenNeRF is the latest method to be supported by Nerfstudio.

Michael Rubloff

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

May 24, 2024

Platforms

PlayCanvas's SuperSplat Updated with PWA support

PlayCanvas's Supersplat has continued to receive additional updates. This time it's coming with a big boost to performance, yet again.

Michael Rubloff

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

May 23, 2024

Research

Reflecting on NeRF-Casting

Late last year we looked at Uni-SDF which introduced dual radiance fields to better represent reflections in a scene. However, I just happened to see NeRF-Casting on Github a little while ago

Michael Rubloff

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV