Platforms

Luma-Web: WebGL Library from Luma AI

Michael Rubloff

Dec 6, 2023

Luma-web
Luma-web

The world of 3D rendering is evolving rapidly, and the expectations associated with each new development continues to rise. The Luma team are stepping expectations up a notch by unveiling Luma-web, a npm package for rendering photoreal interactive scenes captured by the Luma app.

Over the course of the past month, we have seen several companies roll out features, such as editing, a Javascript library, website support, and most recently, VR. Luma-web will support all of those features and more.

Designed to integrate seamlessly with popular 3D frameworks, luma-web offers a sophisticated approach to real time rendering of photorealistic, interactive scenes captured by the Luma app.

Key Features of Luma-Web

  1. LumaSplatsWebGL and LumaSplatsThree: The package includes two core components: LumaSplatsWebGL, a WebGL only gaussian splatting implementation, and LumaSplatsThree, a Three.js implementation leveraging LumaSplatsWebGL. These tools are pivotal for creating detailed, realistic scenes.

  2. Integration with Three.js: Luma-Web is designed to work in tandem with Three.js, one of the most popular 3D JavaScript libraries. This integration ensures that developers can easily incorporate Luma-Web into existing Three.js projects, benefiting from its advanced rendering capabilities.

  3. Comprehensive Documentation: The GitHub page for luma-web provides extensive documentation, covering everything from getting started to advanced techniques like custom shaders and VR integration. This resource is invaluable for developers looking to leverage the full power of the package. Additionally, Luma has a resource dedicated to Luma-web through their website.

Here are some more of the features that are coming with the announcement:

Background Removal: Luma-Web enables selective rendering of semantic layers, such as foreground and background, providing flexibility in scene composition.

Three Fog Integration: The package supports integration with Three.js's fog feature, enhancing the atmospheric quality of scenes. This is part of an integration with the three.js rendering pipeline and also extends to tone mapping and color spaces.

Scene Lighting: Developers can illuminate scenes using Luma splats, capturing a cubemap of the splats and using it as the scene environment.

Custom Shaders: Luma-Web allows for the customization of splat shaders, enabling developers to inject their own GLSL code to achieve unique visual effects.

React Three Fiber Compatibility: The package can be used with React Three Fiber, extending its capabilities to the popular React framework.

Transmission: Special care has been taken to integrate transmission effects with splats, catering to the specific requirements of transparent materials in Three.js.

VR Support: For those who have been patiently waiting for their Luma captures to be accessible in VR (me), your wait is now over. Through their implementation, you are able to enter VR in all of your captures. Luma-Web is VR-ready, allowing for immersive experiences in virtual reality environments. The Luma team recommends viewing your captures on a PC VR device, as standalone VR does not handle the splats well yet.

Depending on your collection of captures, the VR feature may hold significant sentimental value to you. What was once displayed on a screen in photorealistic detail, will now be able to be experience in Virtual Reality. For me, this means captures of my parents, grandmother, and friends. I had previously been utilizing Instant-NGP's VR tool and most recently Gracia, but now will have another extremely compelling platform to experience this with.

Getting Started with Luma-Web

Setting up luma-web is straightforward. Developers can install the package using npm (npm install @lumaai/luma-web) and import the LumaSplatsThree class into their project. The package also supports browser usage through a script tag. Once imported, a new instance of LumaSplatsThree can be created and added to the scene, with the source being either a URL to a capture on lumalabs.ai or a path to a luma splats file or folder.

To optimize performance, developers are advised to disable MSAA when creating the renderer and to set enableThreeShaderIntegration to false if not using features like fog and tone mapping.

The full installation instructions can be found on Luma's Github.

This appears to answer the question from yesterday about the unannounced feature powering the Looking Glass Go integration. However, Luma has yet to confirm if these are one in the same.

There are so many features contained in this announcement that I am inordinately excited to try out. Between VR, custom shaders, and what it could represent for companies looking to integrate photorealism into their ecommerce offerings, the implications are massive. Further, I believe that this will supercharge creators abilities, such as Alexandre Devaux, Jake Oleson, GradeEterna, Franc Lucent, and many more.

Think about how virtual tourism will be transformed by access to photorealistic VR applications of places! You will see explorable places in front you that you would have otherwise never seen. This extends to so many verticals, including museums and cultural heritage. The implications are massive! Every day we are slowly, but surely inching towards the day where photography is evolved into its next form. Launches like this and Looking Glass Go help us get there just a little bit faster than the day before.

You can see the full documentation on both the Luma Github page as well as through their website. If you have any questions or would like to meet with the Luma team, they can be reached through hello@lumalabs.ai.

Featured

Featured

Featured

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

4/11/24

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

4/10/24

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

4/9/24

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

Apr 11, 2024

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

Apr 11, 2024

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

4/11/24

Trending articles

Trending articles

Trending articles

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Recent articles

Recent articles

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

3dgs compress

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

3dgs compress

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024