News|Platforms

Transforming 3D Mapping: Spectacular AI's Radiance Field Revolution

Michael Rubloff

Michael Rubloff

Nov 20, 2023

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Spectacular AI
Spectacular AI

Creating near live radiance fields and bypassing the COLMAP step is amazing and thus, I'm excited to tell you about Spectacular AI's offering. Spectacular AI is a Finnish company offering next-generation visual-inertial tracking solutions that enable real-time relative position estimation.

Their Mapping API enables fast 3D reconstructions on devices supported by the Spectacular AI SDK. With the latest version, NeRFs and Gaussian Splatting reconstructions can be created in minutes on off-the-shelf-supported devices such as OAK-D, RealSense and Kinect cameras. Spectacular AI has also mentioned that out of the box iPhone support will be coming, which will be a massive boost to the radiance field ecosystem.

Technically, it replaces the slow COLMAP phase in fully open-source-based pipelines, by leveraging the proprietary Spectacular AI sensor fusion stack, which is free to use for non-commercial purposes with the public SDK version.

Right now, they offer support to two of the industry leading applications: Instant-NGP and nerfstudio.


While nerfstudio has not formally announced their full Gaussian Splatting pipeline, that hasn't stopped Spectacular AI from incorporating it. As of November 14th, their method is now supported to be used with nerfstudio's gsplat. In order to install it, check out the instructions below.

  1. Install Nerfstudio and activate the Conda environment

  2. Install gsplat to the same Conda environment git clone https://github.com/nerfstudio-project/gsplat; cd gsplat; pip install -e .

  3. Train as ns-train gaussian-splatting --data /example/output/path/my-nerf

For those that would like to use the SDK for commercial purposes, they do offer a commercial license, but you would need to contact them directly to discuss. I, however, will be exploring creating NeRFs in real time, as soon as I can get my hands on a Kinect.

It might sound like science fiction to imagine live NeRFs, but I promise you they exist; I was NeRFed live on stage over a year ago at Infinity Festival by NVIDIA's Jason Schugardt and Jonathan Stephens.

Spectacular AI's advancements in the field of 3D reconstruction and live NeRFs represent a significant leap forward in visual-inertial tracking technology. By eliminating the need for the cumbersome COLMAP step in 3D mapping, they have streamlined the process, making it faster and more accessible. Their support for devices like OAK-D, RealSense, and Kinect, coupled with their upcoming expansion to additional devices and iPhone support, indicates a future where sophisticated 3D mapping and NeRF creation are widely available and easily achievable.

As the barriers to creating live NeRFs and detailed 3D reconstructions continue to fall, we can expect a surge in innovative applications and experiences. Spectacular AI is not just keeping pace with this rapidly evolving field; they are actively pushing its boundaries, promising exciting developments for both commercial and non-commercial users. With live NeRFs transitioning from a science fiction concept to a tangible reality, the possibilities for what can be achieved in virtual and augmented reality, film, gaming, and beyond are truly spectacular.

Featured

Featured

Featured

Platforms

BitbyBit Integrates Gaussian Splatting

Lithuanian company Bit by Bit developers has integrated Gaussian Splatting (3DGS), enabling low-code applications.

Michael Rubloff

Jul 5, 2024

Platforms

BitbyBit Integrates Gaussian Splatting

Lithuanian company Bit by Bit developers has integrated Gaussian Splatting (3DGS), enabling low-code applications.

Michael Rubloff

Jul 5, 2024

Platforms

BitbyBit Integrates Gaussian Splatting

Lithuanian company Bit by Bit developers has integrated Gaussian Splatting (3DGS), enabling low-code applications.

Michael Rubloff

Platforms

Nerfstudio Integrates 3DGS MCMC

Nerfstudio now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

Michael Rubloff

Jul 3, 2024

Platforms

Nerfstudio Integrates 3DGS MCMC

Nerfstudio now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

Michael Rubloff

Jul 3, 2024

Platforms

Nerfstudio Integrates 3DGS MCMC

Nerfstudio now supports 3DGS MCMC, which should increase the fidelity of outputs across the board.

Michael Rubloff

Research

Diffusion Based 3DGS Relighting

Research topics on relighting continue to increase sharply.

Michael Rubloff

Jul 2, 2024

Research

Diffusion Based 3DGS Relighting

Research topics on relighting continue to increase sharply.

Michael Rubloff

Jul 2, 2024

Research

Diffusion Based 3DGS Relighting

Research topics on relighting continue to increase sharply.

Michael Rubloff

Platforms

Voluma.ai adds Timelapse Mode

Voluma.ai, specializing in the creation and augmentation of 3D graphics, has introduced its latest feature: timelapse mode.

Michael Rubloff

Jul 1, 2024

Platforms

Voluma.ai adds Timelapse Mode

Voluma.ai, specializing in the creation and augmentation of 3D graphics, has introduced its latest feature: timelapse mode.

Michael Rubloff

Jul 1, 2024

Platforms

Voluma.ai adds Timelapse Mode

Voluma.ai, specializing in the creation and augmentation of 3D graphics, has introduced its latest feature: timelapse mode.

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

Mip-Splatting: Anti-Aliasing for Gaussian Splatting

If you've been trying out Gaussian Splatting, you may have noticed that the scene tends to degrade quickly, especially when you pull outside of the capture path or change the focal length, compared to NeRF.

Michael Rubloff

Nov 28, 2023

Research

Mip-Splatting: Anti-Aliasing for Gaussian Splatting

If you've been trying out Gaussian Splatting, you may have noticed that the scene tends to degrade quickly, especially when you pull outside of the capture path or change the focal length, compared to NeRF.

Michael Rubloff

Nov 28, 2023

Research

Mip-Splatting: Anti-Aliasing for Gaussian Splatting

If you've been trying out Gaussian Splatting, you may have noticed that the scene tends to degrade quickly, especially when you pull outside of the capture path or change the focal length, compared to NeRF.

Michael Rubloff