News

Google Publishes Blog on Immersive View NeRFs

Michael Rubloff

Michael Rubloff

Jun 14, 2023

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Google Product NeRFs
Google Product NeRFs

As we have reported a couple of times this year, Google's release of Immersive View in Google Maps utilizes NeRFs. This revolutionary feature aims to provide photorealistic, multi-dimensional, and interactive experiences to users, giving them a virtual yet authentic feel of various locations like restaurants, cafes, and other venues.

While exteriors are still utilizing other methods, author Marcos Seefelder— describes the workflow of the NeRF interiors of venues.

Right now Google has been using a combination of Mip-NeRF 360, Block NeRF, and NeRF in the Wild.

Google then creates a specific camera path and enables an interactive 360 degree view for users. Google lists a challenge of dimly lit venues such as bars and night time events. However, filmmaker Jake Oleson has recently proven that it is possible to get high fidelity NeRFs in low light scenarios, capturing locations such as a crowded nightclub.

Google plainly describes the strength of NeRFs as:

"Once a NeRF is trained, we have the ability to produce new photos of a scene from any viewpoint and camera lens we choose."

Seefelder is not exaggerating here and it's one of the reasons why I am personally so excited about NeRFs. As soon as a NeRF has been generated, the options that are available to the user border on infinite and allow for tailored editing for each specific use case.

While that all sounds great, it's important to understand the actual workflow as to how Google ends up with NeRFs in Maps. Below is a video outlining that exactly.

Google claims to be able to fully capture a venue within an hour, but I would be curious to see their method, as that appears significantly longer than expected to me. It's also possible that they're looking at large structures, but on average it takes me roughly 20 seconds to 2 minutes depending on the size of a room. They are using DSLR cameras, which give the advantage of high resolution images, can often slow down the capture process as well as introducing a challenge of looking through a viewfinder.

They build their NeRFs on top of Mip-NeRF 360, which shouldn't be a surprise to anyone. It's a robust NeRF pipeline developed by Google and in my opinion, was long considered the gold standard of methods (until Google's Zip NeRF). It's no secret that NeRFs can suffer from artifacts. For the final product to be seen by end users, Google must ensure that their NeRFs are clean and free of artifacts. That's why they bring in low-dimensional generative latent optimization (GLO) vectors from NeRF in The Wild. These vectors embed appearance information for each image and allow for a cleaner final output. Finally, they take exposure conditioning from Block-NeRF to help with some of the dimly lit venues.

While outdoor NeRFs are yet to come for Google Maps, they did tease a small walk through of an outdoor path.

In doing so, we'd unlock similar experiences to every corner of the world and change how users could experience the outdoor world.

The result is stunning and showcases it handling the large amount of reflections you would find in a city. To me, this represents an exciting preview of what is to come and how we might find ourselves interacting with the world at large within the next year.

Google has said that Immersive View will continue to rollout and expand through the course of this year.

I tend to have option paralysis when it comes to picking a coffee shop to go to. LA takes its coffee (and food in general) very seriously and this results with amazing venues to choose from. Having Immersive View to give me all the information of each coffee shop ahead of time will speed up my decision making time and result in a better overall experience. That is if the coffee lives up to the NeRF.

Featured

Featured

Featured

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

To embed a website or widget, add it to the properties panel.

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV