Research

Noise-NeRF: Hidden Messages in NeRFs

Michael Rubloff

Michael Rubloff

Jan 10, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Noise-NeRF
Noise-NeRF

Back in middle school, I really thought that more of my life would involve decoding hidden messages in images for some reason. With Noise-NeRF, we're seeing that it's possible and it actually works pretty well.

This has always been a question in the back of my head, but I had never been able to understand how it might be done until today. This is, how to encode images or messages within NeRFs without it being noticeable. It's is a technique called steganography (not stenography), where people hide data or messages within an ordinary files. Noise-NeRF expands on this by not only maintain the integrity of the original NeRF model, but also allow for information to be securely embedded into 3D models.

There's a range of what you can do with that, both good and bad, but let's focus on some of the positives. By incorporating steganography, it serves as provenance for copyright verification. This would alleviate a lot of the questions about where data has been trained from or created by. It also would help with licensing conversations and to verify how commercially used projects are created and ensure that they have been properly licensed.

I actually am a little curious if any of the existing platforms, such as Luma AI or Polycam have looked into or are already using steganography with their outputs. Noise-NeRF consumes ~16GB of VRAM in the examples shown and are trained using a 3090, so it's not impossible, but the question is is it worth it right now for these companies? Potentially. I have seen so many conversations take place surrounding the provenance of generative AI and while this doesn't necessarily fall into that category, I believe that as 3D continues to emerge, having some level of imperceptible documentation will be important.

In someways, it reminds me of the Gaussian Painters project from Alexandre Carlier, but this extends in 3D.

Let's take a look now at how Noise-NeRF actually works. They propose two strategies to help power Noise-NeRF: Adaptive Pixel Selection and then a Pixel Perturbation Strategy. These two strategies are pivotal in enhancing the steganography quality and efficiency of Noise-NeRF.

Instead of treating all pixels uniformly, they propose something called a Adaptive Pixel Selective Strategy, which identifies and select pixels that show greater sensitivity or responsiveness to the injected noise. Obviously, it won't be perfect straight away, and thus needs some level of optimization and refinement. As the embedding progresses, it dynamically adjusts its focus, choosing different pixel groups based on their performance in prior iterations.

This iterative optimization ensures that each pixel receives the exact level of attention required. Grouping pixels together wouldn't look quite right due to view-dependent effects. Therefore, different pixels may require varying levels of iteration or noise adjustments

While Adaptive Pixel Selection focuses on efficiency and precision, they introduce the second part to Noise-NeRF, a Pixel Perturbation Strategy. It's designed to enhance the speed and effectiveness of the steganographic embedding. In the initial stages of embedding, early stage deviation aims to create a significant deviation of the rendered image from the original. This is achieved by introducing calculated perturbations or noise.

By causing a more substantial initial deviation, the Pixel Perturbation Strategy helps the steganographic process to reach its desired state more quickly— kind of like giving the process a head start. The strategy ensures that the perturbations are significant enough to speed up the process but not so large as to degrade the quality of the final steganographic output.

The combination of these two strategies allows Noise-NeRF to efficiently embed high-quality steganographic information into NeRFs. While Adaptive Pixel Selection ensures that the embedding is precise and efficient, Pixel Perturbation accelerates the process, ensuring quick convergence to high-quality results.

The authors mention a surprising lack of research in this area and as far as I'm aware, they're correct. It's something I haven't really encountered, but believe that it will be important to have documentation of copyright information, but also the type of data that is being stored. Hopefully we'll steer clear of Snow Crash, and it opens up conversations about what is being messaged.

Featured

Featured

Featured

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

To embed a website or widget, add it to the properties panel.

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV