Research

Gaussian Splashing Splashes Down

Michael Rubloff

Michael Rubloff

Jan 30, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Gaussian Splashing
Gaussian Splashing

When physics based radiance fields landed last year, they certainly made a splash with PhysGaussian and PieNeRF. Now, a follow up paper is taking it up a notch with Gaussian Splashing.

They are able to incorporate dynamic fluids into gaussian splatting and it unlocks so many things. It definitely gets me thinking about what you can do with this paired with GARField.

With their last paper, I got excited about jam spreading itself across a piece of toast and introduced the concept of What you see, is what you simulate. With Gaussian Splashing, I have now seen another thing I never thought that I would. Here is a gaussian splatted cup, which is being filled with water, before dissolving itself.

While the endpoint is unique, the beginning of Gaussian Splashing is the same as all radiance field methods. They takes a collection of 2D images of a 3D scene, each featuring different viewpoints. From there, the preprocessing stage begins and it separates foreground objects from the background. and extracts their surfaces.

GSP uses advanced deep segmentation tools to isolate foreground objects and create a detailed reconstruction of them. Gaussian kernels are placed inside and on the surface of these models, acting as mass particles for simulation. As part of understanding where the boundaries lie, they need to have accurate representations and for this Gaussian Splashing turns to Neus. I would be very curious how this would work with a similar model, such as UniSDF or Instant Angelo.

During the actual training, they use a differentiable version of 3DGS, so that their visual attributes are taken into account. Normal Gaussian Splatting cannot be used here because it does not come with well defined appearance properties.

GSP employs Lagrangian fluids and position-based fluids (PBF) to simulate fluid dynamics. It also includes a position-based tension model to better capture the dynamics of the fluid surface. This approach is essential for creating realistic fluid movements and interactions with solid objects.

To prevent rendering artifacts, especially near the surfaces of solid objects, GSP employs anisotropy loss and regularization. This controls the shape of Gaussian kernels to avoid them becoming excessively elongated or compressed. To address the issue of displaced objects exposing unseen areas, the system uses an inpainting technique called Resolution-robust large mask inpainting with fourier convolutions (LaMA). It removes Gaussian kernels of the displaced object and uses a generative AI approach to inpaint these areas, improving the overall visual quality of the rendered image so that nothing looks out of the ordinary.

It makes sense why you would need that area to be inpainted. When an object is removed, how would it know what should be there?

Gaussian Splashing is accomplished with consumer grade hardware— the researchers are using 3090s and it doesn't take an unusually long time to train. Most of the examples shown in the paper/demo range from just over half an hour, going all the way up to two and a half, which is still faster than nerfacto-huge.

I, for one, am excited to see the recreation of Titanic, sans original boat and actual water. However, you can also expand this to look at simulations and evacuation methods.

Physics are how the world works, so it is always a pleasure to read about how the most lifelike models of our world that we have, can also be subject to the same rules. I can't wait for more methods to build on top of this, so that we can truly be transported and interact with the world as we see it, virtually.

Neither the code for Gaussian Splashing or supplementary video have been made available yet, but I am excited for both. You can join me in refreshing their project page, here.

Featured

Featured

Featured

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Featured

Featured

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio