News

X-Gaussian: Radiance meets Radiation

Michael Rubloff

Michael Rubloff

Mar 11, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
X Gaussian
X Gaussian

The array of use cases for radiance fields is quite staggering. Traditionally, when we discuss radiance fields, we focus on radiance itself. However, today there's a subtle yet significant shift towards radiation—yes, that kind of radiation. It’s not surprising that we’re venturing into an industry that we’ve only begun to explore: healthcare.

While previous radiance methods have ventured into applications like surgical reconstruction, our current focus is on X-rays, specifically through an innovation called "X-Gaussian." This approach utilizes 3D Gaussian splatting to significantly enhance the efficiency and quality of X-ray novel view synthesis.

By enhancing the tools available to physicians, we invariably increase the likelihood of receiving better medical care.

X-Gaussian transcends traditional limitations by utilizing a radiative Gaussian point cloud model, which uniquely ignores the viewing direction when predicting radiation intensity. This model, inspired by the isotropic nature of X-ray imaging, represents a departure from methods not well-suited to X-ray’s distinctive penetration and attenuation characteristics.

This model incorporates a Radiation Intensity Response Function (RIRF)—a novel concept that replaces the spherical harmonics typically used in RGB domain 3DGS. The RIRF facilitates the prediction of radiation intensity at 3D points using a set of learnable parameters, effectively simulating the X-ray interaction with object materials independent of viewing direction. This strategic shift not only aligns with X-ray imaging's physical properties but also streamlines the computational process, resulting in faster inference and reduced training times.

A standout innovation within X-Gaussian is the Differentiable Radiative Rasterization (DRR) with CUDA implementation, enabling the efficient rendering of novel X-ray projections. DRR begins by assigning radiation intensity values to each point in the cloud, based on the novel Radiation Intensity Response Function (RIRF). These points are then meticulously rasterized onto a 2D plane, ensuring that the essential characteristics of X-ray penetration and attenuation are preserved.

What sets DRR apart is its ability to perform this complex rendering in a differentiable manner, allowing for the seamless back-propagation of gradients during the optimization phase. This critical feature enables X-Gaussian to learn and adapt with unprecedented efficiency, refining its understanding of the object's radiative landscape with each iteration.

Additionally, the X-Gausssian k introduces an Angle-pose Cuboid Uniform Initialization (ACUI) strategy for camera calibration. Leveraging the geometric parameters of the X-ray scanner, ACUI accurately calculates the camera's intrinsic and extrinsic matrices, bypassing conventional structure-from-motion methods like COLMAP. This method not only optimizes the initialization process but also markedly decreases training duration.

The ACUI strategy employs a cuboid encompassing the object of interest, within which it uniformly samples initial points for the Gaussian point clouds. This method provides a more structured and efficient starting point for the optimization process, significantly cutting down the time required for training without sacrificing the quality of the synthesized views.

X-Gaussian has been evaluated against state-of-the-art methods, showcasing its superior performance with a notable improvement of 6.5 dB in image quality while achieving more than 73× faster inference speed and requiring less than 15% of the training time. Additionally, the framework's application in sparse-view CT reconstruction has highlighted its practical utility in medical imaging, promising a future where comprehensive imaging is possible with minimal X-ray exposure.

What this all means in practice is 100+ fps, with training times under 10 minutes, but still outputting high fidelity models.

This groundbreaking study not only advances the field of X-ray imaging but also sets a new benchmark for efficiency and quality in novel view synthesis. X-Gaussian represents a significant step forward in harnessing the power of 3D Gaussian Splatting for medical imaging, offering a promising avenue for future research and application in clinical settings.

The potential of radiance fields is bound only by the speed at which various industries recognize and leverage this technology. In all scenarios, it is us, the end-users, who stand to gain from access to superior innovations.

Featured

Featured

Featured

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

May 21, 2024

Platforms

Scaniverse arrives on Android

Gaussian Splatting platform, Scaniverse, is now available on Android.

Michael Rubloff

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

May 21, 2024

Platforms

Gracia launches on Steam and Sidequest

The VR Gaussian Splatting platform, Gracia, has now arrived on Steam and Sidequest, with more to come

Michael Rubloff

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

May 17, 2024

Research

CAT3D Pounces on 3D Scene Generation

We very recently were looking at RealmDreamer, which generates scenes from prompts. Just over a month later, CAT3D, short for "Create Anything in 3D," has emerged and takes things up a notch or two.

Michael Rubloff

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

May 15, 2024

Radiancefields.com launches Job Board

The latest feature has arrived onto the site and it's with the goal of connecting top talent to companies from newly launched start ups to the world's largest companies.

Michael Rubloff

To embed a website or widget, add it to the properties panel.

Trending articles

Trending articles

Trending articles

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Dec 13, 2023

Research

The MERF that turned into a SMERF

For the long time readers of this site, earlier this year, we looked into Google Research's Memory Efficient Radiance Fields (MERF). Now, they're back with another groundbreaking method: Streamable Memory Efficient Radiance Fields, or SMERF.

Michael Rubloff

Featured

Featured

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

SplaTV

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Michael Rubloff

Mar 15, 2024

SplaTV