News

X-Gaussian: Radiance meets Radiation

Michael Rubloff

Mar 11, 2024

X Gaussian
X Gaussian

The array of use cases for radiance fields is quite staggering. Traditionally, when we discuss radiance fields, we focus on radiance itself. However, today there's a subtle yet significant shift towards radiation—yes, that kind of radiation. It’s not surprising that we’re venturing into an industry that we’ve only begun to explore: healthcare.

While previous radiance methods have ventured into applications like surgical reconstruction, our current focus is on X-rays, specifically through an innovation called "X-Gaussian." This approach utilizes 3D Gaussian splatting to significantly enhance the efficiency and quality of X-ray novel view synthesis.

By enhancing the tools available to physicians, we invariably increase the likelihood of receiving better medical care.

X-Gaussian transcends traditional limitations by utilizing a radiative Gaussian point cloud model, which uniquely ignores the viewing direction when predicting radiation intensity. This model, inspired by the isotropic nature of X-ray imaging, represents a departure from methods not well-suited to X-ray’s distinctive penetration and attenuation characteristics.

This model incorporates a Radiation Intensity Response Function (RIRF)—a novel concept that replaces the spherical harmonics typically used in RGB domain 3DGS. The RIRF facilitates the prediction of radiation intensity at 3D points using a set of learnable parameters, effectively simulating the X-ray interaction with object materials independent of viewing direction. This strategic shift not only aligns with X-ray imaging's physical properties but also streamlines the computational process, resulting in faster inference and reduced training times.

A standout innovation within X-Gaussian is the Differentiable Radiative Rasterization (DRR) with CUDA implementation, enabling the efficient rendering of novel X-ray projections. DRR begins by assigning radiation intensity values to each point in the cloud, based on the novel Radiation Intensity Response Function (RIRF). These points are then meticulously rasterized onto a 2D plane, ensuring that the essential characteristics of X-ray penetration and attenuation are preserved.

What sets DRR apart is its ability to perform this complex rendering in a differentiable manner, allowing for the seamless back-propagation of gradients during the optimization phase. This critical feature enables X-Gaussian to learn and adapt with unprecedented efficiency, refining its understanding of the object's radiative landscape with each iteration.

Additionally, the X-Gausssian k introduces an Angle-pose Cuboid Uniform Initialization (ACUI) strategy for camera calibration. Leveraging the geometric parameters of the X-ray scanner, ACUI accurately calculates the camera's intrinsic and extrinsic matrices, bypassing conventional structure-from-motion methods like COLMAP. This method not only optimizes the initialization process but also markedly decreases training duration.

The ACUI strategy employs a cuboid encompassing the object of interest, within which it uniformly samples initial points for the Gaussian point clouds. This method provides a more structured and efficient starting point for the optimization process, significantly cutting down the time required for training without sacrificing the quality of the synthesized views.

X-Gaussian has been evaluated against state-of-the-art methods, showcasing its superior performance with a notable improvement of 6.5 dB in image quality while achieving more than 73× faster inference speed and requiring less than 15% of the training time. Additionally, the framework's application in sparse-view CT reconstruction has highlighted its practical utility in medical imaging, promising a future where comprehensive imaging is possible with minimal X-ray exposure.

What this all means in practice is 100+ fps, with training times under 10 minutes, but still outputting high fidelity models.

This groundbreaking study not only advances the field of X-ray imaging but also sets a new benchmark for efficiency and quality in novel view synthesis. X-Gaussian represents a significant step forward in harnessing the power of 3D Gaussian Splatting for medical imaging, offering a promising avenue for future research and application in clinical settings.

The potential of radiance fields is bound only by the speed at which various industries recognize and leverage this technology. In all scenarios, it is us, the end-users, who stand to gain from access to superior innovations.

Featured

Featured

Featured

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

4/11/24

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

4/10/24

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

4/9/24

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

Apr 11, 2024

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

Apr 11, 2024

Research

RealmDreamer's Generative Scenes

Since the unveiling of the Sora's large-scale generative Radiance Fields, the tech world has been buzzing with anticipation about the future of 3D scene generation. There hasn't been much public work since then showcasing what could be coming, but today we're looking at RealmDreamer, which creates scene level generations based on original text prompts.

Michael Rubloff

4/11/24

Trending articles

Trending articles

Trending articles

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Research

RadSplat's Hybrid NeRFs and 3DGS

Excitingly, we're seeing the arrival of the first result of another widely hyped event. The meeting of NeRFs and Gaussian Splatting.

Michael Rubloff

Mar 21, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Interview

Gaussian Splatting Brings Art Exhibitions Online with Yulei

The advent of radiance fields represents a transformative leap in event photography, aiming to give people the feeling of attending an event asynchronously. Artist Yulei's recent demonstration serves as a compelling example of this technological progression.

Michael Rubloff

Feb 22, 2024

Recent articles

Recent articles

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

3dgs compress

Research

Shrinking 3DGS File Size

Gaussian Splatting has quickly become one of the most exciting research topics in Radiance Fields, thanks to its fast training, real time rendering rates, and easy to create pipeline. The one critique that emerged was the resulting file size from captures, often venturing into the high hundreds of megabytes and up.

Michael Rubloff

Apr 11, 2024

3dgs compress

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Platforms

Luma AI Android Released

Native Android support from Luma AI is finally here. Of all the questions about Luma features I get, Android support is routinely at the top of the list.

Michael Rubloff

Apr 10, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024

Research

PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Apr 9, 2024