Research

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Michael Rubloff

Michael Rubloff

Jan 12, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
TRIPS
TRIPS

Gaussian Splatting has been the hottest radiance field method for the past four months, since SIGGRAPH. Well, it seems like we might have a new radiance field method called TRIPS, Trilinear Point Splatting for Real-Time Radiance Field Rendering that might have something to say about that.

Developed by a team of researchers at Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany, TRIPS builds upon some of the methodology that 3D Gaussian Splatting put forth and increases fidelity, without sacrificing real time capability.

I am very, very excited about this new radiance field type. TRIPS leverages two distinctive methods, 3D Gaussian Splatting and Approximate Differentiable One-Pixel Point Rendering (ADOP) to generate the best from both. If you deviate too far away from the initial capture path with 3DGS, the scene doesn't always hold up, introducing blurriness and artifacts. Whereas, ADOP excels at mapping textures and gaps in space, but does come with a neural network overhead, albeit a smaller one.

TRIPS introduces a novel rasterization process where points from a point cloud are converted into a screen-space image pyramid. Unlike traditional methods that render points at a single resolution, TRIPS employs multiple layers of different resolutions, akin to a pyramid structure (remember PY-NeRF?)

The decision-making process about which pyramid layer to use for rasterizing each point shows off its advanced design. This decision is based on the projected size of the point in the screen space. Larger points are rendered on lower resolution layers, efficiently covering more space in the image pyramid. Conversely, smaller points are mapped to higher resolution layers, ensuring fine details are preserved. This strategic rasterization approach, particularly the use of 2×2×2 trilinear splats, showcases TRIPS's capability to handle diverse point sizes adeptly, without compromising on speed or image quality.

After the rasterization step, TRIPS employs a neural network to reconstruct a detailed, gap-free image. This step is pivotal in ensuring that the final rendered image is devoid of gaps and artifacts. The lack of a neural network is something that people love about Gaussian Splatting, but the network in TRIPS is not just any network. It’s specifically streamlined for efficiency, featuring gated convolutions and a self-bypass connection. This architecture allows the network to perform its complex task with minimal computational demand.

By efficiently managing the rasterization process through the image pyramid, TRIPS significantly reduces the workload on the neural network. The network, therefore, can focus more on refining image quality.

The entire rendering pipeline of TRIPS is differentiable, allowing for automatic optimization of both point sizes and positions, a feature that enhances accuracy and quality. By optimizing input parameters, the system creates robust scene representations that are true to the source material. Despite the complexity of the process, TRIPS maintains a real-time frame rate of 60 frames per second.

TRIPS's performance extends to challenging scenarios, including scenes with intricate geometry and expansive landscapes. One of the remarkable aspects of TRIPS is its ability to handle scenes with complex geometries and large-scale environments efficiently.

These results are impressive no doubt, but TRIPS does take a bit longer to train than 3DGS or Instant NGP. It takes roughly 2-4 hours to train, with an average render speed of 15ms on a 4090. Still that is impressive given the fidelity that is output.

Their Github is currently blank, except for a ReadMe, however a license is listed as MIT (!) We will have to see if whether or not that remains true as it is populated with the code.

It took 3D Gaussian Splatting several months after it was initially published prior to interest being piqued out of SIGGRAPH. While the paper has only newly been announced and the full video and code have not been released yet, on its face it represents an exciting new radiance field method. It goes without saying that we will be watching TRIPS carefully.

Featured

Featured

Featured

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Jun 18, 2024

Platforms

Varjo Unveils Teleport

Finnish VR company Varjo has announced 3DGS platform, Teleport.

Michael Rubloff

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Jun 17, 2024

Platforms

Wayve Announces PRISM-1

Wayve, has announced the launch of PRISM-1, an innovative 4D reconstruction model designed to significantly enhance the testing and training of advanced driver assistance systems and autonomous driving technology.

Michael Rubloff

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Jun 14, 2024

Platforms

Nerfstudio Gradio UI Added

Nerfstudio now includes a Gradio UI, making it easy and intuitive to set commands and run scripts.

Michael Rubloff

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Jun 13, 2024

Platforms

Graswald.ai: 3DGS and Ecommerce Platform

Graswald is a 3D product visualization platform using Gaussian Splatting, aimed primarily at the Ecommerce industry to offer hyper realistic detail at a fraction of current time and cost. 

Michael Rubloff

Trending articles

Trending articles

Trending articles

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Mar 15, 2024

Tools

splaTV: Dynamic Gaussian Splatting Viewer

Kevin Kwok, perhaps better known as Antimatter15, has released something amazing: splaTV.

Michael Rubloff

Featured

Featured

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Just in time for your weekend, Ruilong Li and the team at Nerfstudio are bringing a big gift.

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Nerfstudio Releases gsplat 1.0

Michael Rubloff

Jun 7, 2024

Nerfstudio

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

It doesn't seem like a lot of people know this, but you can run CamP and Zip-NeRF in the cloud, straight through Google and it's actually super easy. It’s called CloudNeRF.

Michael Rubloff

May 8, 2024

Platforms

Google CloudNeRF: Zip-NeRF and CamP in the Cloud

Michael Rubloff

May 8, 2024

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Gaussian Splatting methods have continued to pour in over the first three months of the year. With the rate of adoption, being able to merge and compare these methods, shortly after their release would be amazing.

Michael Rubloff

Apr 8, 2024

Gaustudio

Research

Gaustudio

Michael Rubloff

Apr 8, 2024

Gaustudio