Gaussian Splatting Unity Plugin gets VR Support

Michael Rubloff

Michael Rubloff

Dec 4, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Unity Gaussian Splatting
Unity Gaussian Splatting

Quietly over the Thanksgiving weekend, a PR was merged into the Unity repository by Ninjamode, which brings VR access to 3DGS Unity projects using Aras's plugin. If you haven't tried viewing a Radiance Field in VR, I highly suggest it as the view-dependent effects carry over beautifully into VR.

Aras's plugin has widely been declared as the de facto option for 3DGS in Unity. While Aras is no longer actively developing the plugin, PRs are welcome to the MIT-licensed codebase.

Implementation Instructions

Getting the VR capabilities up and running is not as difficult as you might imagine. Follow these steps once you have added the Unity Plugin:

  1. Use Unity’s Universal Render Pipeline (URP) for optimal results. The Standard Pipeline may encounter matrix-related issues.

  2. Enable XR Plugin Management.

  3. Start OpenXR on initialization.

  4. Set the Render Mode to Multi-Pass.

Some of the under the hood updates include a built-in matrix integration. The new implementation replaces custom P and VP matrix usage with Unity’s built-in equivalents, streamlining compatibility and reducing custom code dependencies.

It also includes improved GPU support by incorporating a newer version of DeviceRadixSort, offering enhanced GPU compatibility, especially tailored for devices like the Meta Quest 3. This resolves prior inconsistencies noted in Android-based devices, referencing b0nes164/GPUSorting#4.

The system now uses eye texture size in screen size calculations when available, optimizing the visual fidelity and performance of VR rendering setups.

While the update delivers significant improvements, it notably does not include sorting optimizations for eye center alignment. The contributor cited the lack of robust implementation as the reason for exclusion but suggested it could be a worthwhile optimization to revisit, particularly for Desktop VR scenarios. A sample implementation is available in the originating repository for developers interested in exploring this further.

The updated system has undergone testing across multiple platforms, including Desktop Steam VR (HTC Vive), Varjo devices (tested on Aero), Meta Quest 3 and Quest Pro (via Link and standalone). Notably excepted is the Apple Vision Pro. However, people with the Vision Pro can use platforms such as MetalSplatter.

This move also triggered Aras to push the 1.0 update to the repository, though the updates are not as large as the "1.0" version number might imply. The repository remains MIT-licensed for those interested in contributing to the project.

As always, collaborative testing and feedback are key to ensuring robust solutions. If you have a VR setup, testing this feature could help refine and validate the implementation for the broader Unity community.

Featured

Recents

Featured

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

Jan 11, 2025

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

Jan 11, 2025

Platforms

Tripo WebApp v2.0

Tripo is giving a refresh to their WebApp and more!

Michael Rubloff

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Jan 9, 2025

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Jan 9, 2025

News

Jensen Huang unveils two Radiance Fields in CES Keynote

Jensen welcomed CES attendees into NVIDIA through Radiance Fields.

Michael Rubloff

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

Jan 9, 2025

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

Jan 9, 2025

Platforms

StorySplat v1.3.6.1: Introducing SplatSwap and Walk Mode

A Discover page and more comes to StorySplat!

Michael Rubloff

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff

Jan 6, 2025

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff

Jan 6, 2025

News

NVIDIA Announces Foundational World Model, Cosmos

Generative “World Foundation Models” Complement NVIDIA’s Three-Computer Solution for AVs and Robots, Turning Thousands of Human-Driven Miles into Billions of Virtual Test Miles

Michael Rubloff