Quietly over the Thanksgiving weekend, a PR was merged into the Unity repository by Ninjamode, which brings VR access to 3DGS Unity projects using Aras’s plugin. If you haven’t tried viewing a Radiance Field in VR, I highly suggest it as the view-dependent effects carry over beautifully into VR.
Aras’s plugin has widely been declared as the de facto option for 3DGS in Unity. While Aras is no longer actively developing the plugin, PRs are welcome to the MIT-licensed codebase.
Implementation Instructions
Getting the VR capabilities up and running is not as difficult as you might imagine. Follow these steps once you have added the Unity Plugin:
Use Unity’s Universal Render Pipeline (URP) for optimal results. The Standard Pipeline may encounter matrix-related issues.
Enable XR Plugin Management.
Start OpenXR on initialization.
Set the Render Mode to Multi-Pass.
Some of the under the hood updates include a built-in matrix integration. The new implementation replaces custom P and VP matrix usage with Unity’s built-in equivalents, streamlining compatibility and reducing custom code dependencies.
It also includes improved GPU support by incorporating a newer version of DeviceRadixSort, offering enhanced GPU compatibility, especially tailored for devices like the Meta Quest 3. This resolves prior inconsistencies noted in Android-based devices, referencing b0nes164/GPUSorting#4.
The system now uses eye texture size in screen size calculations when available, optimizing the visual fidelity and performance of VR rendering setups.
While the update delivers significant improvements, it notably does not include sorting optimizations for eye center alignment. The contributor cited the lack of robust implementation as the reason for exclusion but suggested it could be a worthwhile optimization to revisit, particularly for Desktop VR scenarios. A sample implementation is available in the originating repository for developers interested in exploring this further.
The updated system has undergone testing across multiple platforms, including Desktop Steam VR (HTC Vive), Varjo devices (tested on Aero), Meta Quest 3 and Quest Pro (via Link and standalone). Notably excepted is the Apple Vision Pro. However, people with the Vision Pro can use platforms such as MetalSplatter.
This move also triggered Aras to push the 1.0 update to the repository, though the updates are not as large as the “1.0” version number might imply. The repository remains MIT-licensed for those interested in contributing to the project.
As always, collaborative testing and feedback are key to ensuring robust solutions. If you have a VR setup, testing this feature could help refine and validate the implementation for the broader Unity community.
New to Gaussian Splatting? Start here
