Gaussian Splatting support has officially landed within VRChat. Based on an open source implementation from developer Mykhailo Moroz, real time radiance field rendering has been brought into the social VR universe for free!
The system works through a custom Unity package, which includes a full importer for .ply splat files, an adjustable splat renderer, and a surprisingly fast radix sort implementation built entirely within the constraints of the VRChat platform.
The splats are rendered as quad billboards, but thanks to a custom ellipsoid to ellipse projector, the results are what you would expect from the lifelike nature of radiance fields. This is mainly due to performance reasons, but the visual fidelity was comparable even when using perspective correct splatting.
Users can tune some of the parameters to find the right balance between visual quality and performance. The package includes specific tips for optimizing texture sizes and turning off MSAA, which tends to tank frame rates without any benefit to the splat visuals. As with many of the Unity 3DGS projects, the .ply importer is based off of Aras P’s repository. The radix sort itself leans on d4rkpl4y3r’s mipmap based prefix sum trick.
Support for animated splats, spherical harmonics, and better compression is on the roadmap. But even in its current state, this release is an exciting moment. It opens the door for immersive radiance field reconstructions inside one of the most popular social VR platforms in the world.
The repository also launches with 12 sample scenes to toggle between, seven of which I have provided to Moroz! You can grab the latest release or clone the MIT Licensed repo directly on GitHub.
