Back in late August, PICO XR announced the compatibility of Radiance Field method, Gaussian Splatting into their headsets. With the release of the Ultra 4 shortly after that announcement, they also discussed the eventual release of a Unreal Engine plugin and a dynamic Gaussian Splatting demo.
The first half of those releases is now here, with PICO officially releasing the beta version of their PICO Splat Plugin, a tool designed to bring 3D Gaussian Splatting (3DGS) to Unreal Engine projects on the PICO 4 Ultra. This is an exciting new additional to the world of bringing Radiance Fields into VR and more accessible to developers looking to leverage its capabilities in virtual reality environments. However, the plugin comes with notable limitations and a targeted focus, reflecting its beta status.
PICO Splat is a plugin for Unreal Engine 5.4 and 5.5 that allows developers to load and render 3DGS assets directly on PICO 4 Ultra headsets. With this release, PICO joins other major VR players like Meta, Varjo, Scaniverse, and startups like Gracia in embracing the Radiance Field method, Gaussian Splatting.
The plugin supports Unreal Engine 5.4 and 5.5 and requires Windows 11 as the host operating system. However, it remains exclusively tailored for the PICO 4 Ultra series, making this a focused solution for developers working within the PICO ecosystem.
As expected in a beta release, there are some limitations developers need to consider.
The plugin currently supports only .ply files, without support for the view dependent effects of spherical harmonics. Additionally other file types such as .splat and Scaniverse's .spz has not been supported yet.
Physics and collision systems are also absent, meaning 3DGS assets serve purely as visual elements without interactive properties. Similarly, assets are unaffected by light sources and cannot cast shadows. By default, the plugin uses very low-precision types to pack splat data, and there are no options available to adjust the quality settings. This might not satisfy developers looking for high-fidelity results.
Currently only a single 3DGS asset can be displayed in a scene at any given time, which may restrict more complex compositions. Furthermore, the plugin has not been tested on macOS or Linux, potentially leaving developers on those platforms in the dark regarding compatibility.
Lastly, rendering issues such as incorrect composition of transparency and fog effects may detract from visual quality, and the plugin is incompatible with mobile deferred rendering.
While these limitations may hinder its use in full-scale production, the release still provides a valuable starting point for exploring the integration of 3DGS in VR and will no doubt be upgraded over time!
Getting Started with PICO Splat
For those eager to experiment, PICO has outlined a straightforward workflow for integrating the plugin into Unreal Engine projects:
Enable the Plugin: Download and install the PICO Splat plugin from FAB. Enable it in Unreal Engine by navigating to the Plugins menu under the Rendering category.
Import 3DGS Assets: Use the Content Browser to import
.plyfiles by dragging them into the editor or using the import button.Add to Scene: Drag and drop imported 3DGS assets into your scene to create Splat Actors, which can be positioned and transformed like any other Unreal Engine actor.
Optimize Settings: Given the performance demands of 3DGS rendering, PICO recommends disabling mobile anti-aliasing, using half-precision float modes, and enabling foveated rendering to reduce GPU strain.
While the current iteration of the plugin is limited in scope, PICO has positioned this release as an important first step in exploring 3DGS rendering for VR. Notably, the beta includes support for dynamic Gaussian Splatting, which promises exciting applications such as lifelike animated avatars and fully explorable 3D videos. These advancements are expected to arrive in future updates.
For developers outside the U.S. where the PICO 4 Ultra is generally unavailable, the plugin’s open-source approach is particularly exciting. If you are based in the US, you can buy it through Newegg, but you will need to reinstall the OS to the UK's option.
Once fully released, it will allow for customization and adaptation across various use cases, potentially enabling broader adoption beyond the PICO ecosystem.
The release of the PICO Splat Plugin reflects the growing momentum behind Gaussian Splatting in VR. While it’s clear this beta is not yet ready for large-scale production, it provides a valuable opportunity for developers to experiment and contribute to the technology's evolution.
As PICO continues to refine the plugin and address its current limitations, the potential for 3DGS in VR has become increasingly clear.
