Preferred Networks, Inc. (PFN) has announced a new Unreal Engine plugin designed to bring photorealistic 3D Gaussian Splatting (3DGS) directly into the hands of video producers and game developers. The plugin, developed in-house and currently in preview, but is another exciting way to bring the radiance field representation into professional workflows. 

Virtual production has already revolutionized film and TV workflows by allowing directors to film in front of massive LED walls displaying digital backdrops instead of building physical sets. PFN’s new system takes this one step further by eliminating the need for complex modeling pipelines entirely.

With PFN’s plugin, real-world locations, from heritage buildings to city streets to cozy interiors, can be captured, reconstructed, and rendered inside Unreal Engine with minimal manual effort. This lets production teams:

• Visualize final shots on set, adjusting lighting and camera movement in real time

• Cut down on post-production, since visuals are ready straight out of the camera

• Reduce costs by avoiding expensive physical set construction

Demo footage provided by PFN shows just how seamless this process is. The camera moves naturally through reconstructed environments, with parallax, lighting, and detail comparable to live-action footage.

PFN’s plugin can handle scenes with over 10 million Gaussian splats at 4K resolution, while still exceeding 60 frames per second with less than two frames of latency. According to PFN, this may be the fastest practical 3DGS renderer for game engines in the world.

Some additional features the plugin supports include:

• Directional and point relighting

• Adjustable depth of field

• Object and scene component removal or translation

• Combining 3DGS with polygon meshes

• Shadow casting and receiving

The plugin comes equipped with a wide range of features designed to streamline workflows and enhance creative flexibility. It supports drag-and-drop importing of 3DGS scenes directly into Unreal Engine. 

In addition, PFN has added editing tools that allow users to rearrange 3DGS scenes, combine them with traditional polygon meshes, and build hybrid environments within the UE editor. Support for Unity is also under consideration.

The plugin will be compatible with Unreal Engine 5.2 and 5.3, while the release date and pricing information are currently unknown. However, it does support input of standard trained .ply files, meaning users won’t be required to train scenes using PFN’s pipeline.

The examples shown in the demo video were both captured and reconstructed by PFN’s in-house team. Their training pipeline is similarly proprietary and built from the ground up. There is no word yet on whether their reconstruction engine will be made available for public use.

We’ll continue to keep you posted as more information becomes available, but this is shaping up to be an exciting new plugin to watch! More information about Preferred Networks can be found on their website!