MetalSplatter for Apple Vision Pro

MetalSplatter is a completely free and open source Gaussian Splatting viewer and Swift/Metal library for the Apple Vision Pro. It allows you to experience your captures in full stereo within the Vision Pro. I have gotten significantly more distracted by this than I anticipated, with my plans Friday and yesterday virtually disintegrating.

For those that have gotten their hands on a Vision Pro, MetalSplatter is currently on the Vision App Store to view Gaussian Splatting files.

It's a very easy set up, as it only requires a .ply trained with Gaussian Splatting. If you have any captures on Luma, Polycam, or Kiri Engine, you can simply download the .ply unzip the file and open it in MetalSplatter. If you are looking for demo captures to try out, I have linked three of mine for you to download!

• Amsterdam

• Ferrari

• Watch

The MetalSplatter library facilitates the rendering of frames with efficiency and clarity. Alongside, PLYIO serves as a robust module for reading both binary and ASCII PLY files, encouraging experimentation with 3D models. SplatIO, another critical component, interprets these PLY files as sets of splats, further enhancing the library's versatility.

For those eager to dive into the world of 3D Gaussian Splatting, MetalSplatter includes a SampleApp and a SampleBoxRenderer. These tools, although minimal, are designed to demonstrate the library's capabilities and assist developers in integrating MetalSplatter into their projects.

Despite its early-stage development, MetalSplatter is a work in progress with a vision for significant enhancements. Future updates aim to address current limitations, such as color accuracy, memory optimization, and API documentation. The roadmap includes advanced features like spherical harmonics and improved scalability, ensuring that MetalSplatter remains at the forefront of 3D rendering technology. Right now only .PLYs are accepted, but the developer has .Splat coming soon. Additionally, I have crashed my Vision Pro several times loading .plys across all of the current viewers, so if it crashes, don't be overly concerned. It will turn back on roughly 30 seconds later.

There currently is not any documentation, but that said, I am immensely happier that it released first and find it easy enough to navigate around without it.

MetalSplatter draws inspiration from several key implementations and insightful blog posts within the field, acknowledging the contributions of pioneers like Kevin Kwok, Aras Pranckevičius, and the authors of the original paper.

If there is interest, I have a large library of captures (2K+) that people can try for themselves. If anyone has suggestions on how to easily open source a portion of them, please let me know!

Currently it is the only native viewer for Gaussian Splatting on the Vision Pro, but to my understanding several companies are aiming to change that by the end of this upcoming week. For more information or to fork the code, please visit MetalSplatter's GitHub repository.