StorySplat. Bringing 3DGS into Educational Experiences

One of the use cases of Radiance Fields that I’m particularly excited about is the ability to tell stories and more immersive educational experiences. With this in mind, comes the aptly named StorySplat. 

Originally developed by Sonny Cirasuolo for students at the S.I. Newhouse School of Public Communications at Syracuse University (my alma mater), StorySplat was intended to commemorate the 200th anniversary of the Erie Canal. What started as a proof of concept, whipped up over the course of a night, quickly grew into something more, with Sonny adding layers of functionality and features.

Storytellers/journalists can upload their splats, create a scroll-based storytelling fly-through with custom waypoint interactions and hotspots — and export it as a self-contained HTML file that can be hosted and embedded in stories/articles. Includes spatial audio support and much more. Built with Babylon and React.

All without writing any code.

This project resonates with me not only because of my Newhouse connection, but because it stands as a direct example to how journalism can evolve through creative uses of immersive and hyper real 3D. 

With the release of Version 1.0 on 10/23/2024, StorySplat introduced a host of new features:

• New collision system for the camera: Allows for more intuitive movement and interactions.

• Restricted camera modes on export: Ensures cleaner and more focused outputs.

• Auto-play mode: Lets the camera follow a defined path at a steady speed.

• Loop mode: Seamlessly connects the last waypoint to the first, allowing continuous navigation.

• cors-anywhere options: Enhances flexibility in handling cross-origin resource sharing.

• Improved mobile controls: Includes joystick support for a smoother mobile experience.

• Additional tweaks: Such as requesting XR support on export.

While these features are impressive, Sonny has no intention of slowing down. The roadmap for Story Splat continues to grow, with some exciting updates on the horizon:

Upcoming Features:

• load multiple splats into the same scene, custom loading logic for dynamically loading in new splats

• replace hotspot sphere mesh with custom mesh

• Waypoint pop-up UI Options

  • Add info to the scroll bar UI at the bottom of the screen instead of being a centered pop up

  • Remove close button

  • Customizable UI background color

• Edit waypoint name, add this to the scroll bar UI when jumping between waypoints

• Implement Loader Bar

• Collisions V2 - Upload your own collision mesh, add more shapes than planes.

• Make the auto-framing functionality on splat upload toggleable

Some people have been quick to load in their captures and create guided experiences for people. These include an overview of the Upper Gallery within the National Gallery, information about Eleanor Roosevelt, and the US Capitol. Here’s one I made about the Ferrari Roma! For those interested in learning more about Sonny and his journey leading up to StorySplat, you can check out this article about his background.