MultiSet AI: Gaussian Splats as Native VPS Map Input

Last week coming back from conferences, I found myself standing on an arrivals platform trying to get an Uber. I was standing underneath a concrete column, and my phone’s location kept updating. The dot slid across traffic and throughout the airport terminal. Despite this, the driver found me, but I spent my entire wait wondering whether I was standing where my phone said I was.

While this is not a critical situation, it is also something I have generally accepted that can’t change. It seems like a technological shortcoming that we just need to accept.

However, in reality it is a specific, well understood failure with a similarly specific, well understood fix. It's called a Visual Positioning System, or VPS.

A VPS uses normal cameras, like your phone or smart glasses and compares what that camera sees against a map of the place built ahead of time. From the match it works out where you're standing and which direction you're pointed within a few centimeters, even when you’re indoors. 

There are so many quality of life improvements to everyday life that I can think of off the bat. Finding your car in a garage. Finding gate B14, or the one store in the mall, or the right aisle and then the right shelf. Walking directions could put an arrow on the actual door instead of telling you to head northeast for 200 feet.

However, the industrial version receives the same benefits. A robot can spatially anchor itself for Physical AI applications. Similarly, technicians walk to the specific valve instead of the general vicinity. VPS can only place you inside a map that already exists. Somebody has to capture the space first. For teams working in Physical AI or industrial settings, this can be done in an afternoon. 

Whereas GPS is a timing system. Your phone listens for satellites and works backward from how long each signal took to arrive, and that math assumes the signal came straight down. Under open sky it mostly does. But put a building next to it or go indoors and the assumption breaks. 

It's why my location at the airport kept changing. Walk fully indoors and the satellites drop out entirely, your phone starts guessing from Wi-Fi and cell towers, and the error goes from meters to whatever the nearest access point happens to be.

In a similar vein, photogrammetry was just how you captured a place for decades. Then splatting showed up and people started noticing the same capture data could produce something substantially better. The old default stayed the default, not because anybody compared the two and picked it, but because it was already installed and already what everyone knew.

GPS is in that stretch right now.

Which brings me to MultiSet AI, which not only provides VPS, but takes gaussian splats as a native input. The maps we use are more precise and they can also be lifelike 3D. When gaussian splatting is fed to a VPS it becomes a mapping device. Reconstructions that render a photoreal view of a building turns into the reference a headset or a robot uses to work out where it's standing or where it needs to go.

Using a metric scaled .ply, the MultiSet platform turns it into a map you can query for position. MultiSet reports 5 cm median accuracy from splats. Back at AWE, I got to try it for myself and MultiSet was able to locate me inside of the exhibition hall in about one and a half seconds. 

However, what I think is even cooler is that they ingest E57, point clouds, textured meshes, 360 video, and splats and can reconcile them onto one coordinate system. MultiSet stitches individual maps across floors and buildings so a device can cross between building floors without re-localizing at every doorway.

What stuck with me is that there is only one map. The robot and I are not reading different files. A robot or a drone gets it through an SDK call and receives a pose back, six degrees of freedom, in a coordinate frame it can act in. I get the same map through a phone or a pair of glasses, and the pose becomes an arrow on the actual door instead of a heading. Open it in a browser and it is a gaussian splat you can fly through, or a textured mesh if that is what your pipeline wants. Pull it through the API and it is just data you can query. One capture, four read paths. That is the difference between an asset and infrastructure. Part of why photogrammetry felt finished for so long is that its output only ever had one read path, which was looking at it.

There is so much information about the physical world that has been captured, and yet it feels painfully obvious that we have not yet leveraged its full potential. While Physical AI and industrial applications will be the first beneficiaries of VPS, it’s exciting to imagine how wayfinding and navigation will also impact our everyday lives.

Learn more about MultiSet here.