Esri's Arkadiusz Szadkowski on Splats at National Scale, Formats, and Finding the Early Adopters

Michael Rubloff: Welcome to Esri's User Conference. I'm sitting down today with Arek, and I'm really excited to talk about some of the things happening here at the conference. Arek, could you start by introducing yourself?

Arkadiusz Szadkowski: Sure. It's good to have you here, Michael. Thanks for coming to San Diego. It's a really good place for you to experience Esri, what we do, and the scale of the GIS community, so I'm really happy that you're here.

Hello, everybody. My name is Arkadiusz, Arek for short, and I work at Esri in the global business development division. I'm responsible for reality mapping business development and sales globally, working around the world with all of our distributors on adopting and advancing this technology. Gaussian splatting has become the newest layer, as an output of our reality mapping for GIS.

Rubloff: You've been working in this field for nearly two decades now, and I want to ask you about how the transformative technology that has moved into the world of ArcGIS over the last year or so has started to transform your business.

Szadkowski: This is a very, very deep question — I could talk about it for a long time. In my earlier career, even back in technical high school, where I studied surveying, and later at university, I was always looking toward new, modern ways of mapping. When I was given a pen and paper in technical high school 30 years ago and told to map manually, I said, "No, no, I'm going to do this digitally," and I made my first map with CAD software. Then I discovered photogrammetry, and that was when I understood that we could actually use imagery to create maps at scale and capture the world as it is.

That immediately caught my attention. I thought: this is the way. This is how we will scale and build maps in the future. And fast forward, I'm fortunate to be at the forefront of this adoption, at a GIS company that touches millions of users.

I always like to say to anyone I meet, and to my team, that we are on a mission. We are really on a mission to bridge the GIS world — which has been extremely successful for decades at building complex maps at the crossroads of geography, cartography, and mapping — with the very rich and emerging geospatial world, which today is capable of reconstructing and capturing the world at extremely high fidelity.

We are trying to build that bridge, to bring this technology closer to the large community of GIS users, because we really believe it can improve the way they work across multiple sectors, domains, and use cases.

Reality mapping is still a very fresh technology, and it's really not GIS. Esri invested in imagery, I think, 10 years ago — I'm sorry, I would have to double-check. In reality technology, we started developing and investing around five or six years ago. So it's still very young compared to the Esri tradition. But it's emerging, it's catching on, and it's getting traction and momentum at Esri.

I'm very happy to be part of this, because if we manage to build this bridge and convince even maybe 20% of GIS users, that would be a million users for the reality map providers and for Gaussian splatting. That would make a huge impact on the GIS and geospatial world.

Rubloff: When we look at Gaussian splatting, we're now enabling people to reconstruct very lifelike 3D from the same input they're already capturing out in the world. They're able to model things like structures, vegetation, foliage, and reflective surfaces in a way that really looks true to life. Esri has done a great job of adopting this technology across a variety of its core offerings. When Esri first saw this capability of reconstructing the world around us, was it an all-in bet, in terms of saying, let's ship this to our core platforms?

Szadkowski: I can tell you from my perspective, my personal story. I discovered Gaussian splats for the first time around two — or now it would be three — years ago, in the summertime. Inria was, I think, the first engine I found, and later Luma came out with Gaussian splats.

I started using it on my holidays, capturing videos and images and reconstructing the heritage sites we were visiting with the family. Immediately I thought, wow, this is a much more realistic rendering. I sent it to Konrad Wenzel and said, "Guys, this is something you need to look into as soon as possible."

It took a little while before they picked it up, but once they did and decided to bet on it, we were very quickly able to run it with our engine. You know better than your audience and the people who were living this: in practice, it's the same structure-from-motion principle behind it, compared to all the reality capture capabilities that we already had.

So from the outside, for us it's the same imagery input, the same processing engine, and we're just outputting another layer, another representation, out of the computation. Of course, there are some defaults regarding overlap and how you should capture the data for the best effect. But right now, for us, it's just another layer — and not just an additional layer, but a photorealistic one.

To answer your question fully: my gut feeling immediately was, no, this is something new, this is a game changer. When I saw it, I immediately saw that it had huge potential to solve two problems that meshes have — and you mentioned both of them.

Thin structure reconstruction. That's where lidar has been, and still is, superior as an active measurement tool. But Gaussian splats, while still being a passive sensor — we depend on reflected light as an imaging sensor — are able to beautifully reconstruct all the thin structures. That opens a huge opportunity for the whole infrastructure sector, where you deal with assets that are complicated: often power lines, or all the small telecom towers, and so on. They have very complex geometry.

And then reflective surfaces. We have a lot of reflective surfaces — water, and all the other objects. So I immediately thought that this would add a lot of value to geospatial and GIS users.

You know better than me, but I saw the first Gaussian splatting papers three or four years ago, and the first real GitHub testing around two and a half years ago.

Rubloff: Right.

Szadkowski: And we released production-ready Gaussian splats at scale last year. I think we wrapped it very fast.

Rubloff: Especially at the scale at which you've shipped it, because you're not looking at it just at object scale, or even room scale. You're doing it at national scale, or city size and above, where governments are actually implementing the technology.

Szadkowski: That's true, and it's a little bit because of the nature of our current customers. In imagery and reality, a lot of our customers are national, state, and local government. Those are huge projects, national programs — countries buying coverage of an entire country, the full mesh or imagery only. So we immediately knew that we wanted to address this kind of customer base and take this not only to high detail but also to high scale.

Rubloff: A lot of people say, "Okay, it looks great, but what can we actually analyze with this? What additional capabilities can be enabled using this technology?"

Szadkowski: This is a good question, because from day one we have been thinking that this is more than just a render, more than just visualization. In all of our go-to-market for reality modeling, we position ourselves — because we know this is the biggest strength of our platform and our GIS — to offer the full workflow: from data capture, integrated with the capture workflows, all the way through visualization, data management, and complex analysis and simulation.

So we immediately thought about how we could put this into a workflow that supports decision makers, or situational awareness people, who want not only to look at it but also to analyze it and run simulation scenarios on that data.

Right now we're already able to perform line of sight and viewshed. You can measure, you can snap, you can annotate, you can segment, and you can highlight parts of the splats. In our Scene Viewer and in ArcGIS Online, you can do almost the same things you can do with point clouds. In fact, as you know, Gaussian splats are more or less points represented as ellipsoid primitives, so everything you could do with a point cloud you can do with Gaussian splats. Not yet classification, like ground and vegetation and so on — that is coming, and I know it's being tested.

We haven't shown this that much yet, but our professional services teams are working with a couple of customers on applying geospatial AI on top of Gaussian splats, taking advantage of the texture information that is encoded in this representation. I think this has huge potential for us to explore and learn from.

Rubloff: I completely agree, and I'm excited to talk in a little bit about the geospatial AI applications too. One thing I'd like to touch on is integrations, and creating an ecosystem. It seems that everyone is in agreement that this technology will be quite impactful, and that being able to have interoperable file formats and exchange data with one another matters. Esri has also been a big contributor to the Khronos Group with the glTF standards. Could you talk a little about why it's important for us to create a strong foundation for this?

Szadkowski: It's extremely important — another good question, Michael. You're well prepared.

I can use the example of 3D mesh, going backwards. When 3D mesh was also at the stage where Gaussian splats are today, still very new and very emerging, we were talking to some of the customers and to the market. Some of them wanted to standardize around a format. And a lot of customers, especially mapping authorities and so on, want to be as neutral as possible. They want to be vendor agnostic; they try to operate so that they can invite as many bidders as possible to a project.

Unfortunately, many of them ended up ordering — or are still ordering — 3D meshes as OBJ files. I have nothing against OBJ files; it's a good format. But the moment you are delivering huge, large-scale 3D mesh models in OBJ format, you are introducing challenges around scalability, compression, level of detail, and even georeferencing.

So thinking about the right file format from the beginning secures a few things for you: that you can cooperate with multiple actors, without locking anyone out, so many stakeholders can contribute to products and to adoption of the technology; and also that it will perform well on consumer-based computers. If we are addressing millions of GIS users, it has to run on consumer hardware. And it has to be well compressed, so that you will not have to pay a lot of money for storing and streaming this data.

This initiative — I was not involved there. I was consulted and part of the conversation, but I wasn't involved. I think it is really, really important, because if the big players, who I call frenemies — we cooperate, but we also compete in some areas for the same user base — are agreeing that this is the best file format to go forward with, it will only help adoption, and it will be very beneficial for the end user. Which is the most important thing: that from day one they are able to run projects and do their procurement processes with the correct file format, so they don't burn their fingers on the first one.

Rubloff: I think being able to really set strong expectations, and to meet those expectations too, is going to be critical to letting this technology grow to the scale that we're all hoping it can reach. To that same point, who are some of the early adopters of the Gaussian splatting layers within ArcGIS?

Szadkowski: Here I have to mention the limitations of Gaussian splats, and where they actually bring value. Today I will have a presentation here at the User Conference about best practices. One of the opening slides highlights the different resolutions and scales of the imaging and reality modeling technologies. There, I put a box showing that Gaussian splats deliver the most value when you capture at a resolution of 7.5 cm or better. We need a higher resolution. That is what I personally see as the sweet spot, the resolution where you have a higher signal compared to the noise.

Of course, you can attempt Gaussian splats at 10 cm or 15 cm resolution, but then, in my opinion, you get splats that are a little too big and too noisy, and then there is a high uncertainty that your measurements and your precision are not where you want them to be.

So once you look at the entire market of who operates at resolutions of, say, five or six centimeters and finer, you immediately know that this is heavy infrastructure, and sometimes state and local government — the cities.

What we see is that the biggest traction right now is within the drone industry. Everybody who is flying with a drone — their customers are usually AEC companies, so architecture, engineering, and construction, and infrastructure companies, so power lines and substations. Any type of linear infrastructure, so pipes and the power lines I mentioned already. Mining as well, because in open mines there isn't that much vegetation, so you can do that.

This is where we get the most conversations and where we had the first customers. You talked earlier today with Andrew from ES2 — he was one of the early adopters of Gaussian splats. He's already producing them and introducing them to customers. And we also see aerial survey companies; some of our aerial survey customers are starting to produce these layers.

And here is where we are right now with the technology: when you are introducing that kind of modern layer to the geospatial stack, you are not hitting a wall, but you are meeting the market a little bit where it is. Most of the projects that are procured by the cities or the infrastructure companies are not yet aware of the potential. So the data is being delivered as an extra add-on by these users and customers, just to build trust and convince the end user that this is actually worth ordering as an extra in the next project.

Rubloff: Right.

Szadkowski: So right now we are planting the seeds and nurturing the market with this technology, so that the end user will start trusting it and ordering it as a layer. But to come back to your question: I see the biggest traction right now in the infrastructure sector, utilities, AEC, and transportation, and some cities. A lot of governments as well are picking this up.

Rubloff: When we look toward the future, and the technology of Gaussian splatting and geospatial AI really start to coalesce, what do you think the next era of mapping might look like?

Szadkowski: I have to walk the talk, so in some of my slides and presentations I'm always talking about a future of mapping that will be more automated, where we shorten the time from data capture to delivery.

In my previous job, we were doing a lot of mapping projects based on imagery or point clouds and so on. And I always saw that the customer was issuing the tender and the procurement project in January or February, and by March it was decided who won the project. We were capturing the data somewhere in the spring and the summer, but then the whole mapping production was very manual — there was a lot of manual work in the extraction of the buildings, the roads, and so on. So the customer was getting the delivered project somewhere by Christmas. It was almost a year from the date of the procurement decision, "I want to have an updated map," to the moment it was delivered.

Today's mapping capabilities — not only the splats, but also SAM, photogrammetry, and the meshes — are introducing huge potential to shorten the time from the data capture to the information. Because those maps that we were actually delivering were used for obtaining some information for the user.

So splats are improving our value proposition to these users who are after the information, or that very popular word, insights. If we apply geospatial AI, if we apply some automation and some segmentation, a lot of the information that you normally were mapping manually can be obtained within a day or two, or maybe a few days, after you process the data — if you set up a good, efficient workflow.

So time to market, time to product, will be significantly shorter. And in my conversations with the end users and the customers, I always say, "Hey guys, if we manage to shorten this gap, this time to market, together, then you are introducing something that is very important: a temporal resolution, a frequency, that is much higher."

And if we are able to produce a map at a higher frequency, then suddenly we discover that the same type of data has many more use cases. And then, hopefully, that improves and increases the demand for more data.

Rubloff: The example that you showed earlier in the year, of being able to count the amount of trees in a city area, was astounding. Not only because it was able to successfully do it, but because, I think you were saying, the amount of time it would take to actually identify all of this was shockingly low.

Szadkowski: Yeah, this is a good tree example. It took six minutes to map 7,000 trees within part of a city. Whoever has done mapping anywhere, at any time, knows how long it would take to map 7,000 trees — not only the position, but the canopy size and the height.

Rubloff: I think this is the beginning of being able to utilize geospatial AI out in the world, to either help people make better decisions about the physical world or, hopefully, benefit the citizens of the places that are actually being reconstructed.

Szadkowski: With geospatial AI there is a very important thing I want to mention, because we cannot expect geospatial AI to be 100% accurate from day one. If you hired a young engineer from university today and asked him to make a map, he wouldn't be 100% accurate either.

Rubloff: Right.

Szadkowski: He needs some training, he needs some feedback, he needs some iteration. And geospatial AI would be the same. You need to deploy your model, you need to run it on a small subsample of the data, and you need to provide it feedback: this looks good, this does not. We have to calibrate the model a little bit on your dataset. That's very important for expectation alignment when working with geospatial AI. And being aware of your biases is a very, very important conversation.

Rubloff: I totally agree, and I'm looking forward to the future where it really becomes fine-tuned. Going back toward today, and your session: what are some of the things that you're hoping people come out of it learning?

Szadkowski: What we are hoping for, mainly, is to find more early adopters. Someone who's willing to give it a try, and to support them on the journey. Because those kinds of early adopters become a lighthouse later, and then the word of mouth starts to work: "Hey, I actually did a Gaussian splat delivery, and I won a new project because of it." This has huge potential for becoming a little more unique on the market, in my opinion.

Rubloff: Yeah.

Szadkowski: There is a nice quote I heard recently: advertising is the cost you pay if your products or services are not unique.

So in my opinion, I really hope that there will be more and more users helping us to influence the market, and to influence the end user base, through the regulation of the market. Because at the end of the day, most of the projects out there that are being performed by service providers are tender-based or project-based. So we need to convince the end users to start ordering more of those types of layers. And to convince them, we need the success stories. To have the success stories, we need earlier adopters who invest their time and their energy and give it a try. So my biggest hope right now is that we will find more and more of them. We already have a lot of them, we already have a lot of people using this, but I hope for more.

Rubloff: And if you are an early adopter and you want to give it a try, where can people go to find it?

Szadkowski: For Esri: if you are a US-based organization or customer, reach out to your account manager and ask them for an evaluation of the reality mapping user type extension, or ArcGIS Reality Studio, or Site Scan. Those are the applications that are in the reality mapping user type extension. You have Drone2Map, and you have ArcGIS Reality for ArcGIS Pro. All of those applications today can produce Gaussian splats. If you are outside the US, go to your distributor, and through the distributor you can request an evaluation license from us. We'll be happy to provide the evaluation and support you in processing your first splats as soon as possible.

We also have a lot of learning materials online already. If you look up reality mapping or ArcGIS Reality, you will for sure find sample data as well. If you cannot collect your own data, you can download some of our example data and use that.

Rubloff: Amazing. Well, thank you so much, Arek, for sitting down with me. I'm really excited to be here talking about the world of ArcGIS and Gaussian splatting.

Szadkowski: Thank you, Michael. Thank you.