Sutro Tower 3DGS Reconstruction

Sutro Tower is one of San Francisco’s most recognizable landmarks, visible from almost anywhere in the city. Yet, for most people, it remains remote. But, thanks to Vincent Woo, the tower is now within reach, with a lifelike digital reconstruction.

Woo’s process began with flying a drone around the tower, capturing over 2,800 images with his friend, Daylen Yang. An early challenge was color matching across shots— an issue amplified by the use of four different capture devices at different times of day.

To align the frames, Woo manually processed the images in RealityCapture. Once aligned, he imported the dataset into Nerfstudio for the final reconstruction. This is where one of Nerfstudio’s most powerful features came into play: Bilateral Guided Radiance Field Processing, which significantly improves consistency across varying exposures. Personally, I use Bilagrid on every capture I put into Nerfstudio.

Despite the complexity, the entire process ran on a single RTX 4090. After training the capture, Woo exported it using Self-Organizing Gaussians, Nerfstudio’s method for compressing Gaussian Splatting reconstructions. He then integrated the model into PlayCanvas, where he added interactive annotations with fun facts about the tower’s history.

This project stands as the latest and highest-fidelity example of how radiance fields can be used for educational experiences. 

For those who want an immersive experience, the capture can also be projected in augmented reality. Because it’s hosted on PlayCanvas, this should also be accessible in VR, too. Impressively, it runs smoothly on mobile devices, with the model size reduced to just 30MB. A year ago, a reconstruction of this scale would have been several gigabytes, but there has been so much compression progress.

Adding to the realism, Woo included a peaceful soundscape, further enhancing the feeling of being there.

As more people discover the power of Radiance Fields, we’ll see even more historical and cultural landmarks brought to life in 3D. Woo is excited about the ability of news teams to utilize Radiance Field representations to bring news stories to their audiences and he is correct.

The Weather Channel has been using Radiance Fields to power their FloodFX simulator, but I agree with Woo that all news stories can be made stronger by really bringing the news to people. We should begin to prepare for a world where 2D is no longer the dominant imaging medium on Earth, as we have the ability to explore and tell stories in 3D. The evolution of 2D imaging into the next phase of human history is already well underway. 

Experience the full capture here and follow Vincent on Twitter!