Historical Radiance Fields and Museums

Throughout the twentieth century, the lens of photography and the frames of film have captured moments that define human history. From the stark realities of the Great Depression to the cinematic escapades that have captured our imaginations, these two-dimensional windows have shaped our understanding of the world.

As we progressed from the still image to the moving picture, each snapshot and scene brought us closer to the tapestry of human experience. Yet, there remained a boundary between viewer and subject, a flat plane upon which we could observe but not enter.

Still today, you would be hardpressed to go more than an hour without interacting with some form of imaging. In our quest for immersion, technology has relentlessly pushed the boundaries, seeking to break the plane and invite us into the picture. Now, we stand at the precipice of a new dimension. Radiance fields are the gateway to reliving history as if standing within it.

Radiance fields transform flat images into explorable three-dimensional spaces. Imagine stepping into a photograph, walking around within it, observing every angle as if it were real. This is the promise of radiance fields: full immersion in a previously flat world.

One crowning success of photography is the ability to document specific moments in time for infinite generations to view. The question becomes, how do you step that up a notch? Radiance fields appear to have the answer to that question going forwards, but can they process footage from a long time ago and allow us to become closer to historical events, long passed?

The answer is, yes. Here are two different examples of radiance fields recreating historical moments in time, long gone. The first, a NeRF through Luma AI and second through 3D Gaussian Splatting. This was prior to the Luma integrating Gaussian Splatting through their Interactive Scenes.

As we explore these recreated moments, we gain a new appreciation for the events that have shaped us. Amazingly, the example from Japan was pulled off an existing YouTube video. How much footage already exists that has the power to be transformed into a photorealistic 3D environment already? The answer to that question makes me excited to be able to go back and explore moments in time, forever immortalized, now in three dimensions.

Creative Director, Peter Eszenyi, has also been thinking about this. Peter has worked on films such as Blade Runner 2049, Ghost in the Shell, The Martian, Guardians of the Galaxy, and several others. He was kind enough to share several examples of historical footage that he has revisited, to create 3D content. I spoke with him briefly and we spoke about how much historical footage there is to be revisited and recreated for people to explore and learn from. For him, there is so much to learn from historical footage and bringing it into photorealistic three dimensions is a dream come true.

We also touched on the speed in which this is happening and how it allows him to dedicate more time to the creative process. With prior methods, when he would hit render, it would trigger a deadzone of literal days, depending the project he was working on. Now as the methods continue to get faster, especially with 3D Gaussian Splatting, the scenes are rendering significantly faster, affording Peter the ability to work closer to the deadline on the creative, before rendering.

A few months ago, I received a message from someone who, in partnership with the museum located there, had taken NeRFs of Auschwitz with the goal of providing educational resources about what occurred there. Not all moments that have been captured are meant to be lighthearted, but these are critical to be documented and provide educational resources. By allowing people who otherwise do not have the means to visit the places of tragedies ways to walk through, I believe that the world will become a more educated place.

The integration of NeRF technology, as demonstrated, represents a significant step forward in how we document, perceive, and interact with historical events. This application extends far beyond the confines of the digital realm, finding a natural progression into the world of museums, which serve as custodians of history and culture.

When museums go to display their collection, the vast majority never make it to the public eye. Elliot Bostwick Davis, former Department Head of the Art of the Americas Department at Museum of Fine Arts in Boston states: "Most museums show between 2 and 4 percent of a collection".

That number feels like sad hyperbole, but it's not. There are so many works of art that aren't being seen by the public and it's a shame. There's a wide variety of reasons why museums are unable to display full bodies of work ranging from available space to security, but all reasons net to art and history not being appreciated to their fullest.

By utilizing radiance fields, museums are able to offer a wide variety of benefits to their visitors, both in the museum and abroad. These include interactive and immersive experiences that offer photorealistic details for people located around the world to explore. Additionally, for museums that are worried about health interactions from touch screens, radiance fields present an exciting alternative to interactive materials.

Additionally, because this is in the cloud, museums neither need to worry about finding space on the floor nor hire additional security. By allowing a larger volume of work to be photorealistically displayed, it affords the very artists who are supposed to be highlighted that benefit. There are amazing pieces of art that are no longer able to be displayed because of light degradation. However, I argue that this is no less worthy of being seen and appreciated by as many people as possible. By utilizing radiance fields, these institutions are able to offer experiences that allow them to still show the piece, while preserving it.

This gives the artwork not shown on the museum floor to be discovered and if a piece becomes exceedingly popular, it can be moved to an exhibit and marketed for in person viewing.

Further, museums can leverage XR environments to provide educational materials on top of the radiance field. Let's take a look at how Paolo Tosolini has been using Luma for the Museum of Flight in Seattle. He's created a web app for museum attendees to learn about the various exhibits and key figures.

Museums are not the sole beneficiary of radiance fields. Historical sites, events, and products all merit amazing applications. As we reimagine the museum experience and beyond, radiance fields stand poised to democratize access to history, art, and culture. They promise a world where every moment can be preserved in its fullest dimension, inviting exploration and education in ways we've only begun to imagine.

This is the latest in the series of exploratory articles for how various industries will benefit from Radiance Fields. Prior articles have looked at Live Events, Fashion, and Sports. If you're interested in how your industry will benefit from radiance field's offering, feel free to reach out!