OpenAI Launches Sora and the World

Michael Rubloff

Michael Rubloff

Feb 15, 2024

Email
Copy Link
Twitter
Linkedin
Reddit
Whatsapp
Sora
Sora

Just earlier today OpenAI unveiled Sora, arguably the most consequential announcement in the past twelve months

Sora is a text to video model announced just after midday on the East coast. To be blunt, it's significantly stronger than any existing public method, surpassing others by orders of magnitude.

That's a large claim, given the progression of technology. On its face, generating that level of high fidelity of video is extremely impressive, but where it becomes incredible is the generating of view consistent, parallax filled large areas. This unlocks the ability of large scale generative radiance fields.

What does this mean? It signifies that generating high-fidelity, large-scale radiance fields will soon be a reality for the public. The most challenging part is now behind us. It is now possible to generate view consistent locations and lifelike video output through text. That is so difficult. That is so completely difficult.

Like the vast majority of people, I only have access to the publicly available videos they've published, but that's all I need. According to the Sora announcement, they're able to generate 60 seconds of footage per prompt. I took the demo video of Santorini as my base. It's only 9 seconds, but that's sufficient.

From here, I simply slice the video up the same way I would with any existing video and run it through COLMAP and then into nerfstudio, but you can use any radiance field company. It works on the first try, with all camera poses found and trains like any other radiance field. Here is the resulting NeRF from Nerfacto.

This was taken from a 9 second video. Imagine what you could do with the full 60 seconds. Imagine what you can do with two minutes. This means, not only is it possible, they've already achieved it. I am very curious to see what OpenAI will do from here on out, but as this stands, they can generate large scale radiance fields from a text prompt. I hadn't anticipated seeing large-scale generative radiance fields until much later.

Further, there is absolutely zero reason why this cannot be automated from here on out; in fact it would be trivial. When Sora is unleashed to the world at large, it will allow for the rapid creation of hyper-realistic three dimensional worlds.

Before my thoughts around radiance fields were limited to capturing the existing physical world and generating smaller objects, but what this means expands possibilities to an almost unfathomable level. You can generate hyperrealistic large scale three dimensional radiance field worlds based off of text.

This also has significant implications for existing radiance field companies with generative capabilities, such as Luma and CSM. Given this is now proven to be possible, I can only wonder what's already been created behind closed doors. If they have not been pursuing this already, I imagine they now are.

Featured

Recents

Featured

Platforms

Into the Scaniverse: Scaniverse in VR

Scaniverse is now available on both the Quest 3 and 3S.

Michael Rubloff

Dec 10, 2024

Platforms

Into the Scaniverse: Scaniverse in VR

Scaniverse is now available on both the Quest 3 and 3S.

Michael Rubloff

Dec 10, 2024

Platforms

Into the Scaniverse: Scaniverse in VR

Scaniverse is now available on both the Quest 3 and 3S.

Michael Rubloff

Platforms

Reflct: Simple 3DGS Viewer for Ecommerce

The clean 3DGS viewer for Ecommerce with SH support has begun its closed beta.

Michael Rubloff

Dec 9, 2024

Platforms

Reflct: Simple 3DGS Viewer for Ecommerce

The clean 3DGS viewer for Ecommerce with SH support has begun its closed beta.

Michael Rubloff

Dec 9, 2024

Platforms

Reflct: Simple 3DGS Viewer for Ecommerce

The clean 3DGS viewer for Ecommerce with SH support has begun its closed beta.

Michael Rubloff

Research

3D Convex Splatting Radiance Field Rendering

3D Convex Splatting is another new way to render a Radiance Field. It is not Gaussian Splatting.

Michael Rubloff

Dec 6, 2024

Research

3D Convex Splatting Radiance Field Rendering

3D Convex Splatting is another new way to render a Radiance Field. It is not Gaussian Splatting.

Michael Rubloff

Dec 6, 2024

Research

3D Convex Splatting Radiance Field Rendering

3D Convex Splatting is another new way to render a Radiance Field. It is not Gaussian Splatting.

Michael Rubloff

Platforms

Frame Integrates Gaussian Splatting

Powered by Babylon.js, Frame now supports 3DGS imports.

Michael Rubloff

Dec 5, 2024

Platforms

Frame Integrates Gaussian Splatting

Powered by Babylon.js, Frame now supports 3DGS imports.

Michael Rubloff

Dec 5, 2024

Platforms

Frame Integrates Gaussian Splatting

Powered by Babylon.js, Frame now supports 3DGS imports.

Michael Rubloff