What World Labs has been building for the last year has been a big secret. Even with the occasional new story about their funding leaking, it was never clear what exactly the San Francisco based company was building. This has now changed with World Labs publishing a blog, complete with videos and interactive demos of their offering.
The core of World Labs' announcement is a generative AI system that allows users to transform a static, two-dimensional image into an explorable 3D space. This means that users can "step into" an image, viewing it as a fully realized environment with depth, solidity, and a sense of presence—characteristics often lacking in purely 2D or animated representations. This technology aims to introduce more control and consistency in the world of content creation, providing creators with new tools to manipulate space and enhance storytelling.
The 3D scenes are generated through a process that infers and constructs depth, geometry, and other physical properties from the input image. This capability introduces a layer of interactivity that traditional generative AI cannot currently provide—users can navigate the virtual space, inspect details from different angles, and even interact with elements of the scene. World Labs also highlights that generated scenes maintain their form consistently, meaning once a scene is created, it remains persistent and does not alter unpredictably, unlike some AI-generated videos.
The platform leverages familiar gaming controls to explore these virtual worlds, allowing users to move around using the WASD keys and look around by dragging the mouse. Additionally, World Labs demonstrates the versatility of their generated scenes through various effects, including camera effects like dolly zooms and artistic depth of field simulations. These effects are rendered live in the browser, showcasing the potential for real-time creative adjustments.
World Labs also emphasizes the broader implications of this technology in creative industries. By providing more than just 2D outputs, the AI-generated 3D worlds have the potential to reshape fields such as film, game design, and other creative digital arts. Artists could utilize this system to turn traditional paintings into explorable 3D environments, offering a fresh perspective on iconic works by artists like Vincent van Gogh or Edward Hopper.
Playing around with a couple of the demos, it looks like you can occasionally glimpse some gaussians in the scene, but there's no word yet on what's powering this implementation. It should be well documented the deep and intense Radiance Field experience the co-founding team World Labs has. Some of the more relevant work here would be Reconfusion, of which shares co-founder Ben Mildenhall as an author.
The integration with existing generative models is another intriguing aspect of the system. By working in tandem with popular text-to-image models, World Labs can create diverse versions of the same scene while preserving its 3D integrity. This feature presents an opportunity for artists and content creators to adapt their styles and workflows, combining familiar tools with new 3D capabilities to enrich storytelling and create interactive experiences.
Currently, the system is in an early preview stage, but World Labs has already invited creators to experiment with this technology. Early users have combined it with tools like Midjourney, Runway, and Blender to craft dynamic scenes, short films, and creative narratives that previously would have required significant resources and technical skills to achieve. Moving forward, World Labs aims to enhance the fidelity and detail of their generated worlds and explore new interaction methods for users.
As the boundaries between 2D and 3D content creation blur, World Labs is positioning itself as a key player in developing tools that empower creators with more intuitive and interactive digital environments. This technology not only promises to improve consistency and control in the generative content landscape but also to provide a pathway towards new creative workflows where images can be brought to life in 3D, redefining the way we create and explore digital worlds.
It has not been officially confirmed that WorldLabs is using radiance field based methods, but all signs seem to point to them being included in the pipeline. As more information about the company releases, we will continue to cover it.
