New Radiance Field Platform: Doly

Today there is a new entrant into the Radiance Field community with Doly. They have announced a $2M fundraising round led by Adjacent, who also helped lead Polycam’s Series A, with additional strategic investors such as Julien Chaumond of HuggingFace and Alban Denoyel of Sketchfab.

As founders in the AR industry, Pierre and Rémi have known each other for a while. The opportunity to work together came in 2022 after they had both sold their startups and left their acquirer. Pierre started to tinker on a very promising new approach to 3D, which has been the first emergence of AI in 3D reconstruction: a technology called NeRF pioneered by Google in 2020. He discusses his findings and research with Rémi and they started to iterate on actual immediate use cases of the technology.

Discussing with many merchants, they discovered their immediate need was not 3D models, but doing great videos and keep pace with the rhythm the social media platforms were demanding, especially after the emergence of short form video as a way to market physical products.

This led to the idea of Doly, an app that would use the state of the art of 3DAI research to create compelling videos, on demand. The company is based in the US and the team is remote-first, with a presence in France and Quebec.

I sat down with the founders of Doly, an ecommerce platform utilizing Gaussian Splatting to create high fidelity captures for users.

What is Doly?

Doly is a groundbreaking mobile app that empowers merchants and videographers to create great videos of their products for product pages and advertising, leveraging the latest advances in 3DAI. 

Doly’s mission is to empower small merchants through new imaging tools and prepare them for the spatial commerce era. Our motto is to “capture imagination”.

Radiance fields feel like they could be too good to be true. Could you briefly describe what they are?

Doly was born out of Radiance Field Tech.

Radiance Fields have been a unique enabler for our use case, because : 

• They manage to represent reality with an in incredible fidelity, 

• They do so without any human intervention so it can be automated and way cheaper than traditional 3D workflows

• They can be merged with more traditional CGI approaches. 

We see Radiance Fields at the intersection of Capture and Imagination. It creates a fabulous fidelity, which is key for our ecommerce users, but is also a gateway to the world of GenAI and its imaginative promises.

What about radiance fields prompted you to learn more and build a platform out?

Pierre has worked in the 3D tools industry for the past 20 years, first in the videogame sector at Autodesk, and then at a 3D CAD-to-mesh extraction and optimization software solution, having explored the limitations of mesh-based 3D.  

The promise of radiance fields was new and exciting, and he started exploring the research on NeRFs which were just out. The early days were humbling in terms of results, but promising and offered an alternative to the traditional approach. 

Rémi had worked On VR and AR projects, including with some of the big retail brands, and had seen the complexity of creating and managing 3D data within those companies. It was clearly a bottleneck for the development of those media.

We explored a few use cases together, including used cars, scanning people but ecommerce seemed like a good candidate. Online merchants we spoke to had little interest in 3D, but a lot of interest for a solution that would help them automate their video production. This was the time where TIktok was emerging, and video became a great selling tool for these companies in a short amount of time. 

Doly’s “twist” is to use a brand new research like Radiance Fields to make something as pervasive as videos. From day one, we wanted a product that would be useful today, and not when the Vision Pro 6 is out :-)

Could you talk about the mission/vision of Doly?

3D in ecommerce is inevitable.

3D is used today by big brands to produce traditional 2D visuals (photos/videos), but we think 3DAI tech will progressively become an extension of photography, allowing users to understand, inspect an object, but also to interact with it. It’s the closest technology to teleportation.

Coming from AR and VR, we recognize the long term potential of these platforms for presenting and understanding a product, but it might take a while to reach a mass audience.

Doly in its current form is the first step to explore the potential of 3DAI to unlock new ecommerce experiences, and to bring the full value of 3D to the small brands.

We also want to enable small brands and entrepreneurs that don’t have access to 3D tech today to use the power of 3D to sell ; it’s going to be with video first, but other experiences will be coming soon…. We were inspired by Canva and its use of templates which is an excellent stepping stone to use 3D creatively. 

Fidelity of products is critical to captures being used in commercial settings. How do you approach the quality of captures in the Doly pipeline?

Our philosophy has been to push the quality first, even at the cost of compute time. The level of expectation of our users is quite high, because we are talking about their products. They are quite aware of the alternatives to Doly, which are professional videographers with expensive equipment. If the quality of our output is not good enough, they will simply not use the videos.

Capture is key to good results : garbage in, garbage out. We’re talking about captures “in the wild” from impatient users with their iPhone at home, with no equipment.

We worked a lot on the capture experience, ensuring it is both easy and yet sufficient to ensure good results. 

This is particularly challenging for our app onboarding stage when users want quick convincing results or just ditch the app. We have iterated a lot on this first step, and are still iterating.

Can people use Doly commercially?

Yes, of course, that’s the whole plan 😀! If our videos aren’t commercially useful to our users, then we don’t have a business. Once you subscribe to Doly, you can use all the videos you create for any use case you want.

Customers and benefits

Today it is very expensive and time consuming to do a video shooting, and it involves a lot of logistics. This is why a lot of small merchants don’t produce video, whereas the platforms they are selling on are more and more demanding in terms of visuals.

Doly doesn’t require particular skills or equipment except from a phone, so products can be captured “in the wild” and not in studios, leading to much more efficiency. Doly decentralizes content production : the product is captured somewhere (often at a warehouse) but the content can be produced elsewhere by a marketing specialist.

Doly early users include small and medium businesses selling online products, but also marketplaces and videographers :

• Vitamin B, a videographer from Hawaii is using the technology to market its brand of shoes on social media and on product pages. See an example here

• Sideline Swap, the biggest US marketplace for second hand sports equipment, uses Doly for their ads on social media.

• WeTheNew, the biggest French sneaker marketplace, is using Doly for 360 videos on their product pages

What excites you most about the future of 3D?

On the tech side, we see a lot of promises:

• We start to see promising General approaches to Radiance Fields which will simplify the capture experience, which is good for us. 

• It will also unleash the possibilities of generative 3D for backgrounds.

• Video diffusion models could generate the best animated templates for our user products, which is good for us too.

• Radiance Fields could become a new CGI primitive, supported by all 3D engines, just like video became supported natively on the browsers

Radiance Fields seem like an obvious extension to photography. It might become as pervasive as photography once we figure out what are the right displays for them.

On the use case side, we think about what could be the best experience of Real time 3D experiences for ecommerce for VR/AR platforms, clearly it won’t look like the current product pages. The ecosystem is not mature yet, and our customers don’t seem to be in demand yet, but there are clear signs of its emergence.

What platforms is Doly available on and where can people find you?

Doly is available for download on iPhone only for now and can be found on Instagram too.