Comfy3D for ComfyUI

There's an ever growing list of platforms where you can create with radiance fields. Now, ComfyUI has been added to the list with the introduction of Comfy3D. ComfyUI is a Stable Diffusion staple, serving as a GUI, API and backend with a graph/nodes interface.

Software engineer, Mr. For Example has announced the creation of Comfy3D. The first part of the platform is the inclusion of Triplane Gaussian from VAST AI, but this is just the start. Triplane Gaussian is an image to 3D Gaussian Splatting model. The focus of Comfy3D is to make 3D generative AI as efficient, reliable and accessible as current 2D image generative AI.

When that happens (in the near future), not only you can use Gen AI as it was intended, i.e. to generate 3D models, but you can also feed it back into the existing image/video generation workflows by using the renders of the generated 3D model as input images.

Currently through Comfy3D, you can use Deep Marching Tetrahedrons node to convert the Triplane Gaussian output to mesh (obj, .ply, .glb).

While still a work in progress (WIP) and not yet released into the ComfyUI package database, the Comfy3D already supports a range of functionalities that are set to expand further. Users can preview 3DGS and 3D mesh through 3D visualization inside ComfyUI, leveraging gsplat.js and three.js for dynamic and immersive representations.

Mr For Example has also indicated more features to be coming, including a general SDS/VSD Optimization algorithm, Structure from motion (SfM) initialization, and a NeRF based method too.

Comfy3D represents a significant expansion of ComfyUI's capabilities, moving beyond its original focus on text/image to image/video generation to embrace the complex domain of 3D model generation and visualization. This suite of nodes enables users to process 3D inputs—such as mesh and UV texture—utilizing advanced algorithms that are at the forefront of 3D technology. Given ComfyUI's established proficiency in facilitating intricate workflows without the need for coding, the 3D-Pack further empowers users to explore and innovate within the 3D space using a familiar and user-friendly interface.

Comfy3D can be installed from Mr. For Example's Github page here and has only currently been tested on Windows 10 and 11. The project does come with an MIT License for those who are interested in contributing or expanding to the project.