NVIDIA’s research firehose shows no sign of slowing. Hot on the heels of open-sourcing 3DGRT, 3DGUT, and the dynamic 3DGS framework QUEEN, the company has quietly dropped two more gems: GEN3C and Difix3D+.
GEN3C is an exciting method for generating controllable and temporally consistent 2D video from a single reference image. Where GEN3C connects to the world of radiance fields is you are able to take that temporally consistent 2D video and run it through your standard radiance field pipeline to get the lifelike 3D we are accustomed to. This opens a large door for creativity to be generated or to be used for simulation based purposes. After all, the method is built on top of Cosmos and its PhysicalAI uses.
The second and exciting component is Difix3D+ which helps augments captures that have missing regions of coverage or floaters. It's a way too familiar story of capturing, getting home to process and then realizing that some images are blurry or perhaps you forgot to capture a portion of a scene. Difix3D+ helps address that.
Excitingly, GEN3C has been given a commercially permissible license of Apache 2.0, meaning people can use it for their own work. This has been an exciting trend recently for the tech giant and will hopefully spur more adoption of these methodologies. Whereas Difix3D+ comes with the standard NVIDIA license, meaning you will need to reach out to them to use it commercially.
Both GEN3C and Difix3D+ can only be run on Linux currently.
Difix3D+ is currently nominated for an Award at CVPR and their team will be presenting on Sunday afternoon for those who are in Nashville. The code for GEN3C and Difix3D+ can be accessed here.
