Gauzilla adds Segmentation and Annotation

Late last year, we explored Gauzilla, a Gaussian Splatting viewer developed by Josh Sato and crafted in Rust. A quick refresher: Gauzilla utilizes WebAssembly (WASM) for extensive compatibility and is an open-source (MIT License) 3D Gaussian Splatting (3DGS) renderer designed for efficient operation within web browsers. It appears Sato has been hard at work enhancing Gauzilla’s capabilities, promising a host of further advancements in the pipeline.

Through radiance fields, we’re able to achieve a hyperrealistic output of what’s been captured. Sato aims to elevate this further by introducing a interactive 3D Gaussian segmentation and annotation system that operates entirely within a web browser. It utilizes a state-of-the-art vision transformer (ViT) based segmentation model. This unique approach allows for completely serverless operations and GPU independent inferencing.

This update allows users to segment each object and provide specific annotations for them. Depending on the use case, different documentation may prove more useful. For instance, in retail, details like SKU, available sizing, and return policy are invaluable. Coupled with a semantic model such as GARField, it’s conceivable that this process could be automated.

Additional demonstrations made available by Sato underscore Gauzilla‘s impressive performance in-browser, although a relatively powerful GPU may be helpful. Currently, Gauzilla operates using WebGL, but there’s speculation about a transition to WebGPU, promising significant performance enhancements.

This technology has the potential to revolutionize a multitude of industries. Sato has shown radiance fields’ applicability within the retail sector—a behemoth industry poised to reap substantial benefits from this technology. For consumers, this translates to more efficient item location, enhanced quality assessment, and well-informed purchasing decisions.

From a commercial perspective, these innovations pave the way for improved inventory management, product visualization, immersive virtual reality experiences, and the maintenance of high standards across stores. For example, entities like McDonald’s could leverage this for asynchronous oversight of branding standards among franchisees.

Gauzilla remains MIT Licensed, inviting further exploration and development. Find Gauzilla on its GitHub page and follow Sato on LinkedIn for updates to the project.