PhysAvatar's Dynamic Dances

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Playing as yourself in a video game has always seemed like a fun idea. Now, we're one step closer to making that a reality with PhysAvatar.

Michael Rubloff

Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

Playing as yourself in a video game has always seemed like a fun idea, though perhaps not in GTA. Now, we’re one step closer to making that a reality with PhysAvatar.

Throughout the paper, they reference something called Inverse Rendering, a process that encompasses methods like NeRFs and Gaussian Splatting. Inverse Rendering is essentially about creating 3D objects from 2D origins—a concept that might sound familiar.

PhysAvatar uses dynamic Gaussian Splatting and “inverse physics” to estimate and model the physical parameters of clothing. This offers a comprehensive solution to the long-standing challenge of reconstructing dynamic 3D avatars.

It can be broken down into three main parts: mesh tracking, physics-based dynamic modeling, and physics-based appearance refinement.

Using video and a mesh as input, they adapt 3D Gaussians (they use Dynamic 3D Gaussians) to align with the mesh surface at the outset. This alignment involves positioning, orienting, and scaling the Gaussians to match the initial configuration of the garment’s mesh, ensuring that they accurately represent the garment’s state at the start of the video sequence.

As the avatar moves, the positions and orientations of the 3D Gaussians are dynamically adjusted to follow the deformations of the garment’s mesh. This process is facilitated by optimizing the parameters of the Gaussians—position, rotation, scale, and opacity—to minimize the difference between the rendered images and the reference images from the video data.

The optimized parameters of the 3D Gaussians inform the reconstruction of the garment’s mesh across frames. This reconstruction is not static but evolves with the avatar’s movements, capturing the nuanced dynamics of clothing, such as wrinkles and draping effects, that are critical for realism.

After tracking the meshes, the physical properties of the garments are estimated to ensure their dynamics match observed movements. Once this is complete, these parameters can be applied to new views and motions.

It operates by first establishing dense correspondences across video frames, followed by optimizing fabric physical parameters through an inverse physics approach, and finally refining appearance using a physically based inverse renderer. This ensures avatars can be rendered with high fidelity in any lighting environment, addressing the challenge of baked-in shadows and other artifacts that can detract from realism.

By embedding physics-based simulation into the avatar reconstruction process, PhysAvatar can model the dynamics of clothing with remarkable accuracy, capturing complex behaviors like friction and collision.

Interestingly their outputs can be directly exported into game engines, such as Blender. There is a placeholder for the code to be released, but it is both unclear when that will be and how it will be licensed.

When HumanRF and the Actors dataset (used in PhysAvatar) came out, I spoke about the potential impact of the fashion industry and here’s a closer glimpse of that reality. Imagine fashion shows that take place asynchronously and also out of your living room. There’s still quite a bit of work ahead of us to get there, but it is achievable and I believe inevitable.

While PhysAvatar represents a significant leap forward, it also highlights areas for further research and development. The current need for manual garment segmentation and mesh unwrapping calls for automation to streamline the process. Additionally, improving the accuracy of the underlying human body model used for garment simulation will enhance collision detection and overall realism. Adapting the framework to work with sparse view captures will also broaden its applicability, making high-fidelity avatar reconstruction accessible in less controlled environments.

Thumbnail for Gaussian Splatting News episode 12

11:38

Watch · New every Monday

Splats Come to Fortnite, Nike Shoots in 4D, Splats on a Nintendo 3DS

Episode 12 · Nov 9, 2026

Epic’s UEFN 43.00 brings Gaussian splats into the Unreal Editor for Fortnite, importing PLY and SPZ files that render on every platform UEFN supports, mobile included.

Featured

Recents

Platforms

Splat Prep Brings Splat Export to Blender 5.3

Splat Prep 1.0.1 is a $19 Blender 5.3 extension that bakes transforms into Gaussian splats, joins splat objects and exports PLY, compressed PLY, SPZ and .splat.

Michael Rubloff

Nov 16, 2026

Platforms

Splat Prep Brings Splat Export to Blender 5.3

Splat Prep 1.0.1 is a $19 Blender 5.3 extension that bakes transforms into Gaussian splats, joins splat objects and exports PLY, compressed PLY, SPZ and .splat.

Michael Rubloff

Platforms

Voyager Releases Vauss Splat Trainer as a Research Preview

Voyager releases Vauss, a source-available Gaussian splat trainer built on Brush that turns one video into viewer, Blender, voxel game and Unity exports.

Michael Rubloff

Nov 16, 2026

Platforms

Voyager Releases Vauss Splat Trainer as a Research Preview

Voyager releases Vauss, a source-available Gaussian splat trainer built on Brush that turns one video into viewer, Blender, voxel game and Unity exports.

Michael Rubloff

Platforms

sperid-swift-mlx Brings Idol Image-to-Splat to the Mac

sperid-swift-mlx ports Speri Labs' Idol to Swift and MLX, turning one photo into a 3D Gaussian splat on Apple Silicon Macs with PLY, USDZ, and GLB export.

Michael Rubloff

Nov 15, 2026

Platforms

sperid-swift-mlx Brings Idol Image-to-Splat to the Mac

sperid-swift-mlx ports Speri Labs' Idol to Swift and MLX, turning one photo into a 3D Gaussian splat on Apple Silicon Macs with PLY, USDZ, and GLB export.

Michael Rubloff

Platforms

LCC Studio Universal V2.7.0 Adds AMD and Intel GPU Support

XGRIDS LCC Studio Universal V2.7.0 brings single-scene 3DGS reconstruction, editing and application to AMD and Intel discrete GPUs, alongside its NVIDIA build.

Michael Rubloff

Nov 15, 2026

Platforms

LCC Studio Universal V2.7.0 Adds AMD and Intel GPU Support

XGRIDS LCC Studio Universal V2.7.0 brings single-scene 3DGS reconstruction, editing and application to AMD and Intel discrete GPUs, alongside its NVIDIA build.

Michael Rubloff

Trending Articles
  1. TRENDING
    Loading...
  2. TRENDING
    Loading...
  3. TRENDING
    Loading...
Michael Rubloff

Written by Michael Rubloff

Michael is the Founder and Managing Editor of Radiancefields.com

More from Michael Rubloff

More from Michael Rubloff

Platforms

Splat Prep Brings Splat Export to Blender 5.3

Splat Prep 1.0.1 is a $19 Blender 5.3 extension that bakes transforms into Gaussian splats, joins splat objects and exports PLY, compressed PLY, SPZ and .splat.

Michael Rubloff

Nov 16, 2026

Platforms

Splat Prep Brings Splat Export to Blender 5.3

Michael Rubloff

Nov 16, 2026

Platforms

Splat Prep Brings Splat Export to Blender 5.3

Splat Prep 1.0.1 is a $19 Blender 5.3 extension that bakes transforms into Gaussian splats, joins splat objects and exports PLY, compressed PLY, SPZ and .splat.

Michael Rubloff

Nov 16, 2026

Platforms

Voyager Releases Vauss Splat Trainer as a Research Preview

Voyager releases Vauss, a source-available Gaussian splat trainer built on Brush that turns one video into viewer, Blender, voxel game and Unity exports.

Michael Rubloff

Nov 16, 2026

Vauss

Platforms

Voyager Releases Vauss Splat Trainer as a Research Preview

Michael Rubloff

Nov 16, 2026

Vauss

Platforms

Voyager Releases Vauss Splat Trainer as a Research Preview

Voyager releases Vauss, a source-available Gaussian splat trainer built on Brush that turns one video into viewer, Blender, voxel game and Unity exports.

Michael Rubloff

Nov 16, 2026

Vauss

Platforms

sperid-swift-mlx Brings Idol Image-to-Splat to the Mac

sperid-swift-mlx ports Speri Labs' Idol to Swift and MLX, turning one photo into a 3D Gaussian splat on Apple Silicon Macs with PLY, USDZ, and GLB export.

Michael Rubloff

Nov 15, 2026

Sperid Idol

Platforms

sperid-swift-mlx Brings Idol Image-to-Splat to the Mac

Michael Rubloff

Nov 15, 2026

Sperid Idol

Platforms

sperid-swift-mlx Brings Idol Image-to-Splat to the Mac

sperid-swift-mlx ports Speri Labs' Idol to Swift and MLX, turning one photo into a 3D Gaussian splat on Apple Silicon Macs with PLY, USDZ, and GLB export.

Michael Rubloff

Nov 15, 2026

Sperid Idol

New to Gaussian Splatting? Start here

Radiance fields, explained from the ground up: what a Gaussian splat is, how to capture one, and which tools and gear to start with.

Newsletter

Keep up with the field.

New research, tools and captures from the Radiance Fields newsroom, in your inbox. Free, no spam.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.

Gaussian Splatting News on YouTube

The week in radiance fields, every Monday.