ByteDance MultiMedia Lab is moving its work on 6DoF video from research demonstrations into commercial production.

At the Volcano Engine’s FORCE conference in December 2025, the lab presented an intelligent 3D video platform covering on-demand volumetric content, interactive live streaming, and holographic communication. The lab has also deployed monocular technology that converts existing 2D video into limited-range 6DoF experiences on consumer devices.

The important point is that the lab is not relying on one capture system for every application. Its work now spans three production levels, each balancing fidelity, latency, capture cost, and viewpoint range differently.

At the highest-quality end is a multi-camera 4D Gaussian Splatting pipeline for on-demand content.

Depending on production complexity, between 10 and 100 calibrated cameras capture a dynamic scene for offline reconstruction. The complete workflow covers reconstruction, compression, encoding, streaming decoding, and lightweight rendering across devices.

The lab says the production pipeline can deliver high-quality 4DGS assets below 30 Mbps, with a compression ratio of approximately 5–10× in quality-oriented configurations. It also combines volumetric video with spatial audio and includes AIGC-based super-resolution and quality enhancement. Target applications include concerts, sports, large events, film production, and narrative 6DoF content.

The second level prioritizes immediacy.

LiveGS, presented at SIGGRAPH 2025 Emerging Technologies, uses a small number of synchronized cameras instead of a large volumetric array. The capture system can also use multiple smartphones, including iPhones synchronized through Genlock.

LiveGS replaces lengthy per-scene optimization with a feed-forward reconstruction network. It assigns finer Gaussians to detailed areas such as the face, hands, hair, and clothing while representing flatter regions more coarsely. According to the lab, this reduces the number of Gaussians by approximately 70 percent and enables reconstruction above 30 FPS on an NVIDIA A10.

To make the representation streamable, LiveGS reorganizes Gaussian attributes into two-dimensional video planes that can be processed by established video codecs. Rendering-aware bitrate allocation gives greater precision to the attributes and regions that contribute most to the final image.

The lab reports end-to-end 1080p streaming at 10–20 Mbps, rendering above 30 FPS, and PSNR above 30 dB. Its public materials also report compression of up to 500× with a compression-related PSNR reduction of less than 3 dB.

On the receiving side, a mask-based Gaussian selection strategy reduces the mobile rendering workload. The system has demonstrated interactive rendering at approximately 30 FPS on an iPhone 15.

The same architecture provides the foundation for the lab’s holographic communication system. A lightweight setup of four to five cameras captures a participant before the scene is reconstructed in the cloud, compressed, and rendered on a glasses-free 3D display. The lab reports bandwidth of around 10 Mbps and also supports synchronized smartphone capture.

The third production level removes the multi-camera requirement altogether.

The lab’s monocular system converts an existing image, video, or live stream into content with estimated depth and limited motion parallax. Additional spatial data can travel through the existing live-streaming chain and be rendered using a lightweight client pipeline.

On mobile devices, gyroscope input allows the viewpoint to respond as the user moves the screen. The navigable range is smaller than that of full volumetric video, but existing 2D content can be converted without rebuilding the original capture workflow.

The technology supports real-time inference and has been deployed across Douyin mobile devices, Apple Vision Pro, and PICO. A newer version can also run locally on mobile hardware. This creates a lower-cost entry point for spatial live video while the multi-camera systems serve productions requiring a fuller 6DoF representation.

Published work behind these systems addresses a broader set of challenges in making dynamic Gaussian scenes practical. DeformableGS reconstructs high-fidelity dynamic scenes from monocular video. TaoGS and EvolvingGS extend dynamic Gaussian scenes beyond fixed capture, respectively addressing topology changes and scenes that continue to evolve over time. DisentangledGS separates geometry from appearance for relighting, while Director adds instance-consistent semantics and open-vocabulary querying.

Rather than forming a sequential pipeline, these projects map out the capabilities required by a 4D media system: efficient representation, temporal robustness, controllable appearance, scene understanding, and real-time delivery.

That breadth is what makes ByteDance MultiMedia Lab’s commercialization effort notable. A high-end production can choose dense multi-camera capture for maximum quality. A broadcaster can use LiveGS for real-time free-viewpoint video. Existing 2D content can enter through monocular conversion with almost no change to the original capture setup.

6DoF video is therefore becoming less of a single specialized format and more of a flexible 4D media stack. The applications range from concerts, sports, and interactive narratives to mobile 3D video and holographic communication.

For 4D Gaussian Splatting, the next step is not only to reconstruct dynamic scenes. It is to make those scenes available at the quality, latency, and production cost required by different kinds of media.

字节跳动多媒体实验室构建了面向六自由度直播与点播的 4D 媒体系统

字节跳动

多媒体实验室

4D 媒体流

传媒与娱乐

导语： 字节跳动多媒体实验室的智能 3D 视频技术体系结合了高保真多目 4DGS、实时 LiveGS，以及面向既有二维内容的单目转换技术。

字节跳动多媒体实验室正推动其 6DoF 视频技术从科研演示进入商业化生产。

在 2025 年 12 月举行的火山引擎 FORCE 原动力大会上，实验室展示了一套覆盖体积视频点播、互动直播和全息通信的智能 3D 视频平台。实验室还落地了单目技术，可以将现有二维视频转换为适合消费级终端观看的小范围 6DoF 内容。

关键在于，实验室并未尝试用同一种采集系统覆盖所有应用。其技术目前形成了3个生产层级，分别在画质、延迟、采集成本和观看范围之间进行不同取舍。

画质最高的一端，是面向点播内容的多相机 4D Gaussian Splatting 方案。

根据制作复杂程度，系统使用 10 至 100 台经过标定的相机采集动态场景，并通过离线方式完成重建。完整流程包括重建、压缩、编码、流式解码和跨设备轻量化渲染。

实验室表示，该制作流程能够以低于 30 Mbps 的码率传输高质量 4DGS 资产，在画质优先的配置下实现约 5 至 10 倍压缩。系统还将体积视频与空间音频结合，并引入基于 AIGC 的超分辨率和质量增强技术。目标应用包括演唱会、体育赛事、大型活动、影视制作和剧情类 6DoF 内容。

第二个层级优先解决实时性问题。

在 SIGGRAPH 2025 Emerging Technologies 展示的 LiveGS，使用少量同步相机替代大型体积视频阵列。系统同样支持多台手机采集，包括通过 Genlock 实现同步的 iPhone。

LiveGS 使用前馈式重建网络替代耗时较长的逐场景优化。系统会在面部、手部、头发和服装等细节区域使用更细粒度的高斯，在平坦区域使用更粗粒度的表征。

根据实验室公布的资料，该策略可以将高斯数量减少约 70%，并在 NVIDIA A10 上实现超过 30 FPS 的实时重建。

为了让这种表征具备流式传输能力，LiveGS 将高斯属性重新组织到二维视频平面中，再使用成熟的视频编解码器进行处理。基于渲染重要性的码率分配，会为对最终画面贡献最大的属性和区域保留更高精度。

实验室公布的数据显示，系统可以在 10 至 20 Mbps 下实现 1080p 端到端实时传输，渲染帧率超过 30 FPS，PSNR 高于 30 dB。其公开资料还显示，系统最高可以实现约 500 倍压缩，压缩引起的 PSNR 下降小于 3 dB。

在接收端，基于掩码的高斯选择策略进一步降低了移动设备的渲染负载。系统已在 iPhone 15 上展示约 30 FPS 的交互渲染。

同一套架构也构成了实验室全息通信系统的基础。由约四至五台相机组成的轻量化系统采集参与者，云端完成场景重建和压缩，终端则在裸眼 3D 显示器上进行渲染。实验室公布的带宽约为 10 Mbps，系统也支持多台同步智能手机采集。

第三个生产层级则完全移除了多相机要求。

实验室的单目系统可以将既有图像、视频或直播流转换为带有估计深度和有限运动视差的内容。新增的空间数据能够复用现有直播链路，并通过轻量化客户端流程完成渲染。

在移动设备上，陀螺仪输入可以让观看视点随着用户移动屏幕而变化。其可移动范围小于完整体积视频，但能够直接转换既有二维内容，无需重新建设原始拍摄流程。

该技术支持实时推理，已经在抖音移动端、Apple Vision Pro 和 PICO 上线。新版本还可以在移动设备本地运行。这为实时空间视频提供了成本更低的入口，而多目系统则继续服务于需要更完整 6DoF 表征的专业制作。

这些系统背后的论文工作，覆盖了让动态高斯场景走向实际应用所需解决的一系列问题。DeformableGS 研究如何从单目视频中重建高保真动态场景。TaoGS 和 EvolvingGS 将动态高斯场景从固定采集进一步拓展到拓扑变化和随时间持续演化的场景。DisentangledGS 通过分离几何与外观支持重光照，Director 则加入实例一致语义和开放词汇查询能力。

这些研究并非一条顺序连接的算法流水线，而是共同勾勒出 4D 媒体系统所需的能力：高效表征、时序鲁棒性、可控外观、场景理解与实时交付。

这种技术广度正是字节跳动多媒体实验室商业化探索值得关注的地方。高端制作可以使用稠密多相机采集追求最高质量；直播方可以使用 LiveGS 制作实时自由视角视频；大量既有二维内容则可以通过单目转换进入空间媒体体系，而几乎不需要改变原有采集流程。

因此，6DoF 视频正在从一种单一的专业内容格式，转变为一套可以灵活选择的4D媒体技术栈。 其应用可以覆盖演唱会、体育赛事和交互式剧情内容，也可以延伸到移动端立体视频与全息通信。

对于 4D Gaussian Splatting，下一步不只是重建动态场景，而是按照不同媒体所需要的画质、延迟和制作成本，将这些场景真正交付给观众。