Recent Radiance Field Papers
Search...
Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS
Abstract
3D Gaussian Splatting (3DGS) has emerged as a powerful technique for radiance field rendering, but it typically requires millions of redundant Gaussian primitives, overwhelming memory and rendering budgets. Existing compaction approaches address this by pruning Gaussians based on heuristic importance scores, without global fidelity guarantee. To bridge this gap, we propose a novel optimal transport perspective that casts 3DGS compaction as global Gaussian mixture reduction. Specifically, we first minimize the composite transport divergence over a KD-tree partition to produce a compact geometric representation, and then decouple appearance from geometry by fine-tuning color and opacity attributes with far fewer Gaussian primitives. Experiments on benchmark datasets show that our method (i) yields negligible loss in rendering quality (PSNR, SSIM, LPIPS) compared to vanilla 3DGS with only 10% Gaussians; and (ii) consistently outperforms state-of-the-art 3DGS compaction techniques. Notably, our method is applicable to any stage of vanilla or accelerated 3DGS pipelines, providing an efficient and agnostic pathway to lightweight neural rendering. The code is publicly available at https://github.com/DrunkenPoet/GHAP
Abstract
Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress. These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance fields of existing frameworks are not directly editable, limiting their applicability in downstream tasks. We propose a solution to these challenges by developing a 3D GAN framework that learns to generate radiance fields of human bodies or faces in a canonical pose and warp them using an explicit deformation field into a desired body pose or facial expression. Using our framework, we demonstrate the first high-quality radiance field generation results for human bodies. Moreover, we show that our deformation-aware training procedure significantly improves the quality of generated bodies or faces when editing their poses or facial expressions compared to a 3D GAN that is not trained with explicit deformations.
Abstract
3D Gaussian Splatting (3DGS) has shown immense potential for novel view synthesis. However, achieving rate-distortion-optimized compression of 3DGS representations for transmission and/or storage applications remains a challenge. CAT-3DGS introduces a context-adaptive triplane hyperprior for end-to-end optimized compression, delivering state-of-the-art coding performance. Despite this, it requires prolonged training and decoding time. To address these limitations, we propose CAT-3DGS Pro, an enhanced version of CAT-3DGS that improves both compression performance and computational efficiency. First, we introduce a PCA-guided vector-matrix hyperprior, which replaces the triplane-based hyperprior to reduce redundant parameters. To achieve a more balanced rate-distortion trade-off and faster encoding, we propose an alternate optimization strategy (A-RDO). Additionally, we refine the sampling rate optimization method in CAT-3DGS, leading to significant improvements in rate-distortion performance. These enhancements result in a 46.6% BD-rate reduction and 3x speedup in training time on BungeeNeRF, while achieving 5x acceleration in decoding speed for the Amsterdam scene compared to CAT-3DGS.
Abstract
We present resolved NACO photometry of the close binary AB Dor B in H- and Ks-band. AB Dor B is itself known to be a wide binary companion to AB Dor A, which in turn has a very low-mass close companion named AB Dor C. These four known components make up the young and dynamically interesting system AB Dor, which will likely become a benchmark system for calibrating theoretical pre-main sequence evolutionary mass tracks for low-mass stars. However, for this purpose the actual age has to be known, and this subject has been a matter of discussion in the recent scientific literature. We compare our resolved photometry of AB Dor Ba and Bb with theoretical and empirical isochrones in order to constrain the age of the system. This leads to an age estimate of about 50 to 100 Myr. We discuss the implications of such an age range for the case of AB Dor C, and compare with other results in the literature.
Abstract
Recent work on Neural Radiance Fields (NeRF) has demonstrated significant advances in high-quality view synthesis. A major limitation of NeRF is its low rendering efficiency due to the need for multiple network forwardings to render a single pixel. Existing methods to improve NeRF either reduce the number of required samples or optimize the implementation to accelerate the network forwarding. Despite these efforts, the problem of multiple sampling persists due to the intrinsic representation of radiance fields. In contrast, Neural Light Fields (NeLF) reduce the computation cost of NeRF by querying only one single network forwarding per pixel. To achieve a close visual quality to NeRF, existing NeLF methods require significantly larger network capacities which limits their rendering efficiency in practice. In this work, we propose a new representation called Neural Radiance Distribution Field (NeRDF) that targets efficient view synthesis in real-time. Specifically, we use a small network similar to NeRF while preserving the rendering speed with a single network forwarding per pixel as in NeLF. The key is to model the radiance distribution along each ray with frequency basis and predict frequency weights using the network. Pixel values are then computed via volume rendering on radiance distributions. Experiments show that our proposed method offers a better trade-off among speed, quality, and network size than existing methods: we achieve a ~254x speed-up over NeRF with similar network size, with only a marginal performance decline. Our project page is at yushuang-wu.github.io/NeRDF.
Abstract
We introduce DiffRF, a novel approach for 3D radiance field synthesis based on denoising diffusion probabilistic models. While existing diffusion-based methods operate on images, latent codes, or point cloud data, we are the first to directly generate volumetric radiance fields. To this end, we propose a 3D denoising model which directly operates on an explicit voxel grid representation. However, as radiance fields generated from a set of posed images can be ambiguous and contain artifacts, obtaining ground truth radiance field samples is non-trivial. We address this challenge by pairing the denoising formulation with a rendering loss, enabling our model to learn a deviated prior that favours good image quality instead of trying to replicate fitting errors like floating artifacts. In contrast to 2D-diffusion models, our model learns multi-view consistent priors, enabling free-view synthesis and accurate shape generation. Compared to 3D GANs, our diffusion-based approach naturally enables conditional generation such as masked completion or single-view 3D synthesis at inference time.
Abstract
Despite Neural Radiance Fields (NeRF) showing compelling results in photorealistic novel views synthesis of real-world scenes, most existing approaches require accurate prior camera poses. Although approaches for jointly recovering the radiance field and camera pose exist (BARF), they rely on a cumbersome coarse-to-fine auxiliary positional embedding to ensure good performance. We present Gaussian Activated neural Radiance Fields (GARF), a new positional embedding-free neural radiance field architecture - employing Gaussian activations - that outperforms the current state-of-the-art in terms of high fidelity reconstruction and pose estimation.
Abstract
While key effects of the many-body problem---such as Kondo and Mott physics---can be understood in terms of on-site correlations, non-local fluctuations of charge, spin, and pairing amplitudes are at the heart of the most fascinating and unresolved phenomena in condensed matter physics. Here, we review recent progress in diagrammatic extensions to dynamical mean-field theory for ab initio materials calculations. We first recapitulate the quantum field theoretical background behind the two-particle vertex. Next we discuss latest algorithmic advances in quantum Monte Carlo simulations for calculating such two-particle quantities using worm sampling and vertex asymptotics, before giving an introduction to the ab initio dynamical vertex approximation (AbinitioD$Γ$A). Finally, we highlight the potential of AbinitioD$Γ$A by detailing results for the prototypical correlated metal SrVO$_3$.
Abstract
The interaction potential energy surface (PES) of He-H2 is of great importance for quantum chemistry, as the simplest test case for interactions between a molecule and a closed-shell atom. It is also required for a detailed understanding of certain astrophysical processes, namely collisional excitation and dissociation of H2 in molecular clouds, at densities too low to be accessible experimentally. A new set of 23703 ab initio energies was computed, for He-H2 geometries where the interaction energy was expected to be non-negligible. These have an estimated rms "random" error of about 0.2 millihartree and a systematic error of about 0.6 millihartree (0.4 kcal/mol). A new analytic He-H2 PES, with 112 parameters, was fitted to 20203 of these new ab initio energies (and to an additional 4862 points generated at large separations). This yielded an improvement by better than an order of magnitude in the fit to the interaction region, relative to the best previous surfaces (which were accurate only for near-equilibrium H2 molecule sizes). This new PES has an rms error of 0.95 millihartree (0.60 kcal/mole) relative to the the 14585 ab initio energies that lie below twice the H2 dissociation energy, and 2.97 millihartree (1.87 kcal/mole) relative to the full set of 20203 ab initio energies (the fitting procedure used a reduced weight for high energies, yielding a weighted rms error of 1.42 millihartree, i.e., 0.89 kcal/mole). These rms errors are comparable to the estimated error in the ab initio energies themselves; the conical intersection between the ground state and the first excited state is the largest source of error in the PES.
Abstract
Novel view synthesis via Neural Radiance Fields (NeRFs) or 3D Gaussian Splatting (3DGS) typically necessitates dense observations with hundreds of input images to circumvent artifacts. We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images, by leveraging a diffusion model pre-trained from multiview datasets. Different from using diffusion priors to regularize representation optimization, our method directly uses diffusion-generated images to train NeRF/3DGS as if they were real input views. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality photorealistic pseudo-observations. To resolve consistency among pseudo-observations and real input views, we develop an uncertainty measure to guide the diffusion model's generation. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times. Extensive experiments across diverse and challenging datasets validate that our approach outperforms existing state-of-the-art methods and is capable of synthesizing novel views with super-resolution in the few-view setting.