A Beginner’s Guide to Capturing Radiance Field Datasets

Introduction

Over the past three years, I've captured nearly ten thousand radiance fields, both as a personal passion and as a consultant, helping clients discover optimal techniques for capturing people, locations, and objects. This guide is designed to make radiance field technology accessible for newcomers and enthusiasts alike, highlighting key lessons and best practices.

What is a Radiance Field?

A Radiance Field is a 3D representation reconstructed from a collection of 2D images or video that creates lifelike outputs. There are several different radiance field representations, but the two most popular ones are Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting. Gaussian Splatting has broken through in the last year as it renders in real time.

This is an incredibly hot area of research and it is overwhelmingly likely that these radiance field methods will continue to grow exponentially in strength.

Key Terms

  • Radiance Field: A method for generating photorealistic 3D models from images.
  • Gaussian Splatting: A recent, efficient radiance field representation.
  • Parallax: The effect where objects appear to move relative to each other when viewed from different positions.
  • Structure from Motion (SfM): Technique of estimating 3D structures from overlapping 2D images.

For additional terms, please visit our dedicated Glossary.

Gear Recommendations

  • Camera: DSLR or mirrorless cameras are preferred; smartphones are also excellent options.
  • Lenses: Wide-angle lenses (16mm to 50mm).
  • Stability Equipment: Tripods, gimbals, drones.
  • Lighting: Natural lighting is preferred. Cloudy days are great for capturing!

Preparing Your Scene

Understanding Your Environment

  • Walk through the space to identify obstacles, reflections, and high-detail areas.
  • Ensure consistent lighting to minimize shadows and exposure variations.
  • Remove or stabilize any moving objects to avoid reconstruction artifacts.
  • Avoid capturing people unless they are the subject; still images are easier to control.
  • Adapt your capture method depending on your subject, whether it’s a portrait, an entire room, or an inanimate object.

Capture Techniques

Introducing Parallax

A key to a successful reconstruction is parallax—the apparent shift in position of an object when viewed from different angles. This phenomenon allows the system to triangulate the position of points in space accurately. Without sufficient parallax, reconstructions may lack depth and structural accuracy.

To maximize parallax, imagine an infinite number of rays extending from your camera lens and capture as many intersections of these rays as possible. Vary your angle by moving slightly left or right rather than moving in a straight line.

  • Introduce vertical variations (high, mid, low angles).
  • Avoid symmetrical paths to reduce reconstruction ambiguity.

How to Maximize Parallax Effect

  • Capture from multiple viewpoints: Move around the subject in a structured manner, ensuring that each frame has at least 60% overlap with the adjacent frame.
  • Ensure depth variation: Capture images at multiple distances from the subject to introduce both horizontal and vertical parallax.
  • Maintain a structured movement pattern:
    • For small objects: Use an orbit-based capture method with multiple elevations.
    • For large scenes: Incorporate a spiral or grid pattern, ensuring wide coverage while maintaining depth consistency.
  • Use a variety of angles: Introduce different tilts to your camera, capturing high, mid, and low-angle shots to help reconstruct fine details.
  • Avoid symmetrical capture paths: Symmetric paths can create reconstruction ambiguities, making depth estimation more challenging.

Optimal Camera Settings

  • Shutter Speed: Minimum 1/500s (ideally 1/1000s) to eliminate motion blur.
  • ISO: Adjust to achieve higher shutter speeds while maintaining image quality.
  • Aperture: f/8 to f/11 for sharpness and depth of field.
  • Resolution: Use the highest available resolution.
  • Shooting Mode: RAW format is recommended for post-processing flexibility.

Recommended Capture Strategies

  • Spiral Method (Preferred): Cascading spiral paths consistently deliver high-quality results across various subjects.
  • Grid Method: Effective in structured environments with natural markers.

    An example of this would be this capture I took in Grand Central Terminal. I used the pillars as natural grid markers and orbited around them, choosing them to be the center of my cascading spirals.

  • Star Method: Simpler for beginners. I have found results can be less optimal.
    Star Method Example
    Star Method Camera Path
  • Looping Strategy: Capture multiple passes at different heights (eye-level, chest-level, low-level).

Common Challenges and Solutions

Blurry Frames

  • Target shutter speeds of 1/500th second or faster.
  • Increase ISO rather than sacrificing shutter speed.
  • Use tripods or gimbals.
  • Review frames thoroughly and discard any blurred images prior to processing.

Shadows

Depending on the time of day, weather, and location, shadows can present some annoying problems. If you capture the shadows in frame, you’ll see them reconstructed. Sometimes this is unavoidable, but revisit the principle of holding your camera at a slight angle and calculate your capture path around the sun so you don’t capture your subject in shadow. This approach is significantly easier for unbounded scenes; when it comes to people or central objects, you may need to get more creative.

Still Images vs. Video Capture

A common question is whether to capture videos or still images for reconstruction. Both are incredible choices. Based on my personal experience:

I get the highest quality of reconstruction by using still images because I know exactly what images and angles I need, and I can intentionally press the shutter at the perfect moment to reduce handheld camera shake. Videos are great for beginners since you don’t have to worry about pressing the shutter. However, if you use video, you must extract steady, sharp frames from the footage—tools like Reflct’s Sharp Frames can help. Remember, more images do not equal better quality; blurry images should be discarded.

  • Still Images: Higher quality when carefully selected.
  • Video Capture: Easier to capture, but requires frame extraction.

How Many Images Do You Need for Gaussian Splatting?

Here’s another area of debate. In the vast majority of scenes, you will not need more than 300 images.

  • Bounded Scenes: Between 120-140 images is typically ideal.
  • Large-Scale/Unbounded Scenes: Up to 600 frames may be necessary.
  • For example, if you’re looking to capture something like Sutro Tower and the surrounding cityscape, note that one creator took almost three thousand images.

Processing & Reconstruction

There are several platforms available for processing your captures. Depending on your setup and technical familiarity, some options might be easier than others. Remember, most local training Radiance Field platforms require an NVIDIA GPU. An exception to this is Brush.

  • Luma AI (Cloud Based)
  • Kiri Engine (Cloud Based)
  • Postshot (Local)
  • Brush (Local)
  • Nerfstudio (Local)

FAQ & Troubleshooting

  • How many frames do I need?
    • Small scenes: ~120–140 images.
    • Larger scenes: Up to 600 images.
    • Focus on sharpness and sufficient parallax.
  • Can I capture featureless white walls? Yes, but maximize parallax and capture from varied angles.
  • Common Mistakes:
    • Insufficient overlap: Aim for 60-80% overlap.
    • Poor lighting conditions: Capture during optimal daylight.
    • Blurry images: Remove them rather than including them.

Final Thoughts

Radiance field technology is evolving, and capturing strategies will continue to improve. This guide serves as a starting point for newcomers. Experiment, collaborate, and share your insights. For those looking to push the envelope further, please reach out.

Keep exploring, capturing, and sharing your insights. Together, we'll shape the future of radiance field technology.