nerfstudio Command Guide

If you're like me, you have a note saved in a text editor with a bunch of commands, because remembering things can be tough. So I figured I would share some of my commands that I use for nerfstudio, with easily understandable fill in blanks for you to use as well.

If any of this still remains confusing, feel free to reach out to me!

For those looking for a Instant-NGP guide, use this page!

Typically I'm either using a standard video or a collection of images. The first thing I will do in a new anaconda window will be to type

cd nerfstudio

After that I will type

conda activate nerfstudio

Now I'm ready to begin processing the NeRF.

Process a NeRF in nerfstudio - Photos

This is the same process for Gaussian Splatting. This is to be used for processing images. Here's the standard code, followed by an example folder, named bulldozer.

ns-process-data images --data data/nerfstudio/insertfoldername --output-dir data/nerfstudio/insertfoldername

Everything leading up to the bolded words should be standard for you. What you need to customize is the highlighted portion. So, if I had my images in a folder called bulldozer, my code would look like this:

ns-process-data images --data data/nerfstudio/bulldozer --output-dir data/nerfstudio/bulldozer

When it is just images, you only need to point the command to the folder in which the images are located.

Depending on how fancy you want to get, you can add on additional arguments. Arguments are essentially just extra parameters that you can fine tune to your liking. A lot of experimenting is required to figure out what you like to use, but to help save some time, here's an example of one that I prefer to use!

--matching-method

Possible choices: exhaustive, sequential, vocab_tree

I like to select exhaustive. Using exhaustive will be slower, but for me, my goal is to get the cleanest NeRF possible, so I'm ok with waiting. If that isn't you, that's totally fine! All of the options are still good. With that in mind, here is how the code is affected, with the new part highlighted in bolded.

ns-process-data images --matching-method exhaustive --data data/nerfstudio/bulldozer --output-dir data/nerfstudio/bulldozer

If you want to see the full list of options, they can be found here!

Process a NeRF in nerfstudio - Video

When you're starting with a video, it's ever so slightly more complicated. This time, instead of just giving the folder location, you need to specify the name of the video file as well. We'll stick with the same Bulldozer example from Images, but pretend that the name of the video file we are trying to process is called IMG_2892.mov. Notice that I need to specify the video file name in the first part, but I only need to give the folder name for the output.

ns-process-data video --data data/nerfstudio/Bulldozer/IMG_2898.mov --output-dir data/nerfstudio/Bulldozer

If it fails, double check that you have the correct file extension written. I've spent way too much time trying to figure out why it wasn't running, only to realize that I had written IMG_2898.mp4 instead of IMG_2898.mov and vice versa.

nerfstudio will automatically pull out a percentage of the frames for you.

Training a NeRF or Gaussian Splatting in nerfstudio

The processing stage will normally take somewhere between 10-20 minutes. Once you have completed the processing step, your Anaconda window should look something like this.

nerfstudio will tell you the percentage of matches that were located.

This is where you the amount of options really opens up. Depending on your VRAM and what is your trying to do with the NeRF, you have several options. The most confusing part of the addition of gaussian splatting is that the initial processing method is exactly the same as a NeRF.

As of right now, here are all of the supported methods in alphabetical order:

• Instant NGP: Instant Neural Graphics Primitives with a Multi Resolution Hash Encoding

• Instruct-GS2GS: Editing Gaussian Splatting Scenes with Instructions

• Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions

• Generfacto: Generate 3D models from text

• K-Planes: Unified 3D and 4D Radiance Fields

• LERF: Language Embedded Radiance Fields

• Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields

• NeRF: The original NeRF paper.

• Nerfacto: our de facto NeRF method combines modules focused on quality with modules focused on faster rendering. nerfstudio easily lets us experiment with the best of both worlds! There is also Nerfacto-Big and Nerfacto-Huge, which are more powerful and VRAM consuming methods.

• Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs

• NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields

• PyNeRF: Pyramidal Neural Radiance Fields for fast NeRF anti-aliasing.

• Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra

• TensoRF: Tensorial Radiance Fields

• Splatfacto: Nerfstudio's homemade implementation of 3D Gaussian Splatting. There is another variation named Splatfacto-Big, for more detailed scenes.

• SeaThru-NeRF: A NeRF method targeting underwater, foggy, or hazy scenes.

• Zip-NeRF: Google's SoTA method.

Here I will demonstrate both the standard Nerfacto, Nerfacto-huge, and Gaussian Splatting code.

ns-train nerfacto --data data/nerfstudio/bulldozer

Again, the yellow highlighting is what will be unique to you. And if I want to use nerfacto-huge, the code would look like this:

ns-train nerfacto-huge --data data/nerfstudio/bulldozer

If you want to train guassian splatting, the code will look like this:

ns-train gaussian-splatting --data data/nerfstudio/bulldozer

Assuming that everything is working, your anaconda window will look something like this.

How to load a checkpoint in nerfstudio

Once your NeRF has been fully trained, nerfstudio saves a checkpoint of it automatically. Checkpoints, or saved files are located in your outputs folder. If you're looking for that Outputs folder, go to the initial nerfstudio folder and it's located right there! For a while I couldn't figure out how I could load my checkpoint, but it's very simple. The file called config.yml is your checkpoint! You need to use this code in Anaconda and then customize the part that's bolded to what's unique to you.

ns-viewer --load-config outputs\bench2\nerfacto\2023-07-09_151026/config.yml

Hopefully this guide was helpful to you!