If you're like me, you have a note saved in a text editor with a bunch of commands, because remembering things can be tough. So I figured I would share some of my commands that I use for nerfstudio, with easily understandable fill in blanks for you to use as well.
If any of this still remains confusing, feel free to reach out to me!
For those looking for a Instant-NGP guide, use this page!
Typically I'm either using a standard video or a collection of images. The first thing I will do in a new anaconda window will be to type
After that I will type
Now I'm ready to begin processing the NeRF.
Process a NeRF in nerfstudio - Photos
This is the same process for Gaussian Splatting. This is to be used for processing images. Here's the standard code, followed by an example folder, named bulldozer.
Everything leading up to the bolded words should be standard for you. What you need to customize is the highlighted portion. So, if I had my images in a folder called bulldozer, my code would look like this:
When it is just images, you only need to point the command to the folder in which the images are located.
Depending on how fancy you want to get, you can add on additional arguments. Arguments are essentially just extra parameters that you can fine tune to your liking. A lot of experimenting is required to figure out what you like to use, but to help save some time, here's an example of one that I prefer to use!
Possible choices: exhaustive, sequential, vocab_tree
I like to select exhaustive. Using exhaustive will be slower, but for me, my goal is to get the cleanest NeRF possible, so I'm ok with waiting. If that isn't you, that's totally fine! All of the options are still good. With that in mind, here is how the code is affected, with the new part highlighted in bolded.
If you want to see the full list of options, they can be found here!
Process a NeRF in nerfstudio - Video
When you're starting with a video, it's ever so slightly more complicated. This time, instead of just giving the folder location, you need to specify the name of the video file as well. We'll stick with the same Bulldozer example from Images, but pretend that the name of the video file we are trying to process is called IMG_2892.mov. Notice that I need to specify the video file name in the first part, but I only need to give the folder name for the output.
If it fails, double check that you have the correct file extension written. I've spent way too much time trying to figure out why it wasn't running, only to realize that I had written IMG_2898.mp4 instead of IMG_2898.mov and vice versa.
Nerfstudio will automatically pull out a percentage of the frames for you.
Training a NeRF or Gaussian Splatting in nerfstudio
The processing stage will normally take somewhere between 10-20 minutes. Once you have completed the processing step, your Anaconda window should look something like this.
nerfstudio will tell you the percentage of matches that were located.
This is where you the amount of options really opens up. Depending on your VRAM and what is your trying to do with the NeRF, you have several options. The most confusing part of the addition of gaussian splatting is that the initial processing method is exactly the same as a NeRF.
As of right now, here are all of the supported methods in alphabetical order:
Instant NGP: Instant Neural Graphics Primitives with a Multi Resolution Hash Encoding
Instruct-GS2GS: Editing Gaussian Splatting Scenes with Instructions
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
Generfacto: Generate 3D models from text
K-Planes: Unified 3D and 4D Radiance Fields
LERF: Language Embedded Radiance Fields
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
NeRF: The original NeRF paper.
Nerfacto: our de facto NeRF method combines modules focused on quality with modules focused on faster rendering. nerfstudio easily lets us experiment with the best of both worlds! There is also Nerfacto-Big and Nerfacto-Huge, which are more powerful and VRAM consuming methods.
Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFs
NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields
PyNeRF: Pyramidal Neural Radiance Fields for fast NeRF anti-aliasing.
Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedra
TensoRF: Tensorial Radiance Fields
Splatfacto: Nerfstudio's homemade implementation of 3D Gaussian Splatting. There is another variation named Splatfacto-Big, for more detailed scenes.
SeaThru-NeRF: A NeRF method targeting underwater, foggy, or hazy scenes.
Zip-NeRF: Google's SoTA method.
Here I will demonstrate both the standard Nerfacto, Nerfacto-huge, and Gaussian Splatting code.
Again, the yellow highlighting is what will be unique to you. And if I want to use nerfacto-huge, the code would look like this:
If you want to train guassian splatting, the code will look like this:
Assuming that everything is working, your anaconda window will look something like this.
How to load a checkpoint in nerfstudio
Once your NeRF has been fully trained, nerfstudio saves a checkpoint of it automatically. Checkpoints, or saved files are located in your outputs folder. If you're looking for that Outputs folder, go to the initial nerfstudio folder and it's located right there! For a while I couldn't figure out how I could load my checkpoint, but it's very simple. The file called config.yml is your checkpoint! You need to use this code in Anaconda and then customize the part that's bolded to what's unique to you.
Hopefully this guide was helpful to you!
