NeRF Dataset Preparer

Inspired by the auto sharp image culling method that we highlighted recently, another dataset preparer has hit Github.

Just like FFFDeblur, the image sharpness selection is based on Laplacian Variance and this script allows a user to specify a certain percentage of sharp images to utilize in a data set.

This comes with a bit more features, in that not only are you able to specify a specific percentage of photos to eliminate, but you can also see what section of the NeRF contain those images. For instance, if it’s removing a bunch of images from the back of the dataset, you would know that it’s likely the final parts of your data capture might have suffered from low quality data and thus that section might not come out well.

Personally I’m quite interested in the ability for anyone to shoot a mediocre video with any off-the-shelf phone camera and turn that into a good NeRF, these scripts were written with the intention to help with that.SharkWipfTweet

There are two main scripts to take a look at:01_filter_raw_data.py and 02_filter_colmap_data.py. Both scripts are completely written in Python.

Like its name implies, 01 filter raw data is the step where you specify the specific top percentage of sharp frames to be selected. This then automatically culls the least sharp frames, with the ability to send the removed frames to a new folder. You can choose to filter out a certain percentage of images or you can specify the total number of resulting images you want. Depending on your dataset, one option may make more sense than the other.

It visualizes the frequency of where the soon to be deleted images are coming from and showcases how image quality varies across the scene. Ideally, you would want an even distribution to come back. For instance, it could be a problem, if one section of your input data was so sharp, that the rest of the scene didn’t get equal representation in the final images selected. Luckily for all of us, unless otherwise specified, it tries to distribute the deleted images evenly across the dataset.

The method introduces two different tweakable parameters, Scalars and Groups. On default the method is set for Scalar 1, which creates the most image sections to pull from, ensuring that the final images removed are evenly distributed. Whereas Groups 1, would just be large group, where it’s pulling the worst images from, even if it’s just in one area of the video. Both parameters are able to be adjusted i.e. Scalar 1, 2, 3 or Groups 1, 2 , 3, etc..

This is an easy way to ensure that not only are you getting the sharpest images, but also control where the image distribution is coming from.

The second script, filter colmap data is where things get really cool. Once COLMAP has been run from the first script, you’re able to prune the image count down further, while still retaining the camera poses from COLMAP. This step will select the sharpest images based upon your input and allows for a more manageable VRAM load. This is a super helpful feature, depending on your GPU and its VRAM capacity.

This runs with overlapping dependencies as nerfstudio, so if you’ve already built and activated a nerfstudio environment, you can run it straight away! The code is MIT Licensed, so it is accessible for everyone to utilize. I love seeing extensions of the community that create methods to make NeRFs more accessible.

If you’re developing or have already created something that benefits the NeRF/radiance field community, please get in touch! I’d love to feature it.