When we covered NVIDIA 3DGRUT’s NCore v4 dataset support in March, NCore appeared as a format, a standardized way to describe multi sensor capture data that NVIDIA’s reconstruction pipelines could consume. NVIDIA’s Spatial Intelligence Lab has open sourced NCore itself, releasing the platform that defines the format, the associated tooling, and the SDK that ties it together.
NCore is NVIDIA’s answer to a persistent problem in real world 3D reconstruction of sensors not agreeing with one another. A typical capture rig for autonomous vehicle research or outdoor scene reconstruction combines lidar, multiple cameras in pinhole, fisheye, and equirectangular configurations, GNSS and IMU positioning data, and sometimes radar. Each sensor has its own calibration, coordinate system, and data format. Getting that heterogeneous data into a reconstruction pipeline requires significant per project engineering. NCore standardizes the interface so that pipelines like NuRec, which reconstructs radiance fields from multi-sensor inputs, don’t have to reinvent that glue layer for every dataset.
The open source release includes the NCore data model (the format specification itself), a C++ SDK for reading and writing NCore datasets, Python bindings, and a set of tools for sensor intrinsic and extrinsic calibration. The calibration tooling handles the geometric relationships between sensors and each sensor’s internal parameters in a unified way.
NuRec, which powers the AlpaSim reconstruction pipeline NVIDIA showcased at GTC, uses NCore as its native data format. Open sourcing NCore means that researchers and teams building their own Gaussian splat reconstruction pipelines can now feed NuRec, or other NVIDIA tools, without needing proprietary capture hardware or first party dataset formats. The format is generic enough to accommodate a wide range of capture setups. The NVIDIA implementation details are in the SDK.
Large scale outdoor reconstruction requires exactly the kind of multi sensor fusion NCore enables. Single camera capture works well for rooms and objects. For anything at street scale, LiDAR depth and multi camera coverage become load bearing, and the tooling to fuse those sources has been the bottleneck.
NVIDIA’s move to open source NCore is consistent with a recent pattern visible across their spatial AI work to release the infrastructure layer openly. NCore being open means the community can build interoperable capture pipelines. The repository is available on GitHub under the NVIDIA organization, but can also be found in gsplat and 3DGRUT.
New to Gaussian Splatting? Start here
