Perception Algorithm Engineer

About XDOF

At XDOF, we're at an inflection point. Frontier AI labs are racing to build general-purpose robots, and high-quality training data has become the primary bottleneck.

We're building the foundation behind the foundation models—the data collection systems, operational infrastructure, exabyte-scale data warehouse, and software toolchain that enable our partners to accelerate the future of embodied AI.

About the Team

The Perception Algorithm Team transforms raw multimodal sensor data into high-quality robot training annotations.

You will work across the entire perception pipeline, from data collection through model deployment, including:

• Sensor calibration

• SLAM localization

• Human pose estimation

• Perception model training

• Embedded deployment

Your work will directly determine the quality ceiling of the training data used to build next-generation robotics foundation models.

Core Responsibilities

Human Pose Estimation

• Design and optimize hand pose estimation pipelines capable of accurate joint-angle extraction from teleoperation data

• Build full-body pose estimation systems for motion capture and teleoperation ground-truth annotation

• Research and apply markerless vision-based pose estimation methods to reduce data collection costs

• Fuse pose estimation outputs with robot joint-angle data to generate high-quality, consistent training annotations

Robot Perception & Calibration

• Design and maintain intrinsic and extrinsic calibration pipelines for multi-camera systems, including factory calibration and online recalibration

• Develop visual SLAM (V-SLAM) systems supporting real-time localization and scene reconstruction for data collection platforms

• Implement hand-eye calibration between camera systems and robotic end-effectors

• Develop temporal synchronization solutions across multimodal sensors, including:

  • Cameras

  • IMUs

  • Data gloves

  • Force sensors

Perception Model Training & Deployment

• Train and improve perception models for:

  • Object detection

  • Instance segmentation

  • 6DoF object pose estimation

• Optimize inference using TensorRT and CUDA for real-time deployment on embedded robotics platforms

• Develop custom CUDA kernels for low-level perception acceleration

• Design evaluation frameworks that continuously measure perception quality and its impact on downstream training data

End-to-End Data Collection Pipeline

• Design automated annotation pipelines that convert raw sensor data into structured training labels

• Build automated quality assurance systems that detect:

  • Anomalous frames

  • Failed demonstrations

  • Sensor dropouts

• Collaborate with machine learning engineers and data infrastructure teams to ensure perception outputs integrate seamlessly with downstream Vision-Language-Action (VLA) model training

• Establish feedback loops connecting perception accuracy to model performance, continuously improving annotation quality

Requirements

Must-Have Qualifications

• 5+ years of industry experience in robotics perception or computer vision

• Strong foundation in 3D vision, including:

  • Stereo vision

  • Structured-light cameras

  • 3D reconstruction

• Experience developing or deploying SLAM systems such as:

  • ORB-SLAM

  • VINS-Mono

  • FastLIO

  • Similar visual SLAM frameworks

• Hands-on experience with human pose estimation, including:

  • Hand pose estimation (MediaPipe, MANO)

  • Full-body pose estimation (OpenPose, SMPLify, or similar)

• Experience training, tuning, and evaluating deep learning perception models

• TensorRT deployment experience for real-time inference on embedded platforms such as NVIDIA Jetson or Horizon Robotics hardware

• CUDA programming skills with the ability to write or debug custom CUDA kernels

• Strong proficiency in C++ and Python

• Experience developing robotics applications using ROS or ROS2

• Proficiency using AI coding agents within the software development workflow

Nice-to-Have Qualifications

• Experience with 6DoF object pose estimation methods such as:

  • FoundPose

  • FoundationPose

  • GDR-Net

• Familiarity with Gaussian Splatting or NeRF for scene reconstruction or synthetic data generation

• Experience with robot manipulation or teleoperation systems

• End-to-end experience building automated annotation pipelines or ground-truth generation systems

• Published research in robotics perception, pose estimation, or computer vision

What We Offer

• Direct involvement in one of the most important technical challenges in embodied AI: producing high-quality robot training data

• The opportunity to work alongside world-class robotics engineers and machine learning researchers

• Access to proprietary robotics hardware, including:

  • Humanoid robots

  • Multi-camera arrays

  • Data gloves

• A fast-paced, high-autonomy, zero-to-one engineering environment where you'll help define the future of robotic perception