A pure-Julia stereo visual odometry library with a CUDA.jl-accelerated frontend. Designed to be camera-agnostic — KITTI is the validation harness, not the algorithm boundary.
Mean t% 1.021 across all 11 KITTI odometry sequences (00–10) at 33 fps median on an RTX 4050. Validated bit-identical CPU↔GPU descriptor pipeline (6.75M per-pixel descriptor comparisons, zero mismatches).
| seq | t% | rot deg/m | ATE (m) | drift (m) |
|---|---|---|---|---|
| 00 | 0.698 | 0.00286 | 4.89 | 4.23 |
| 01 | 1.210 | 0.00303 | 11.99 | 14.82 |
| 02 | 0.909 | 0.00288 | 13.12 | 25.42 |
| 03 | 1.799 | 0.00416 | 5.98 | 10.83 |
| 04 | 0.612 | 0.00269 | 0.73 | 1.80 |
| 05 | 0.885 | 0.00350 | 7.04 | 20.09 |
| 06 | 0.967 | 0.00265 | 4.04 | 4.60 |
| 07 | 1.112 | 0.00728 | 3.72 | 6.53 |
| 08 | 1.262 | 0.00367 | 14.47 | 5.99 |
| 09 | 1.122 | 0.00281 | 8.49 | 13.80 |
| 10 | 0.654 | 0.00272 | 4.58 | 9.41 |
| mean | 1.021 | 0.00348 | 7.19 | 10.68 |
Per-frame median wall time: 30.3 ms with the default GPU descriptor pipeline (33 fps).
A complete real-time stereo VO frontend in pure Julia + CUDA.jl, end to end:
- GPU image preprocessing: Sobel structure tensor, λ_min field, peak detection, per-cell aggregation.
- GPU NCC matching: full-image stereo + temporal NCC with sub-pixel parabolic refinement, phase-correlation refinement, adaptive quality thresholds.
- GPU track-identity descriptor: 128-bit hierarchical chroma+texture descriptor with a velocity-adaptive Hamming gate. Bit-identical to a CPU reference (validated on 6.75M per-pixel comparisons).
- CPU joint stereo bundle adjustment: 6-DoF Levenberg-Marquardt with anisotropic KLT-tensor noise model, Cauchy IRLS, inverse-depth disparity residual.
-
CPU backend filter: forward-only adaptive Bayesian filter on the SE(3) twist. Combines a VAR(4) closed-form motion prior (blended with previous-frame twist via a
$\chi^2(6)$ disagreement gate) with the BA measurement, plus adaptive log-EMA covariance scales tuned online by NIS/NEES residual statistics. No backward smoothing pass — past poses are not revised in light of future evidence.
- Julia 1.10 – 1.12.
- A CUDA-capable NVIDIA GPU with a working CUDA.jl install. The NCC matching and descriptor frontend run entirely on the GPU; there is no CPU fallback in v0.1.0. Without a functional device,
process!raises a clear error. (using SimpleStereoVOitself works on CPU, so the module loads in CI.) - Reported timings (≈33 fps median) are on a laptop RTX 4050; they are hardware-dependent, not a floor.
The package is not yet in the Julia General registry. Clone and dev it:
using Pkg
Pkg.develop(path = "/path/to/SimpleStereoVO") # the cloned repo root
Pkg.instantiate()
using SimpleStereoVO # verify it loadsOr work inside the package environment directly:
git clone <repo-url> SimpleStereoVO
julia --project=SimpleStereoVO -e 'using Pkg; Pkg.instantiate(); using SimpleStereoVO'A self-contained demo that synthesizes a stereo pair (no dataset needed) and runs the full pipeline:
julia --project=. examples/minimal_stereo_example.jlusing SimpleStereoVO
# Rectified stereo camera: focal lengths + principal point (px) + baseline (m)
cam = StereoCamera(718.856, 718.856, 607.193, 185.216, 0.5371)
# Camera-aware VOConfig auto-derives the disparity range from your depth range.
# Defaults assume KITTI-scale outdoor; override for indoor / drone:
cfg = VOConfig(cam; min_depth_m = 0.5, max_depth_m = 80.0)
frontend = SimpleStereoVOFrontend(cam; cfg = cfg)
# Per-frame loop. `prepare_stereo_frame` uploads a rectified left/right image
# pair (Matrix of luma or color) to the GPU and builds the pyramid.
for (left_img, right_img, t) in your_stereo_stream
frame = prepare_stereo_frame(left_img, right_img, cfg, t)
evidence = process!(frontend, frame)
# `se3_point_reprojection` is `nothing` on the first frame and on any frame
# where the solve did not produce a pose (too few inliers) — always check.
lin = evidence.factor.se3_point_reprojection
if lin !== nothing
ξ = lin.linearization_point # 6-twist (ω, t), cur-relative-to-prev
# ...integrate ξ, fuse with IMU, render, etc.
end
endSee SimpleStereoVO.KittiHarness for a worked example that builds StereoFrames
from KITTI image files.
The main SimpleStereoVO module exports only camera-agnostic algorithm symbols:
- Camera + frame data:
StereoCamera,PreparedImage,StereoFrame - VO pipeline:
VOConfig,SimpleStereoVOFrontend,process!,reset!,set_backend_prior!,MotionEvidence - Joint stereo BA (callable standalone):
MatchObservation,JointBAOptions,JointBAResult,solve_motion_ba,solve_motion_ba_decoupled - Backend filter:
AdaptiveOdometryConfig,init_adaptive_odometry,update_adaptive_odometry!,best_se3_predictions, ... - Geometry primitives:
pixel_to_bearing,stereo_triangulate,relative_pose_to_xi,xi_to_relative_pose,compose_pose,integrate_se3 - Trajectory metrics (camera-agnostic):
ate,rmse,gaussian_nll,calibration_error
Three sub-modules contain non-core functionality:
SimpleStereoVO.KittiHarness— KITTI dataset loading, segment-error evaluation, pose-file I/O.SimpleStereoVO.Diagnostics— process-timing instrumentation, BA replay-dump format.SimpleStereoVO.Descriptors— track-identity descriptor internals (used automatically whenDESC_GATE_MODE != :off, exposed for diagnostics).
Download the KITTI odometry benchmark from the KITTI Vision Benchmark Suite (registration required). You need two of its packages:
data_odometry_color— the color stereo images. The eval defaults to the color pairimage_2(left) /image_3(right), and the track-identity descriptor uses chroma channels, so the grayscale package (image_0/image_1) is not a drop-in substitute.data_odometry_poses— the ground-truth poses for sequences 00–10.
Unpack them into a single root so the layout is:
$KITTI_ROOT/
├── sequences/
│ └── 00/
│ ├── image_2/ left color frames (000000.png, …)
│ ├── image_3/ right color frames
│ └── calib.txt P0..P3 projection matrices
└── poses/
└── 00.txt ground-truth poses (seqs 00–10 only)
The calibration files ship with the color package. Sequences 11–21 have no public ground-truth poses, so use 00–10 for evaluation.
Point KITTI_ROOT at that root and run from the package root:
KITTI_ROOT=/path/to/kitti_odometry/dataset \
KITTI_SEQS=00 MAX_FRAMES=5000 EVAL_TAG=my_run \
julia --project=. scripts/eval/run_eval.jlOutputs land in outputs/eval_<tag>/:
summary.md— t%, rot, ATE, drift, per-stage timingstrajectories_seq_XX.png— predicted vs GT trajectory + per-frame error traceseq_XX_framewise.csv— per-frame metrics- KITTI-format pose file (
seq_XX_pred_kitti.txt) for leaderboard submission
Multi-sequence cross-eval: KITTI_SEQS=00,01,02,...,10.
By default the frontend runs the track-identity gate with a velocity-adaptive Hamming threshold:
DESC_GATE_MODE=reject(track-identity gate active)DESC_GATE_THRESH=60,DESC_VEL_COEFF=5.0,DESC_VEL_REF=1.0— the gate threshold is60 + 5·max(0, ‖t_prior‖ − 1.0), loosening as speed risesDESC_GPU=1(full-GPU descriptor pipeline)
To disable the descriptor identity gate (the pre-gate baseline): DESC_GATE_MODE=off.
.
├── src/
│ ├── SimpleStereoVO.jl module root + submodule definitions
│ ├── algorithm/
│ │ ├── constants.jl
│ │ ├── geometry/
│ │ │ ├── motion_geometry.jl SE(3) primitives
│ │ │ ├── joint_ba.jl stereo BA solver
│ │ │ ├── joint_ba_window.jl windowed K≥2 BA
│ │ │ ├── inter_camera_rotation.jl per-keyframe rig estimator
│ │ │ └── ba_dump.jl replay dump format
│ │ ├── frontend/
│ │ │ ├── simple_stereo_vo_core.jl frontend top-level
│ │ │ ├── gpu_ncc.jl CUDA NCC kernels
│ │ │ ├── descriptor_v3.jl CPU descriptor reference
│ │ │ └── gpu_descriptor.jl full-GPU descriptor pipeline
│ │ └── backend/
│ │ ├── types.jl
│ │ ├── var4_prior.jl VAR(4) closed-form prior
│ │ └── filter.jl adaptive forward filter
│ └── harness/
│ ├── kitti_dataset.jl KITTI sample / pose loading
│ ├── kitti_oxts.jl OXTS IMU export
│ ├── metrics_utils.jl ATE / NLL / KITTI segment errors
│ └── simple_stereo_vo_kitti.jl StereoFrame from KITTI image files
│
├── examples/
│ └── minimal_stereo_example.jl self-contained synthetic demo (no KITTI)
├── scripts/
│ ├── eval/ KITTI eval entry point + framewise metric tools
│ ├── diagnostics/ bit-identity verify + per-component probes
│ ├── benchmark/ frontend throughput profiling
│ └── tools/ OXTS/IMU CSV export
├── docs/
│ └── descriptor_spec.md descriptor bit-layout walkthrough
├── test/runtests.jl SE(3)/BA numeric checks (no GPU/KITTI)
├── outputs/ eval_<tag>/ artifacts (gitignored)
├── .github/workflows/ci.yml CPU load + numeric-check CI
├── Project.toml package manifest
├── CITATION.cff citation metadata
├── LICENSE MIT
└── README.md
Defaults assume KITTI-scale automotive scenes (~10 Hz, ~376×1241 px, ~0.5–80 m). Knobs you may need to scale for a different rig:
| Field | Coupling | When to override |
|---|---|---|
stereo_min/max_disparity |
per-pixel; tied to fx · baseline / Z |
different baseline or depth range |
min_depth_m, max_depth_m |
physical | different operating envelope |
temporal_search_x/y |
pixels of per-frame motion at typical speed | different fps or speed |
track_verify_radius_slope |
pixels per (m/frame translation) | different fps or m/s typical motion |
patch_radius, harris_window_radius |
pixels of corner scale / smoothing | different image resolution |
The two-argument constructor VOConfig(cam) automatically derives the disparity range from the camera intrinsics; the rest are documented in the VOConfig docstring.
- CUDA-only, GPU-required. The NCC matcher and descriptor frontend run on the GPU and there is no CPU fallback in v0.1.0 — only the descriptor has a CPU reference (used for the bit-identity check, not as a runnable pipeline). No ROCm / Metal path yet.
- Determinism. The descriptor path is bit-identical CPU↔GPU. The full VO pipeline is not bit-reproducible run-to-run: CUDA atomics in feature selection and Float32 λ_min tie-breaking can pick a different equally-scored peak. Ties are rare on natural imagery (typically sub-0.1% metric variation); for regression-testing, compare with a tolerance (e.g. t% within ±0.01%) rather than exact equality.
- Not SOTA on KITTI. Mean t% 1.02 is roughly mid-pack on the leaderboard; the contribution here is the open Julia + CUDA.jl reference implementation and the documented numerical-equivalence story, not best-in-class accuracy.
- Validated on KITTI only. The library is camera-agnostic by design but has not been tested on non-automotive platforms (indoor robot, UAV, off-road).
- No loop closure. Pure visual odometry; no place-recognition or pose-graph optimization.
- Persistent yaw bias. The information-matrix diagnostic (
scripts/diagnostics/yaw_tx_aliasing_diag.jl) shows a (ω_y, t_x) Hessian aliasing of ≈−0.9 posterior correlation. A structural fix requires a residual-form change (bearing + log-depth instead of pixel + disparity) and is a v2 followup.
- Main runner:
scripts/eval/run_eval.jl - Frontend core:
src/algorithm/frontend/simple_stereo_vo_core.jl - Joint BA solver:
src/algorithm/geometry/joint_ba.jl - GPU descriptor pipeline:
src/algorithm/frontend/gpu_descriptor.jl - Descriptor spec:
docs/descriptor_spec.md - Diagnostic suite:
scripts/diagnostics/descriptor_gpu_verify.jl,scripts/diagnostics/descriptor_gpu_exhaustive.jl,scripts/diagnostics/yaw_tx_aliasing_diag.jl
If you use SimpleStereoVO in academic work, please cite it (see CITATION.cff):
@software{habacivch2026simplestereovo,
author = {Habacivch, Will},
title = {{SimpleStereoVO}: A Pure-Julia + CUDA.jl Stereo Visual Odometry
Reference with Bit-Identical Multi-Architecture Numerics},
year = {2026},
version = {0.1.0},
license = {MIT},
}MIT — see LICENSE.