Depth Anything 3 Technical Reference¶
spatialhub.models.depth_anything_3 provides an ONNX Runtime adapter for Depth Anything 3 (DA3), a foundation model series supporting monocular relative and metric depth estimation, multi-view camera pose alignment, and nested dual-model scale projection.
Supported Model Presets & Series¶
The DepthAnything3Adapter accepts official model presets defined in MODEL_REGISTRY via model_name:
DA3 Any-View Foundation Series¶
Foundation models supporting single-image and multi-view sequences (\(N \ge 1\)), joint depth prediction, and relative camera pose estimation:
Model Preset (model_name) |
ONNX File | Parameter Count | Backbone Architecture | Primary Operational Role |
|---|---|---|---|---|
"da3_small" |
da3_small.onnx |
~25M | ViT-Small (DINOv2) | High-throughput, real-time edge processing and low-latency robotics. |
"da3_base" (Default) |
da3_base.onnx |
~98M | ViT-Base (DINOv2) | Balanced spatial accuracy and computational throughput. |
"da3_large" |
da3_large.onnx |
~335M | ViT-Large (DINOv2) | High-fidelity depth mapping and dense multi-view geometry. |
"da3_giant" |
da3_giant.onnx |
~1.35B | ViT-Giant (DINOv2) | Flagship foundation model with maximum geometric precision. |
Specialized Monocular Variants¶
Single-image models fine-tuned for specific monocular tasks:
Model Preset (model_name) |
ONNX File | Parameter Count | Output Modality | Description |
|---|---|---|---|---|
"da3mono_large" |
da3mono_large.onnx |
~335M | Relative Depth + Sky Mask | High-resolution monocular relative depth with sky probability estimation. |
"da3metric_large" |
da3metric_large.onnx |
~335M | Metric Depth (Meters) | Direct absolute metric depth prediction using camera focal length scaling. |
Nested Dual-Model Series¶
Combines the high-frequency geometric detail of an Any-View model with the physical scale of da3metric_large via least-squares scale-and-shift alignment:
Model Preset (model_name) |
Primary Model | Metric Reference | Primary Operational Role |
|---|---|---|---|
"da3nested_small_large" |
da3_small.onnx |
da3metric_large.onnx |
Lightweight metric estimation with high frame throughput. |
"da3nested_base_large" |
da3_base.onnx |
da3metric_large.onnx |
Balanced detail resolution and physical metric projection. |
"da3nested_large_large" |
da3_large.onnx |
da3metric_large.onnx |
High-fidelity dense metric reconstruction. |
"da3nested_giant_large" |
da3_giant.onnx |
da3metric_large.onnx |
Maximum visual fidelity projected to absolute metric units. |
Preprocessing & Post-processing Flow¶
Depth Anything 3 processes an input sequence of \(N\) images, producing aligned depth maps, confidence masks, and optional camera trajectory transformations.
graph TD
A["Input Images (N views)"] --> B["Preprocessing: Resize (divisible by 14) & ImageNet Normalize"]
B --> C["ONNX Runtime Forward Pass"]
C --> D["Extract Outputs: depth, conf, sky, extrinsics, intrinsics"]
D --> E{"Model Architecture?"}
E -- "da3metric_large" --> F["Direct Metric Output (Physical Meters)"]
E -- "da3nested_*_large" --> G["Least-Squares Scale Alignment against Metric Head"]
E -- "da3_* any-view / mono" --> H["Relative Disparity Prediction"]
F --> I{"Input Extrinsics Provided?"}
G --> I
H --> I
I -- "No (extrinsics=None)" --> J["Return Predicted Camera Trajectory in Canonical Frame"]
I -- "Yes (N=1)" --> K["Adopt Reference World Pose Directly"]
I -- "Yes (N=2)" --> L["Scale Depth by Camera Baseline Ratio & Adopt Reference Pose"]
I -- "Yes (N>=3)" --> M["Sim(3) Umeyama Alignment (Rescale Depth & Align Trajectory)"]
J --> N["DepthPredictionResult"]
K --> N
L --> N
M --> NDimension Resizing & Normalization¶
Inputs are scaled to spatial dimensions \((W, H)\) that are positive integer multiples of the ViT patch size \(14\):
\[
W = 14 \cdot \left\lfloor \frac{W_{\text{target}}}{14} \right\rfloor, \qquad H = 14 \cdot \left\lfloor \frac{H_{\text{target}}}{14} \right\rfloor
\]
Pixel color channels are normalized using standard ImageNet mean \(\mu\) and standard deviation \(\sigma\):
\[
x_{\text{norm}} = \frac{\frac{x}{255.0} - \mu}{\sigma}, \qquad \mu = [0.485, 0.456, 0.406], \quad \sigma = [0.229, 0.224, 0.225]
\]
Depth Output Modalities & Scale Projection¶
Depth Anything 3 supports three depth output modes:
- Relative Depth (
da3_small,da3_base,da3_large,da3_giant,da3mono_large): Foundation models output non-metric relative depth maps.da3mono_largeadditionally outputs a binary-thresholded sky segmentation mask \(M_{\text{sky}}\) where non-zero pixels represent sky regions. - Standalone Metric Depth (
da3metric_large): Directly predicts dense depth maps in physical meters without requiring post-hoc focal normalization. - Nested Dual-Model Metric Projection (
da3nested_*_large): Combines the high-frequency geometric detail of an Any-View model (\(D_{\text{rel}}\)) with the absolute physical scale ofda3metric_large(\(D_{\text{metric}}\)). A closed-form linear least-squares scale parameter \(s^*\) is calculated over high-confidence, non-sky pixels \(\Omega\):
\[
\Omega = \{ (u, v) \mid C_{\text{rel}}(u, v) > \tau_{\text{conf}} \land M_{\text{sky}}(u, v) \le 0.5 \}
\]
\[
s^* = \arg\min_s \sum_{(u, v) \in \Omega} \left( s \cdot D_{\text{rel}}(u, v) - D_{\text{metric}}(u, v) \right)^2 = \frac{\sum_{(u, v) \in \Omega} D_{\text{rel}}(u, v) \cdot D_{\text{metric}}(u, v)}{\sum_{(u, v) \in \Omega} D_{\text{rel}}(u, v)^2}
\]
\[
D_{\text{aligned}} = s^* \cdot D_{\text{rel}}
\]
Camera Calibration & Trajectory Alignment¶
The adapter handles camera intrinsics and extrinsics dynamically based on whether input poses are provided:
+----------------------------------------------------------------------------------------------------+
| CAMERA PARAMETER FLOW |
+----------------------------------------------------------------------------------------------------+
| Input Parameter | Execution Mode | Behavior & Depth Scaling | Returned Result |
+-----------------------+--------------------+-----------------------------+-------------------------+
| extrinsics = None | Unposed Sequence | Camera decoder estimates | result.extrinsics |
| | | relative camera poses | contains predicted |
| | | in internal canonical | trajectory (N, 4, 4) |
| | | coordinate frame. | |
+-----------------------+--------------------+-----------------------------+-------------------------+
| extrinsics provided | Posed Single-View | No scaling performed. | result.extrinsics |
| (N = 1) | (N = 1) | Input world pose adopted. | matches input pose. |
+-----------------------+--------------------+-----------------------------+-------------------------+
| extrinsics provided | Posed Two-View | Scale computed from camera | Depth scaled by 1/s; |
| (N = 2) | (N = 2) | center baseline ratio: | result.extrinsics |
| | | s = ||ΔC_pred|| / ||ΔC_gt||| matches input poses. |
+-----------------------+--------------------+-----------------------------+-------------------------+
| extrinsics provided | Posed Multi-View | Umeyama Sim(3) fit aligns | Depth scaled by 1/s; |
| (N >= 3) | (N >= 3) | predicted trajectory to GT | result.extrinsics |
| | | (with RANSAC if N >= 10). | matches input poses. |
+-----------------------+--------------------+-----------------------------+-------------------------+
| intrinsics = None | Uncalibrated | Pinhole focal & principal | result.intrinsics |
| | | points estimated by head. | contains (N, 3, 3). |
+-----------------------+--------------------+-----------------------------+-------------------------+
| intrinsics provided | Calibrated | Model conditioned on input | result.intrinsics |
| | | calibration parameters. | contains (N, 3, 3). |
+----------------------------------------------------------------------------------------------------+
Trajectory Alignment Modes¶
When reference camera extrinsics \(T_{\text{ref}} = [R \mid t] \in \mathrm{SE}(3)\) are provided:
Single-View (\(N = 1\)): Direct assignment of the reference world coordinate frame:
\[
T_{\text{aligned}} = T_{\text{ref}}
\]
Two-View (\(N = 2\)): True 3D camera optical centers in world coordinates \(C = T_{\text{w2c}}^{-1}[:3, 3]\) are computed for both reference and predicted poses. Depth and translation are scaled by the baseline distance ratio:
\[
s_{\text{baseline}} = \frac{\|C_{\text{pred}, 2} - C_{\text{pred}, 1}\|_2}{\|C_{\text{ref}, 2} - C_{\text{ref}, 1}\|_2}, \qquad D_{\text{aligned}} = \frac{D_{\text{pred}}}{s_{\text{baseline}}}
\]
Multi-View (\(N \ge 3\)): A rigid similarity transformation \((R^*, t^*, s^*) \in \mathrm{Sim}(3)\) is computed via Umeyama SVD factorization (with optional RANSAC for outlier suppression on \(N \ge 10\) views):
\[
\min_{R \in \mathrm{SO}(3),\, t \in \mathbb{R}^3,\, s > 0} \sum_{i=1}^N \left\| s R C_{\text{pred}, i} + t - C_{\text{ref}, i} \right\|_2^2, \qquad D_{\text{aligned}} = \frac{D_{\text{pred}}}{s^*}
\]
Unposed Trajectory Evaluation (align_to_input_ext_scale=False): When evaluating raw camera decoder drift against ground truth without locking poses to the input, setting align_to_input_ext_scale=False outputs the Umeyama-aligned predicted trajectory \(T_{\text{aligned}}\) directly without overwriting with \(T_{\text{ref}}\).
Numerical Parity Verification¶
Evaluates numerical agreement between PyTorch reference checkpoints and the SpatialHub ONNX Runtime adapter on the HiRoom evaluation dataset at \(504 \times 504\) resolution across both Posed (input camera parameters supplied) and Unposed (camera trajectory estimated from visual features alone) evaluation sequences.
Any-View Foundation Series¶
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Conf MAE | Extrinsics Rot Error (PT vs ORT) | Extrinsics Trans Error (PT vs ORT) |
|---|---|---|---|---|---|---|---|
da3_small |
1 | 0.000096 | 0.001976 | 0.01% | 0.001233 | 0.0000° | 0.000000 |
| 2 | 0.001272 | 0.085482 | 0.04% | 0.014252 | 0.0000° | 0.000000 | |
| 4 | 0.004254 | 0.246895 | 0.05% | 0.000582 | 0.0000° | 0.000000 | |
da3_base |
1 | 0.000044 | 0.002224 | 0.00% | 0.003377 | 0.0000° | 0.000000 |
| 2 | 0.001764 | 0.081946 | 0.06% | 0.021171 | 0.0000° | 0.000000 | |
| 4 | 0.002598 | 0.281878 | 0.03% | 0.000913 | 0.0000° | 0.000000 | |
da3_large |
1 | 0.000050 | 0.002824 | 0.00% | 0.004860 | 0.0000° | 0.000000 |
| 2 | 0.002046 | 0.098485 | 0.06% | 0.016927 | 0.0000° | 0.000000 | |
| 4 | 0.006935 | 0.441951 | 0.09% | 0.001602 | 0.0000° | 0.000000 | |
da3_giant |
1 | 0.000188 | 0.005118 | 0.00% | 0.004523 | 0.0000° | 0.000000 |
| 2 | 0.001602 | 0.084178 | 0.04% | 0.021703 | 0.0000° | 0.000000 | |
| 4 | 0.020584 | 1.637156 | 0.23% | 0.008453 | 0.0000° | 0.000000 |
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Conf MAE | Extrinsics Rot Error (PT vs ORT) | Extrinsics Trans Error (PT vs ORT) | Trajectory Rot (ORT vs GT) | Trajectory Trans (ORT vs GT) |
|---|---|---|---|---|---|---|---|---|---|
da3_small |
1 | 0.000107 | 0.001337 | 0.01% | 0.001294 | 0.0000° | 0.000000 | N/A | N/A |
| 2 | 0.000261 | 0.034103 | 0.03% | 0.015273 | 0.0033° | 0.000038 | 100.2812° | 4.740358 | |
| 4 | 0.000205 | 0.026474 | 0.02% | 0.009382 | 0.0083° | 0.000190 | 4.7084° | 0.363254 | |
da3_base |
1 | 0.000040 | 0.002288 | 0.00% | 0.002276 | 0.0001° | 0.000001 | N/A | N/A |
| 2 | 0.000224 | 0.023873 | 0.02% | 0.021786 | 0.0081° | 0.000039 | 100.5857° | 4.721515 | |
| 4 | 0.000122 | 0.021366 | 0.02% | 0.017614 | 0.0097° | 0.000142 | 5.2677° | 0.379212 | |
da3_large |
1 | 0.000037 | 0.003703 | 0.00% | 0.008109 | 0.0001° | 0.000001 | N/A | N/A |
| 2 | 0.000286 | 0.076807 | 0.03% | 0.045860 | 0.0051° | 0.000089 | 100.5322° | 4.727690 | |
| 4 | 0.000241 | 0.055369 | 0.03% | 0.025738 | 0.0074° | 0.000181 | 2.0173° | 0.151687 | |
da3_giant |
1 | 0.000032 | 0.002410 | 0.00% | 0.002995 | 0.0001° | 0.000004 | N/A | N/A |
| 2 | 0.000365 | 0.062983 | 0.03% | 0.050428 | 0.0037° | 0.000053 | 100.4026° | 4.733258 | |
| 4 | 0.000165 | 0.093573 | 0.02% | 0.202900 | 0.0108° | 0.000096 | 0.6106° | 0.026799 |
Note
For \(N=2\) views, the two camera positions define a 1D line segment rather than a full 3D point cloud, so a unique 3D \(\mathrm{Sim}(3)\) rotation alignment cannot be geometrically constrained against ground truth. For \(N \ge 3\) views, full 3D Umeyama \(\mathrm{Sim}(3)\) alignment operates, demonstrating monotonic trajectory recovery improvements as model capacity scales (\(4.71^\circ \rightarrow 0.61^\circ\)).
Specialized Monocular Series¶
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Output Modality |
|---|---|---|---|---|---|
da3mono_large |
1 | 0.000062 | 0.008396 | 0.01% | Relative Depth + Sky Mask |
da3metric_large |
1 | 0.000120 | 0.008575 | 0.00% | Metric Depth (Meters) |
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Output Modality |
|---|---|---|---|---|---|
da3mono_large |
1 | 0.000063 | 0.009497 | 0.01% | Relative Depth + Sky Mask |
da3metric_large |
1 | 0.000119 | 0.008165 | 0.00% | Metric Depth (Meters) |
Nested Dual-Model Series¶
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Conf MAE | Extrinsics Rot Error (PT vs ORT) | Extrinsics Trans Error (PT vs ORT) |
|---|---|---|---|---|---|---|---|
da3nested_small_large |
1 | 0.000593 | 0.006289 | 0.02% | 0.001255 | 0.0000° | 0.000000 |
da3nested_base_large |
1 | 0.000351 | 0.007988 | 0.01% | 0.003424 | 0.0000° | 0.000000 |
da3nested_large_large |
1 | 0.000620 | 0.008171 | 0.02% | 0.004896 | 0.0000° | 0.000000 |
da3nested_giant_large |
1 | 0.004001 | 0.019733 | 0.13% | 0.005890 | 0.0000° | 0.000000 |
| Model Variant | Views (\(N\)) | Depth MAE | Depth Max Diff | Relative Error (%) | Conf MAE | Extrinsics Rot Error (PT vs ORT) | Extrinsics Trans Error (PT vs ORT) |
|---|---|---|---|---|---|---|---|
da3nested_small_large |
1 | 0.000566 | 0.006497 | 0.01% | 0.001228 | 0.0000° | 0.000001 |
da3nested_base_large |
1 | 0.000991 | 0.008332 | 0.03% | 0.002253 | 0.0001° | 0.000002 |
da3nested_large_large |
1 | 0.000258 | 0.014519 | 0.01% | 0.008264 | 0.0001° | 0.000002 |
da3nested_giant_large |
1 | 0.001533 | 0.010335 | 0.05% | 0.006238 | 0.0001° | 0.000012 |
To execute local parity testing:
# Verify both posed and unposed sequence parity
uv run tools/benchmark/parity_depth_anything_3.py \
--variant all \
--mode both \
--view-counts 1 2 4 \
--output-file .profile/parity_da3.md
Performance Benchmarks¶
Latencies, throughput, host process RAM deltas, and peak device VRAM footprint measured on the HiRoom evaluation dataset at \(504 \times 504\) resolution.
Test Environment & Hardware Specification¶
- Operating System: Windows 11 (64-bit)
- GPU: NVIDIA GeForce RTX (8,192 MB GDDR6 physical VRAM, CUDA Execution Provider)
- CPU: Multi-core x86_64 host processor (
CPUExecutionProvider) - ONNX Runtime: 1.20.1 with native CUDA driver synchronization (
cuCtxSynchronize) and memory telemetry (cuMemGetInfo_v2) - Evaluation Dataset: HiRoom multi-view benchmark scene (
504 x 504spatial resolution)
Any-View Foundation Series¶
| Model Variant | Views (\(N\)) | Preprocess (ms) | Inference (ms) | Postprocess (ms) | End-to-End Latency (ms) | Throughput (FPS) | Peak Host RAM | Peak VRAM |
|---|---|---|---|---|---|---|---|---|
da3_small |
1 | 13.02 +/- 0.44 | 39.74 +/- 1.43 | 0.00 +/- 0.00 | 53.08 +/- 3.32 | 18.8 | +1.9 MB | +478.0 MB |
| 2 | 27.24 +/- 1.08 | 68.66 +/- 2.87 | 0.65 +/- 0.04 | 98.78 +/- 3.22 | 10.1 | +3.8 MB | +1226.0 MB | |
| 4 | 55.83 +/- 3.94 | 161.14 +/- 5.44 | 1.17 +/- 0.12 | 212.83 +/- 5.19 | 4.7 | +7.8 MB | +3644.0 MB | |
da3_base |
1 | 14.05 +/- 0.45 | 86.54 +/- 5.87 | 0.01 +/- 0.00 | 133.84 +/- 56.90 | 7.5 | +2.7 MB | +932.0 MB |
| 2 | 64.89 +/- 7.83 | 347.98 +/- 69.96 | 1.17 +/- 0.25 | 353.12 +/- 67.34 | 2.8 | +3.8 MB | +2430.0 MB | |
| 4 | 121.48 +/- 12.98 | 1543.72 +/- 151.22 | 1.90 +/- 0.41 | 2221.34 +/- 131.51 | 0.5 | +7.8 MB | +6485.0 MB | |
da3_large |
1 | 15.41 +/- 0.43 | 329.05 +/- 39.97 | 0.00 +/- 0.00 | 387.64 +/- 40.25 | 2.6 | +3.0 MB | +1724.0 MB |
| 2 | 34.17 +/- 2.46 | 1561.09 +/- 170.85 | 1.83 +/- 0.18 | 1794.84 +/- 33.33 | 0.6 | +3.1 MB | +3530.0 MB | |
da3_giant |
1 | 18.27 +/- 1.63 | 1037.12 +/- 110.10 | 0.00 +/- 0.00 | 1553.26 +/- 174.39 | 0.6 | +0.8 MB | +1945.0 MB |
| Model Variant | Views (\(N\)) | Preprocess (ms) | Inference (ms) | Postprocess (ms) | End-to-End Latency (ms) | Throughput (FPS) | Peak Host RAM |
|---|---|---|---|---|---|---|---|
da3_small |
1 | 14.86 +/- 1.48 | 901.48 +/- 153.93 | 0.01 +/- 0.00 | 937.77 +/- 119.60 | 1.1 | +0.0 MB |
| 2 | 37.16 +/- 4.48 | 2179.39 +/- 372.05 | 1.45 +/- 0.54 | 2220.46 +/- 130.41 | 0.5 | +0.0 MB | |
| 4 | 111.65 +/- 10.09 | 5075.71 +/- 479.09 | 3.08 +/- 0.63 | 5061.65 +/- 329.81 | 0.2 | +5.1 MB | |
da3_base |
1 | 15.67 +/- 1.01 | 2843.99 +/- 271.45 | 0.02 +/- 0.01 | 2950.36 +/- 209.83 | 0.3 | +0.3 MB |
| 2 | 70.62 +/- 7.38 | 5223.72 +/- 154.85 | 1.20 +/- 0.28 | 5422.77 +/- 531.76 | 0.2 | +0.0 MB |
Specialized Monocular Series¶
| Model Variant | Views (\(N\)) | Preprocess (ms) | Inference (ms) | Postprocess (ms) | End-to-End Latency (ms) | Throughput (FPS) | Peak Host RAM | Peak VRAM |
|---|---|---|---|---|---|---|---|---|
da3mono_large |
1 | 19.07 +/- 1.26 | 653.90 +/- 59.21 | 0.32 +/- 0.01 | 794.54 +/- 39.21 | 1.3 | +2.7 MB | +1400.0 MB |
da3metric_large |
1 | 17.75 +/- 0.62 | 541.70 +/- 72.42 | 0.66 +/- 0.04 | 725.21 +/- 98.15 | 1.4 | +0.1 MB | +1400.0 MB |
| Model Variant | Views (\(N\)) | Preprocess (ms) | Inference (ms) | Postprocess (ms) | End-to-End Latency (ms) | Throughput (FPS) | Peak Host RAM |
|---|---|---|---|---|---|---|---|
da3mono_large |
1 | 21.11 +/- 0.64 | 5914.68 +/- 174.94 | 0.66 +/- 0.02 | 6274.85 +/- 813.43 | 0.2 | +0.0 MB |
da3metric_large |
1 | 19.16 +/- 1.23 | 7622.34 +/- 449.96 | 1.08 +/- 0.05 | 7223.87 +/- 121.01 | 0.1 | +0.1 MB |
Nested Dual-Model Series (Single-View, CUDA)¶
Evaluates sequential dual-session forward execution (main_session + da3metric_large) with least-squares scale alignment (\(N=1\)):
| Variant | Primary Backbone | Metric Backbone | Inference Latency (ms) | Postprocess (ms) | End-to-End Latency (ms) | Throughput (FPS) | Peak Host RAM | Peak VRAM |
|---|---|---|---|---|---|---|---|---|
da3nested_small_large |
ViT-Small (~25M) | ViT-Large (~335M) | 835.85 +/- 79.97 | 10.67 +/- 0.40 | 1073.51 +/- 110.41 | 0.9 | +1.0 MB | +1594.0 MB |
da3nested_base_large |
ViT-Base (~98M) | ViT-Large (~335M) | 1297.79 +/- 58.76 | 13.24 +/- 0.82 | 1327.52 +/- 62.49 | 0.8 | +2.0 MB | +2048.0 MB |
da3nested_large_large |
ViT-Large (~335M) | ViT-Large (~335M) | 2078.13 +/- 155.49 | 11.03 +/- 0.26 | 3574.00 +/- 493.71 | 0.3 | +0.1 MB | +2844.0 MB |
da3nested_giant_large * |
ViT-Giant (~1.35B) | ViT-Large (~335M) | 83770.78 +/- 62867.91 | 71.32 +/- 1.73 | 135745.27 +/- 84610.80 | 0.01 | +3036.8 MB | +629.0 MB |
VRAM Footprint Note on da3nested_giant_large
The concurrent active parameter footprint of ViT-Giant (~5.4 GB weights) and ViT-Large (~1.4 GB weights) along with intermediate attention activation tensors exceeds the 8,192 MB physical VRAM capacity of the test GPU. On Windows WDDM, excess tensors are paged across the PCIe bus into host shared RAM, resulting in elevated inference latency. For dual-model ViT-Giant deployment, a hardware device with \(\ge 16\text{ GB}\) dedicated VRAM is recommended.
SpatialHub Adapter API & Usage¶
Single-View Relative Depth Estimation¶
import cv2
from spatialhub import DepthAnything3
# Initialize adapter with DA3 Base preset
with DepthAnything3(model_name="da3_base", providers=["CUDAExecutionProvider"]) as estimator:
result = estimator.estimate_depth(images=["room_view.jpg"])
# Visualize colorized depth map
colorized_depth = estimator.visualize(result.depth[0])
cv2.imwrite("depth_output.png", cv2.cvtColor(colorized_depth, cv2.COLOR_RGB2BGR))
Multi-View Trajectory Alignment & Depth Scaling¶
from spatialhub import DepthAnything3
with DepthAnything3(
model_name="da3_large",
align_to_input_ext_scale=True,
ransac_view_thresh=10,
providers=["CUDAExecutionProvider"],
) as estimator:
result = estimator.estimate_depth(
images=["view_01.jpg", "view_02.jpg", "view_03.jpg", "view_04.jpg"],
extrinsics=reference_extrinsics,
)
print("Estimated depth shape:", result.depth.shape)
print("Aligned camera extrinsics shape:", result.extrinsics.shape)
Direct Metric Depth Estimation¶
from spatialhub import DepthAnything3
# Direct absolute metric depth prediction (meters)
with DepthAnything3(model_name="da3metric_large", providers=["CUDAExecutionProvider"]) as estimator:
result = estimator.estimate_depth(images=["indoor_scene.jpg"])
print("Metric depth output range (meters):", result.depth.min(), result.depth.max())
Nested Dual-Model Metric Projection¶
from spatialhub import DepthAnything3
# Combine ViT-Base geometric detail with ViT-Large physical metric scale
with DepthAnything3(model_name="da3nested_base_large", providers=["CUDAExecutionProvider"]) as estimator:
result = estimator.estimate_depth(images=["indoor_scene.jpg"])
print("Metric depth output range (meters):", result.depth.min(), result.depth.max())
Tooling & Verification Commands¶
Export pretrained PyTorch Depth Anything 3 checkpoints to ONNX graphs:
uv run tools/export/export_depth_anything_3.py \
--variant base \
--output-folder onnx_weight \
--width 504 \
--height 504 \
--opset 18
| Parameter | Type | Default | Description |
|---|---|---|---|
--variant |
str |
"base" |
Target model variant (small, base, large, giant, mono_large, metric_large, or all). |
--checkpoint |
str |
None |
Path to pretrained checkpoint directory or Hugging Face repo ID (downloaded if omitted). |
--output-folder |
str |
onnx_weight |
Destination directory for exported .onnx model files. |
--width |
int |
504 |
Input image width in pixels (must be a multiple of 14, \(\ge 392\)). |
--height |
int |
504 |
Input image height in pixels (must be a multiple of 14, \(\ge 392\)). |
--opset |
int |
18 |
ONNX Operator Set version. |
Validate numerical depth and camera trajectory parity against PyTorch reference:
uv run tools/benchmark/parity_depth_anything_3.py \
--variant all \
--view-counts 1 2 4 \
--dataset-dir .cache/hiroom \
--output-file .profile/parity_da3.md
| Parameter | Type | Default | Description |
|---|---|---|---|
--variant |
str |
"all" |
Target model variant (small, base, large, giant, mono_large, metric_large, nested_*, or all). |
--view-counts |
list[int] |
[1, 2, 4] |
Sequence view counts to evaluate (\(N=1\) for single-view, \(N \ge 2\) for multi-view). |
--dataset-dir |
str |
None |
Path to extracted HiRoom benchmark dataset directory (defaults to .cache/hiroom). |
--model-dir |
str |
None |
Optional directory containing local .onnx model weight files. |
--output-file |
str |
None |
Optional file path to save Markdown parity summary table. |
Benchmark pipeline stage latency, throughput, and memory footprint:
uv run tools/benchmark/profile_depth_anything_3.py \
--variant da3_small \
--provider all \
--process-res 504 \
--view-counts 1 2 4 \
--warmup 3 \
--iterations 10 \
--dataset-dir .cache/hiroom \
--output-file .profile/profile_da3_small.md
| Parameter | Type | Default | Description |
|---|---|---|---|
--variant |
str |
"all" |
Model preset name to profile (da3_small, da3_base, etc., or all). |
--provider |
str |
"all" |
Execution provider filter (cuda, cpu, or all). |
--process-res |
list[int] |
[504] |
Spatial input resolutions to evaluate (must be multiples of 14). |
--view-counts |
list[int] |
[1, 2] |
View counts to evaluate per run (\(N=1\) for single-image, \(\ge 2\) for multi-view). |
--dataset-dir |
str |
None |
Path to extracted HiRoom dataset directory (defaults to .cache/hiroom). |
--model-dir |
str |
None |
Optional directory containing local .onnx weight files. |
--warmup |
int |
10 |
Number of unmeasured warmup iterations. |
--iterations |
int |
50 |
Number of timed measurement iterations. |
--output-file |
str |
None |
Markdown report path to incrementally append completed results. |
Returned Result Data Structure¶
Returns a DepthPredictionResult dataclass:
| Attribute | Type | Shape | Description |
|---|---|---|---|
image |
np.ndarray |
(N, H, W, 3) uint8 |
Input RGB image batch in native spatial resolution. |
depth |
np.ndarray |
(N, H, W) float32 |
Predicted depth maps (in physical meters or relative disparity). |
conf |
np.ndarray | None |
(N, H, W) float32 |
Confidence score maps normalized in [0.0, 1.0]. |
intrinsics |
np.ndarray | None |
(N, 3, 3) float32 |
Predicted or input camera intrinsic calibration matrices. |
extrinsics |
np.ndarray | None |
(N, 4, 4) float32 |
Estimated or aligned world-to-camera extrinsic matrices. |
depth_type |
str |
N/A | Scale identifier ("metric" or "relative"). |