EfficientLoFTR Technical Reference¶

spatialhub.models.efficient_loftr provides an ONNX Runtime adapter for EfficientLoFTR, a semi-dense local feature matching model using sparse transformers.

Supported Model Variants¶

The EfficientLoFTRAdapter accepts two model precision variants via model_type:

Model Variant (`model_type`)	ONNX File	Description	Target Use Case
`"full"` (Default)	`eloftr_outdoor_full.onnx`	Full precision semi-dense feature matching model.	Maximum matching precision & robust keypoint coverage.
`"opt"`	`eloftr_outdoor_opt.onnx`	Optimized/reparameterized lightweight model variant.	High throughput, lower memory footprint, and edge processing.

Preprocessing & Post-processing Flow¶

EfficientLoFTR matches coarse-to-fine keypoints across image pairs without requiring PyTorch during inference.

graph TD
    A["Image Pair (A, B): Paths / Arrays"] --> B["Aspect Ratio Downscaling (max_dim)"]
    B --> C["Multiple-of-32 Dimension Alignment"]
    C --> D["Grayscale Conversion & Normalization"]
    D --> E["Bottom-Right Zero Padding to Shared (W_pad, H_pad)"]
    E --> F["ONNX Runtime Forward Pass"]
    F --> G["Filter Matches Outside Unpadded Boundaries"]
    G --> H["Project Coordinates to Native Resolution"]
    H --> I["MatchResult"]

Maximum Dimension Scaling¶

Given an input image with native dimensions \((W_{\text{orig}}, H_{\text{orig}})\) and optional maximum dimension limit \(D_{\max}\) (max_dim), dimensions are scaled preserving aspect ratio:

\[ (W_{\text{scaled}}, H_{\text{scaled}}) = \begin{cases} \left(\left\lfloor W_{\text{orig}} \cdot \frac{D_{\max}}{\max(W_{\text{orig}}, H_{\text{orig}})} \right\rfloor,\; \left\lfloor H_{\text{orig}} \cdot \frac{D_{\max}}{\max(W_{\text{orig}}, H_{\text{orig}})} \right\rfloor\right) & \text{if } D_{\max} \text{ and } \max(W_{\text{orig}}, H_{\text{orig}}) > D_{\max} \\[8pt] (W_{\text{orig}}, H_{\text{orig}}) & \text{otherwise} \end{cases} \]

Multiple-of-32 Alignment & Normalization¶

Spatial dimensions are aligned to the nearest lower multiples of 32 required by sparse transformer downsampling, and pixel values are normalized to \([0, 1]\) float32 tensors of shape \((1, 1, H, W)\):

\[ W = \max\left(32,\ \left\lfloor\frac{W_{\text{scaled}}}{32}\right\rfloor \times 32\right), \qquad H = \max\left(32,\ \left\lfloor\frac{H_{\text{scaled}}}{32}\right\rfloor \times 32\right) \]

Bottom-Right Batch Padding¶

For an image pair \((A, B)\) with individual aligned dimensions \((W_a, H_a)\) and \((W_b, H_b)\), both tensors are bottom-right zero-padded to shared maximum spatial dimensions \((W_{\text{pad}}, H_{\text{pad}})\):

\[ W_{\text{pad}} = \max(W_a, W_b), \qquad H_{\text{pad}} = \max(H_a, H_b) \]

Post-processing: Boundary Filtering & Coordinate Projection¶

Matches \((P_0, P_1)\) detected inside bottom-right zero-padded regions are filtered out using boundary mask \(V\):

\[ V = \{ (P_0, P_1) \mid x_0 < W_a \land y_0 < H_a \land x_1 < W_b \land y_1 < H_b \} \]

Valid raw coordinates \(P_{\text{raw}} = (x_{\text{raw}}, y_{\text{raw}})\) are projected back to native image dimensions \((W_{\text{orig}}, H_{\text{orig}})\):

\[ P_{\text{orig}} = \left(x_{\text{raw}} \cdot \frac{W_{\text{orig}}}{W},\; y_{\text{raw}} \cdot \frac{H_{\text{orig}}}{H}\right) \]

Numerical Parity Verification¶

Numerical parity verifies mathematical equivalence between the original PyTorch implementation and the exported ONNX Runtime CUDA graph.

In sparse transformer models, keypoint extraction relies on top-k sorting and parallel score reduction. Due to non-associative floating-point operations across parallel GPU threads, PyTorch and ONNX Runtime may produce keypoints in slightly different array order. SpatialHub embeds every matched point pair into a 4D spatial vector \((x_0, y_0, x_1, y_1)\) and resolves correspondences using KD-tree (scipy.spatial.cKDTree) nearest-neighbor queries.

Parity was evaluated on 30 image pairs from the MegaDepth-1500 outdoor evaluation dataset (max_dim denotes the maximum spatial dimension scaling threshold):

Variant	Max Dim	Pairs	Mean Matches (PyTorch / ONNX)	Match Ratio	Confidence MAE	Confidence Max Diff
`full`	`640`	30	1091.4 / 1091.4	99.9%	0.000534	0.005604
	`768`	30	1552.0 / 1552.0	100.0%	0.000543	0.005379
	`832`	30	1850.8 / 1850.6	100.0%	0.000488	0.005781
`opt`	`640`	30	1150.2 / 1149.9	100.0%	0.003513	0.023308
	`768`	30	1638.7 / 1638.5	100.0%	0.003573	0.030832
	`832`	30	1956.6 / 1956.6	100.0%	0.003477	0.034790

Note

Keypoint coordinate error is \(0.0000\text{ px}\) MAE and Max Diff across all tested configurations. Minor confidence variance (\(\sim 10^{-4}\) in full, \(\sim 10^{-3}\) in opt) stems from floating-point reduction order in the attention layers. Evaluated up to max_dim=832.

To verify parity on a custom dataset organized into two folders with matching filenames (folder_a/, folder_b/):

uv run tools/benchmark/parity_efficient_loftr.py \
    --folder-a data/custom_pairs/folder_a \
    --folder-b data/custom_pairs/folder_b \
    --variant all \
    --max-dims 640 768 832 \
    --max-pairs 30

Performance Benchmarks¶

Latency, throughput, and host process RAM deltas measured on the first pair of the MegaDepth-1500 outdoor dataset (with native input resolutions of \(1271 \times 953\) for image A and \(1600 \times 1174\) for image B). Measurements reflect 5 unmeasured warmup iterations and 20 timed measurement iterations.

CUDA Execution (CUDAExecutionProvider)CPU Execution (CPUExecutionProvider)

Variant	Max Dim Limit	Latency Mean (ms)	Median (ms)	P95 (ms)	Throughput (Pairs/s)	Host RAM Delta	Peak VRAM Delta
`full`	`640`	85.42 +/- 5.27	85.48	96.85	11.7	+564.5 MB	+1808.0 MB
	`832`	205.73 +/- 40.02	197.83	265.53	4.9	+200.1 MB	+4338.0 MB
	`960`	358.43 +/- 86.24	314.15	541.61	2.8	+360.7 MB	+6953.0 MB
	`1024`	3560.52 +/- 334.86	3437.02	4224.72	0.3	+1859.1 MB	+6953.0 MB
`opt`	`640`	121.49 +/- 14.46	122.42	139.37	8.2	+101.2 MB	+1626.0 MB
	`832`	456.78 +/- 80.48	503.46	562.17	2.2	+207.7 MB	+4220.0 MB
	`960`	878.12 +/- 80.74	879.32	1009.08	1.1	+317.9 MB	+5904.0 MB
	`1024`	1453.20 +/- 133.25	1463.95	1660.77	0.7	+490.8 MB	+6953.0 MB

Variant	Max Dim Limit	Latency Mean (ms)	Median (ms)	P95 (ms)	Throughput (Pairs/s)	Host RAM Delta
`full`	`640`	3085.78 +/- 333.01	3018.24	3690.02	0.3	+1175.2 MB
	`832`	5789.84 +/- 268.09	5704.12	6189.51	0.2	+2731.2 MB
	`960`	9123.22 +/- 450.85	9049.17	9671.42	0.1	+4065.2 MB
	`1024`	11077.42 +/- 399.25	11022.39	11817.81	0.1	+3581.9 MB
`opt`	`640`	2833.60 +/- 120.07	2821.49	3020.90	0.4	+1332.2 MB
	`832`	6070.13 +/- 1001.03	5627.47	8153.02	0.2	+1490.8 MB
	`960`	8201.68 +/- 455.54	8263.09	8992.29	0.1	+3115.0 MB
	`1024`	10058.98 +/- 647.43	10141.71	10939.23	0.1	+3940.5 MB

SpatialHub Adapter API & Usage¶

from spatialhub import EfficientLoFTR

# Initialize with 'opt' variant and CUDA acceleration
matcher = EfficientLoFTR(
    model_type="opt",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"]
)

# Execute matching between two images
result = matcher.match("image_a.jpg", "image_b.jpg", max_dim=1024)

# Render side-by-side visualization
result.visualize(top_k=50, save_path="matches.png")

Tooling & Verification Commands¶

Export ONNXParity CheckPerformance Profile

Export PyTorch checkpoint weights to standalone ONNX graphs:

uv run tools/export/export_efficient_loftr.py \
    --checkpoint weights/eloftr_outdoor.ckpt \
    --output-folder onnx_weight \
    --opset 17

Parameter	Type	Default	Description
`--variant`	`str`	`"all"`	Model variant to export (`full`, `opt`, or `all`).
`--checkpoint`	`str`	`None`	Path to source `.ckpt` PyTorch weights file (downloaded if omitted).
`--output-folder`	`str`	`onnx_weight`	Destination directory for exported `.onnx` model files.
`--width`	`int`	`640`	Input image width in pixels (must be a multiple of 32).
`--height`	`int`	`480`	Input image height in pixels (must be a multiple of 32).
`--opset`	`int`	`17`	ONNX Operator Set version.

Validate numerical coordinate and confidence parity against PyTorch:

uv run tools/benchmark/parity_efficient_loftr.py \
    --folder-a upstream/efficient_loftr/data/0015_pairs/folder_a \
    --folder-b upstream/efficient_loftr/data/0015_pairs/folder_b \
    --variant all \
    --max-dims 640 768 832 \
    --max-pairs 30 \
    --coord-tol 0.01 \
    --conf-tol 0.05 \
    --output-file .profile/parity_eloftr.txt

Parameter	Type	Default	Description
`--folder-a`	`str`	Required	Path to first folder of paired evaluation images.
`--folder-b`	`str`	Required	Path to second folder of paired evaluation images.
`--variant`	`str`	`"all"`	Model variant to test (`full`, `opt`, or `all`).
`--max-dims`	`list[str]`	`["640", "1024"]`	Input spatial scale limits to evaluate.
`--max-pairs`	`int`	`None`	Maximum number of image pairs to evaluate.
`--coord-tol`	`float`	`0.01`	Keypoint coordinate tolerance threshold in pixels.
`--conf-tol`	`float`	`0.01`	Confidence score tolerance threshold.
`--checkpoint`	`str`	`None`	Optional path to custom PyTorch checkpoint file.
`--model-dir`	`str`	`None`	Directory containing local `.onnx` model files.
`--output-file`	`str`	`None`	File path to write markdown parity report.

Benchmark execution latency, throughput, and memory deltas:

uv run tools/benchmark/profile_efficient_loftr.py \
    --variant all \
    --provider all \
    --image-a "upstream/efficient_loftr/data/0015_pairs/folder_a/0000.jpg" \
    --image-b "upstream/efficient_loftr/data/0015_pairs/folder_b/0000.jpg" \
    --max-dims 640 832 960 1024 \
    --warmup 5 \
    --iters 20 \
    --output-file .profile/profile_eloftr.txt

Parameter	Type	Default	Description
`--variant`	`str`	`"all"`	Model variant to profile (`full`, `opt`, or `all`).
`--provider`	`str`	`"all"`	Execution provider filter (`cuda`, `cpu`, or `all`).
`--image-a`	`str`	`None`	Path to first evaluation image.
`--image-b`	`str`	`None`	Path to second evaluation image.
`--resolutions`	`list[str]`	`["640x480"]`	Synthetic spatial resolutions if no images are given.
`--max-dims`	`list[str]`	`["640", "1024"]`	Input spatial scale limits to test.
`--warmup`	`int`	`5`	Number of unmeasured warmup iterations.
`--iters`	`int`	`20`	Number of timed measurement iterations.
`--model-dir`	`str`	`None`	Directory containing local `.onnx` model files.
`--output-file`	`str`	`None`	File path to write markdown summary table.

Returned Result Data Structure¶

Returns a MatchResult dataclass:

Attribute	Type	Shape	Description
`image_a`	`str \| Path \| np.ndarray`	Input	First image reference or NumPy array.
`image_b`	`str \| Path \| np.ndarray`	Input	Second image reference or NumPy array.
`keypoints_a`	`np.ndarray`	`(N, 2)` float32	Verified keypoint `[x, y]` coordinates in `image_a`.
`keypoints_b`	`np.ndarray`	`(N, 2)` float32	Verified keypoint `[x, y]` coordinates in `image_b`.
`confidence`	`np.ndarray`	`(N,)` float32	Match confidence scores in `[0.0, 1.0]`.