COLMAP & Modern Reconstruction Pipelines

"You spent four sections teaching me to find tracks, register cameras, triangulate points, and balance the whole thing with bundle adjustment. Adorable. There is a button for that. It is called colmap automatic_reconstructor, and it has read your homework."

A Reconstruction Pipeline That Has Seen Every Tutorial Ever Written

The last four sections were a from-scratch education: Section 14.1 chained matches into tracks, Section 14.2 grew a model camera by camera, Section 14.3 polished it with bundle adjustment, and Section 14.4 turned the same machinery into a real-time map. That education was not wasted effort to be replaced by a library call; it is the only thing that makes the library call diagnosable. When COLMAP returns a reconstruction that has split into two disconnected pieces, or registers eleven of your forty images and gives up, the fix is never in the documentation. It is in knowing that two-view seeding needs a wide baseline, that PnP registration needs enough existing points in view, and that a drifting model is a bundle-adjustment-and-track-length problem. This section is the bridge from the textbook pipeline to the one practitioners actually run, and it keeps one foot on each side of that bridge.

1. COLMAP: The Pipeline in Five Commands Beginner

COLMAP (Schönberger and Frahm, "Structure-from-Motion Revisited," CVPR 2016) is the default open-source structure-from-motion tool, and its design follows the chapter exactly. It runs in three logical stages: feature extraction (detect and describe keypoints in every image, the SIFT of Chapter 10), matching (find correspondences between image pairs and verify them with the two-view geometry of Chapter 13), and mapping (the incremental loop of Section 14.2 with the bundle adjustment of Section 14.3 folded in). Figure 14.5.1 lays the stages out alongside the section that taught each one.

The fastest path is the one-shot driver, which chooses sensible defaults and runs all three stages. Given a directory project/images/, the entire reconstruction is one command:

# One-shot reconstruction. COLMAP picks GPU SIFT, exhaustive matching for small
# sets, and writes everything under project/.
colmap automatic_reconstructor \
    --workspace_path project \
    --image_path project/images

# project/ now contains:
#   database.db                  SQLite: keypoints, descriptors, matches, two-view geometries
#   sparse/0/{cameras,images,points3D}.bin   the reconstructed model(s)
#   dense/                       (if dense stereo ran) depth maps + fused point cloud

For anything beyond a quick look you run the stages explicitly, because each exposes parameters that decide success. The matcher choice is the most consequential: exhaustive_matcher compares every pair (quadratic, fine for tens of images), sequential_matcher matches each frame to its temporal neighbors plus loop-closure candidates (ideal for video, echoing Section 14.4), and vocab_tree_matcher uses a visual vocabulary to propose likely pairs for large unordered collections, the "Building Rome in a Day" idea from Section 14.1.

# Explicit five-command pipeline, the form you actually tune.
colmap feature_extractor \
    --database_path project/database.db \
    --image_path project/images \
    --SiftExtraction.estimate_affine_shape 1   # better repeatability under viewpoint change

colmap sequential_matcher \                    # use exhaustive_ or vocab_tree_ for unordered sets
    --database_path project/database.db \
    --SequentialMatching.loop_detection 1      # enable loop closure for video

colmap mapper \
    --database_path project/database.db \
    --image_path project/images \
    --output_path project/sparse \
    --Mapper.ba_global_function_tolerance 1e-6 # bundle-adjustment convergence (Section 14.3)

colmap model_converter \                        # export to a human-readable text model
    --input_path project/sparse/0 \
    --output_path project/sparse_text \
    --output_type TXT

2. Reading the Database and the Model Intermediate

COLMAP's two outputs are a SQLite database and a reconstruction model, and reading them directly turns the tool from a black box into an instrument. The database (database.db) holds everything from the front end: a cameras table of intrinsic models, an images table, a keypoints table (one blob of detected points per image), a descriptors table, a matches table (raw putative matches per pair), and a two_view_geometries table (the inlier matches that survived geometric verification, plus the fundamental, essential, or homography matrix). That last table is the chapter's correspondence-and-RANSAC pipeline frozen on disk.

The model is three binary files in sparse/0/: cameras.bin (intrinsics), images.bin (one entry per registered image: its quaternion-and-translation pose plus the 2D-to-3D observations), and points3D.bin (each triangulated point's XYZ, color, reprojection error, and the track of images that see it, the track from Section 14.1 made concrete). COLMAP ships pycolmap, a Python binding that reads both without you parsing the binary format. The following listing opens a finished model and computes the diagnostics every reconstruction should be checked against.

import pycolmap
import numpy as np

# Load a finished reconstruction (the sparse/0 folder COLMAP wrote).
rec = pycolmap.Reconstruction("project/sparse/0")
print(rec.summary())            # quick stats: #images registered, #points, mean track length

# How many images actually got registered, and which were dropped?
n_reg = rec.num_reg_images()
print(f"registered {n_reg} images, {len(rec.points3D)} points")

# Track length: the count of images observing each 3D point. Short tracks
# (seen by only 2 cameras) are poorly constrained; long tracks are gold.
track_lengths = np.array([p.track.length() for p in rec.points3D.values()])
print(f"mean track length {track_lengths.mean():.2f}, "
      f"fraction length-2 {np.mean(track_lengths == 2):.2%}")

# Reprojection error: the bundle-adjustment residual (Section 14.3) per point.
errors = np.array([p.error for p in rec.points3D.values()])
print(f"mean reprojection error {errors.mean():.3f} px")  # healthy models: well under 1 px

# Per-image camera centers, for plotting the trajectory or a top-down map.
centers = np.array([img.projection_center() for img in rec.images.values()])
print("scene extent (m, in the model's arbitrary scale):",
      centers.max(0) - centers.min(0))

A typical healthy run on a 40-image object scan prints something like registered 40 images, 18204 points, a mean track length near 4, a length-2 fraction below 25 percent, and a mean reprojection error around 0.6 px. Those numbers are the quantitative version of "the bundle adjustment converged and the tracks are long." When the registration count is far below the image count, the cure is upstream, in matching, which is exactly where the next subsection improves the tool.

3. Upgrading the Front End: Learned Features with hloc Advanced

COLMAP's classical SIFT front end is excellent on well-textured, well-lit, densely-overlapping image sets and fragile everywhere else: low texture, repetitive structure, day-to-night or seasonal change, and especially wide baselines, the cases where the hand-crafted descriptors of Chapter 10 were always going to struggle. The modern fix is to keep COLMAP's mapper (the geometry is sound) and replace only the front end with learned detection and matching. The Hierarchical Localization toolbox, hloc, is the standard wrapper for doing exactly that. It detects keypoints with SuperPoint, matches them with SuperGlue or the lighter LightGlue, optionally proposes image pairs with a global retrieval network (NetVLAD), and feeds the verified matches into COLMAP's database for triangulation and mapping.

The architectural point is the same as Section 14.4's "geometry is solved, learning replaces perception": the bundle adjustment, triangulation, and pose estimation are unchanged classical machinery, while the perception layer (what is a keypoint, do these two patches match) becomes a network trained on the failures that broke SIFT. The result is dramatically higher registration rates on hard scenes for a few extra lines of orchestration.

from pathlib import Path
from hloc import extract_features, match_features, reconstruction, pairs_from_retrieval

images = Path("project/images")
outputs = Path("project/hloc")

# Pre-defined configs select the network and its weights.
retrieval_conf = extract_features.confs["netvlad"]      # global descriptor for pair proposal
feature_conf   = extract_features.confs["superpoint_aachen"]  # learned local keypoints
matcher_conf   = match_features.confs["superglue"]      # learned matcher (or "lightglue")

# 1. Global descriptors, then propose the 20 most similar pairs per image
#    (the learned analogue of vocab-tree matching from Section 14.1).
retrieval_path = extract_features.main(retrieval_conf, images, outputs)
sfm_pairs = outputs / "pairs-netvlad.txt"
pairs_from_retrieval.main(retrieval_path, sfm_pairs, num_matched=20)

# 2. Learned local features and matches.
feature_path = extract_features.main(feature_conf, images, outputs)
match_path = match_features.main(matcher_conf, sfm_pairs, feature_conf["output"], outputs)

# 3. Hand the verified matches to COLMAP's incremental mapper unchanged.
model = reconstruction.main(outputs / "sfm", images, sfm_pairs, feature_path, match_path)
print(model.summary())   # same pycolmap Reconstruction object as Section 2

4. Global SfM and the Neural Hand-off Advanced

Incremental SfM (Section 14.2) is accurate but slow: it solves a fresh bundle adjustment after every camera, so cost grows steeply with image count. Global SfM takes a different route. It first estimates all relative rotations and translations from the pairwise two-view geometries, then solves two global problems, a rotation averaging that makes every pairwise rotation mutually consistent and a translation averaging (and triangulation) that fixes the global geometry, finishing with a single bundle adjustment over everything at once. Historically global methods were faster but more fragile, especially in translation averaging. GLOMAP (Pan et al., ECCV 2024) closed that gap: it reuses COLMAP's database and front end, runs a robust global solver, and produces reconstructions of comparable accuracy to incremental COLMAP while being an order of magnitude or more faster on large scenes.

# GLOMAP consumes a COLMAP database (same feature_extractor + matcher as Section 1),
# then solves the whole reconstruction globally instead of camera-by-camera.
colmap feature_extractor --database_path project/database.db --image_path project/images
colmap exhaustive_matcher --database_path project/database.db

glomap mapper \
    --database_path project/database.db \
    --image_path project/images \
    --output_path project/sparse_glomap   # writes the same cameras/images/points3D format

Whichever solver produces it, the sparse model is rarely the final deliverable. A sparse cloud of triangulated keypoints is enough to localize cameras but too thin to look like a scene. Two routes densify it. The classical route is multi-view stereo (MVS): COLMAP's patch_match_stereo estimates a depth map per image by the dense matching ideas of Chapter 13 generalized from two views to many, then stereo_fusion merges them into a dense point cloud, which meshing turns into a textured surface. The modern route hands the poses to a neural scene representation, and this is the chapter's most important forward link.

The hand-off is mechanical. NeRF (Mildenhall et al., 2020) trains a small network to map a 3D position and viewing direction to color and density, optimizing it so that volume-rendering the network along camera rays reproduces the input images; 3D Gaussian Splatting (3DGS) (Kerbl et al., SIGGRAPH 2023) replaces the network with millions of explicit oriented Gaussians that rasterize in real time, and crucially initializes them from the COLMAP sparse point cloud, so this chapter's points3D.bin is literally the splatting model's starting state. Both are the subject of Chapter 27; here it is enough to see that the COLMAP folder feeds them directly.

# The neural hand-off in its most common form: convert a COLMAP model into the
# transforms.json that NeRF / Gaussian-splatting trainers (e.g. nerfstudio) expect.
import pycolmap, numpy as np, json

rec = pycolmap.Reconstruction("project/sparse/0")
frames = []
for img in rec.images.values():
    # COLMAP stores world-to-camera; renderers want camera-to-world (the inverse).
    world_to_cam = img.cam_from_world.matrix()          # 3x4 [R | t]
    T = np.eye(4); T[:3, :4] = world_to_cam
    cam_to_world = np.linalg.inv(T)
    frames.append({"file_path": f"images/{img.name}",
                   "transform_matrix": cam_to_world.tolist()})

cam = next(iter(rec.cameras.values()))                   # shared intrinsics (Chapter 12)
meta = {"fl_x": cam.focal_length_x, "fl_y": cam.focal_length_y,
        "cx": cam.principal_point_x, "cy": cam.principal_point_y,
        "w": cam.width, "h": cam.height, "frames": frames}
json.dump(meta, open("project/transforms.json", "w"), indent=2)
print(f"wrote {len(frames)} posed frames for neural training")

5. Choosing a Pipeline Intermediate

With several tools on the table, the practical question is which to reach for. The decision turns on three properties of your images: are they ordered (video or a planned capture) or an unordered pile; how many are there; and how hard is the appearance (texture, lighting, baseline). The table below summarizes the chapter's working advice.

Reading the table top to bottom traces the chapter's own arc: the classical tool first, then the upgrades that target a specific weakness (scale, perception, rendering, speed), each leaving the core geometry intact. That is the honest state of the field in 2026. The neural reconstructors are extraordinary and getting better monthly, but the structure-from-motion pipeline of this chapter remains the accuracy reference they are measured against and the silent first stage inside most of them.

The chapter's five verbs are easiest to feel when you drive them yourself. The Hands-On Lab at the end of this section runs all of them on a real image set with pycolmap, then reads back the same vital signs that Exercises 14.5.1 through 14.5.3 ask you to interpret on paper.

pip install pycolmap numpy

from pathlib import Path
import pycolmap

image_dir = Path("images")
work = Path("work"); work.mkdir(exist_ok=True)
db_path = work / "database.db"

# TODO: extract SIFT features from every image into db_path.
# Hint: pycolmap.extract_features(database_path=db_path, image_path=image_dir)
...
print("Features extracted into", db_path)

# TODO: run exhaustive matching over the database you just built.
# Hint: pycolmap.match_exhaustive(database_path=db_path)
...
print("Matching done; the database now holds verified two-view geometries")

sparse = work / "sparse"; sparse.mkdir(exist_ok=True)

# TODO: run incremental mapping and capture the returned dict of reconstructions.
# Hint: maps = pycolmap.incremental_mapping(db_path, image_dir, sparse)
maps = ...
print(f"Produced {len(maps)} reconstruction(s)")

rec = max(maps.values(), key=lambda r: r.num_reg_images())

track_lengths = [p.track.length() for p in rec.points3D.values()]
mean_track = sum(track_lengths) / len(track_lengths)
len2_frac = sum(t == 2 for t in track_lengths) / len(track_lengths)

# TODO: compute the mean reprojection error over all 3D points.
# Hint: each point exposes rec.points3D[id].error (in pixels)
mean_reproj = ...

print(f"Registered images : {rec.num_reg_images()}")
print(f"3D points         : {rec.num_points3D()}")
print(f"Mean track length : {mean_track:.2f}")
print(f"Length-2 fraction : {len2_frac:.0%}")
print(f"Mean reproj error : {mean_reproj:.2f} px")

img = next(iter(rec.images.values()))
cam = rec.cameras[img.camera_id]

print("Camera model :", cam.model.name)
print("Params (f, cx, cy, ...):", cam.params)
# TODO: print the 3x4 world-to-camera matrix for this image.
# Hint: img.cam_from_world.matrix() gives the [R | t] extrinsics
print("Pose [R | t]:\n", ...)

model_out = work / "model"; model_out.mkdir(exist_ok=True)
rec.write(model_out)               # cameras.bin, images.bin, points3D.bin

# TODO: export the sparse point cloud as a PLY file for a 3D viewer.
# Hint: rec.export_PLY(str(work / "points.ply"))
...
print("Wrote model to", model_out, "and points.ply")

# TODO: run the all-in-one automatic pipeline (extract, match, map) in one call.
# Hint: pycolmap.AutomaticReconstructor or the automatic_reconstruction helper
...
# Compare the registered-image count and mean reprojection error against your
# hand-driven run from Steps 1 through 4; they should match closely.

from pathlib import Path
import pycolmap

image_dir = Path("images")
work = Path("work"); work.mkdir(exist_ok=True)
db_path = work / "database.db"

# Step 1: feature extraction (detect + describe, Chapter 10 front end)
pycolmap.extract_features(database_path=db_path, image_path=image_dir)

# Step 2: exhaustive matching with RANSAC verification (Section 14.1 match graph)
pycolmap.match_exhaustive(database_path=db_path)

# Step 3: incremental mapping = seed, register, triangulate, adjust (14.2 + 14.3)
sparse = work / "sparse"; sparse.mkdir(exist_ok=True)
maps = pycolmap.incremental_mapping(db_path, image_dir, sparse)
print(f"Produced {len(maps)} reconstruction(s)")

# Step 4: vital signs of the largest model
rec = max(maps.values(), key=lambda r: r.num_reg_images())
track_lengths = [p.track.length() for p in rec.points3D.values()]
mean_track = sum(track_lengths) / len(track_lengths)
len2_frac = sum(t == 2 for t in track_lengths) / len(track_lengths)
mean_reproj = sum(p.error for p in rec.points3D.values()) / rec.num_points3D()
print(f"Registered images : {rec.num_reg_images()}")
print(f"3D points         : {rec.num_points3D()}")
print(f"Mean track length : {mean_track:.2f}")
print(f"Length-2 fraction : {len2_frac:.0%}")
print(f"Mean reproj error : {mean_reproj:.2f} px")

# Step 5: one camera's intrinsics and pose (Chapter 12 quantities, solved from pixels)
img = next(iter(rec.images.values()))
cam = rec.cameras[img.camera_id]
print("Camera model :", cam.model.name)
print("Params (f, cx, cy, ...):", cam.params)
print("Pose [R | t]:\n", img.cam_from_world.matrix())

# Step 6: export model and a PLY point cloud
model_out = work / "model"; model_out.mkdir(exist_ok=True)
rec.write(model_out)
rec.export_PLY(str(work / "points.ply"))
print("Wrote model to", model_out, "and points.ply")

# Step 7: the Right Tool, one-call automatic reconstruction
auto = work / "auto"; auto.mkdir(exist_ok=True)
pycolmap.AutomaticReconstructor(
    pycolmap.AutomaticReconstructionOptions(
        workspace_path=str(auto),
        image_path=str(image_dir),
        dense=False,          # sparse model only; no GPU needed
    )
).run()
print("Automatic pipeline finished; compare its sparse/ model to your hand-driven run")