Application Reference Maps

"I asked for one map and got ten. Apparently every application believes it is the center of computer vision, and after reading their evaluation protocols, I regret to report that each one has a point."

A Cartographer of Very Opinionated Pipelines

To use this appendix, start with Figure G.1 as the application lens, then find your application in Table G.1, follow the chapter path, and read the matching domain playbook. Section 2 covers medical imaging and microscopy; Section 3 covers autonomous driving and robotics; Section 4 covers industrial inspection; Section 5 covers Earth observation and biometrics; Section 6 covers spatial computing, creative AI, document intelligence, retail, and visual search. The tables show the first data formats to learn, the usual starting methods, the benchmarks that define the literature, and the metrics that matter after the demo works. The playbooks then add the details that turn a chapter path into domain practice.

1. Pick Your Application Path Reference

Table G.1 gives the short version: the application, representative systems, book chapters to treat as required reading, the domain additions that are not fully covered elsewhere, and the benchmarks or protocols that make a claim credible. The rows are ranked by how naturally the current book already supports the application, not by market size. Rows near the top require fewer chapters outside this book and fewer domain-specific standards before a credible project can begin. After this map, the appendix changes from ranking order to teaching order. Sections 2 through 6 group domains by the contract a builder must learn first: regulated instrument data, closed-loop action, factory inspection, governed identity or geography, and product-facing multimodal systems.

Use the rows as a routing table. Medical imaging and microscopy continue in Section 2; driving and robotics continue in Section 3; industrial inspection continues in Section 4; Earth observation, biometrics, and security continue in Section 5; spatial computing, creative AI, documents, retail, and visual search continue in Section 6. After the matching domain playbook, read Sections 7 and 8 to choose metrics and tools, then Section 9 to translate the domain into a capstone variant.

2. Medical Imaging and Computational Microscopy Advanced

Medical imaging and microscopy are grouped here because both punish natural-image assumptions. Their pixels come from physical instruments, their labels are expensive, and their final claims are about bodies, cells, tissues, or materials rather than pixels alone. The core methods live in Chapter 7 for restoration, Chapter 24 for segmentation, and Chapter 25 for foundation models. The domain additions below tell you what changes before those methods become credible research tools.

Medical imaging: validation before architecture

A leading medical-imaging paper usually starts with a familiar architecture and wins or loses on validation design. For patient-derived medical data, a split by image, slice, or patch is usually invalid when samples from the same patient can cross sets. A credible split is patient-level; stronger external validation often holds out acquisition sites, scanner families, or both, depending on the deployment claim. This is the medical version of the dataset-contamination warning in Appendix B. The model should be evaluated not only with Dice or AUROC but also with calibration, uncertainty, clinically meaningful thresholds, and reader-study design when the output enters diagnosis or treatment planning.

Leakage is not only duplicate files. Adjacent CT slices share anatomy, pathology, scanner noise, and reconstruction settings, so a model can recognize the patient context without learning the disease boundary. Patch-level splits in pathology have the same problem: tiles from one slide inherit stain, scanner, and tissue-preparation signatures. Patient-level and site-level splits force the model to solve the clinical task under new acquisition conditions.

Microscopy: measurement before pixel score

Picture a fluorescence time-lapse experiment as a measurement instrument, not just an image sequence. If a denoiser makes cells look smoother but shifts their intensity trend, the biological conclusion can change even when PSNR improves. A useful first check is concrete: count the same 200 cells before and after restoration, compare lineage breaks, and verify that the treatment-control intensity ratio moves less than the biological effect being claimed.

3. Autonomous Driving, Robotics, and Embodied AI Advanced

Driving and robotics share the central pressure that a perception model must close a loop. A detector that is slow, unstable, or badly calibrated is not merely inaccurate; it changes the controller's behavior. The geometry stack of Chapter 12 through Chapter 14, the detection and segmentation stack of Chapter 23 and Chapter 24, and the world-model material of Chapter 36 are the minimum path. The domain addition is the control interface.

Driving: perception in planner coordinates

For autonomous driving, the representation that changed the stack is the bird's-eye view grid. Image features from several cameras are lifted into 3D or depth-conditioned space, splatted into a ground-plane or voxel representation, fused over time, and decoded into lanes, boxes, occupancy, or future motion. BEV is not just an implementation detail; it aligns perception with planning. A pedestrian box in image coordinates asks the planner to reason through projection; an occupied cell in ego coordinates can be consumed directly by a cost map. That is why occupancy forecasting and 4D scene prediction now sit next to detection in frontier driving papers.

BEV also changes which calibration errors matter. A one-degree camera pitch error may be visually subtle in the image but can move projected objects meters away on the ground plane. Temporal fusion then compounds the mistake because stale or misregistered cells look like motion. That is why BEV systems need calibration checks, timestamp alignment, and ego-motion compensation as part of evaluation, not as preprocessing trivia.

Robotics: perception in action coordinates

For robotics, the newest question is how much of the old perception stack to keep. The aha is that a robot does not need the prettiest mask; it needs the mask that makes the next action stable. A segmentation model with slightly lower mIoU can be the better robot model if its boundary gives the gripper a repeatable contact patch, while a higher-mIoU model can fail if its mask flickers frame to frame. A VLA policy may learn a more direct mapping from observations and instructions to actions, but it inherits new burdens: action-space definition, teleoperation data quality, proprioception, latency, recovery behavior, and simulation-to-real transfer. Treat VLA policies as a new layer above the perception chapters, not a replacement for understanding them.

4. Industrial Inspection and Quality Control Intermediate

Industrial inspection is the application for which the book is closest to complete. The capstone already asks the reader to build a defect-inspection system, and the necessary tools span morphology, restoration, detection, segmentation, and synthetic data. What the core chapters do not fully spell out is the anomaly-detection research line and the factory metric contract.

The factory metric shift is severe. Academic anomaly detection often reports image AUROC and pixel AUROC. A quality engineer asks a different question: at the false reject rate the line can tolerate, what fraction of dangerous defects escape? That means evaluation should report operating points, not just ranking quality. A model that improves AUROC by smoothing easy cases but misses one rare safety-critical crack is worse than a noisy model that catches every dangerous crack and sends more parts to human review.

The operating point should be chosen from the production constraint, not from the prettiest ROC curve. If the line can send 2 percent of parts to manual review, fix the false reject rate at 2 percent and measure how many true defects still escape. Then stratify that escape rate by SKU, defect type, lighting station, and lot, because a single average can hide the exact rare crack the inspection system exists to catch.

5. Earth Observation, Biometrics, and Security Advanced

Earth observation and biometrics look unrelated, but both are governed by domain contracts that generic vision papers often understate. Earth observation ties pixels to a place, a time, a sensor band, and a coordinate reference system. Biometrics ties pixels to identity, consent, demographic fairness, and operating thresholds. A researcher who ignores either contract can still train a model, but cannot make a trustworthy claim.

Earth observation: pixels tied to place and time

An Earth-observation pipeline should begin with the raster metadata. Coordinate reference system, affine transform, spatial resolution, band ordering, acquisition time, solar angle, and cloud mask are not side information; they are part of the sample. A flood map that is shifted by a few pixels is not a slightly worse segmentation. It is an incorrect map. A land-cover classifier that performs well in one climate and fails in another is not robust just because its random split was balanced. Geographic and temporal holdouts are the Earth-observation version of patient-level splits.

In Earth observation, an example is not just an image tensor and a label mask. It is an image tensor plus a map projection, ground sample distance, sensor bandpass, acquisition date, sun geometry, and sometimes atmospheric correction state. Two tensors with the same pixel values can mean different physical reflectance if they came from different sensors or seasons. That is why normalization, reprojection, and temporal matching are part of the scientific claim.

Biometrics: pixels tied to identity and governance

Biometrics has the opposite problem: the data can look deceptively ordinary, but the deployment context is sensitive. Face verification compares one claimed identity against one template; identification searches a gallery; person re-identification links appearances across cameras without necessarily naming the person. These are different tasks with different risk profiles. Accuracy is not a sufficient metric. Systems are chosen at an operating threshold: false accept rate, false reject rate, false positive identification rate, and false negative identification rate define the cost tradeoff, and demographic differentials must be reported when identity or access is at stake.

The metric names encode different search problems. FAR and FRR belong to verification, where the system accepts or rejects one claimed identity. FPIR and FNIR belong to identification, where the system searches a gallery and may accuse the wrong person or miss the right one. A threshold that is acceptable for unlocking a phone can be unacceptable for public-space identification because the number of comparisons and the harm of a false candidate are different.

6. Spatial Computing, Creative AI, Documents, Retail, and Search Intermediate

The remaining application families are unified by one production pattern: they combine perception, generation, retrieval, and interaction. Spatial capture turns camera geometry into a navigable asset. Creative tools turn generative models into controllable workflows. Document and retail systems turn images into structured data or embeddings that a user can search. In each case, a single model is rarely the product.

Spatial computing

Spatial computing is where the old and new geometry chapters meet most cleanly. The reveal is that the same phone video can become three different products depending on the representation: a point cloud for measurement, a Gaussian splat for fast photorealistic viewing, or a mesh for editing and collision. The classical route uses calibrated cameras, feature matching, SfM, multi-view stereo, and a mesh or point cloud. The neural route may use NeRF, 3D Gaussian splatting, or a feed-forward geometry transformer. The production route usually mixes them: learned geometry bootstraps or repairs the capture, classical optimization enforces consistency when precision matters, and a real-time rendering representation carries the result to a phone or headset. That is why Chapter 14 and Chapter 27 should be read together.

Creative generation

Creative AI has a different metric problem. FID and CLIPScore help compare models, but product teams need accepted-asset cost. That metric includes prompt retries, failed generations, human edits, safety-filter rejections, latency, and whether brand identity stayed intact. A model with slightly worse public benchmark numbers can be better if it follows instructions reliably, preserves a product's shape, and costs less per approved output. This is the practical form of the evaluation lessons in Chapter 37.

Accepted-asset cost should be measured as a workflow metric: total generation cost plus review time plus edit time divided by the number of assets that pass the final use gate. A model that generates beautiful images but needs ten retries per usable product shot can lose to a less glamorous model that obeys layout, brand, and object-identity constraints on the first or second attempt. This metric turns subjective quality into a production constraint the team can optimize.

Documents, retail, and visual search

7. Metric Shifts by Domain Intermediate

Before you trust a result table, ask what the application would punish. The table below is a compact checklist for the moment a general computer vision metric stops being enough. Use it before writing an abstract, building a results table, or choosing a capstone variant.

The pattern is stricter than a metric swap: generic and domain metrics should be co-computed on the same panel, model, split, and seed, then saved as one artifact so the comparison is defensible. Code Fragment 1 grounds the distinction with an inspection threshold: the generic score changes smoothly, but the domain decision depends on escape rate at a fixed false reject budget.

# Computes the inspection operating point from defect scores.
# The threshold is chosen to respect the false reject budget first.
import numpy as np

good_part_scores = np.array([0.02, 0.04, 0.07, 0.08, 0.10])
defect_scores = np.array([0.35, 0.62, 0.74, 0.91, 0.96])

threshold = np.quantile(good_part_scores, 0.80)
false_reject_rate = np.mean(good_part_scores > threshold)
escape_rate = np.mean(defect_scores <= threshold)

print(f"threshold={threshold:.2f}")
print(f"false_reject_rate={false_reject_rate:.2f}")
print(f"escape_rate={escape_rate:.2f}")

When both generic and domain metrics appear in a paper, first check that the compared numbers come from the same evaluation run; then use the domain metric to interpret whether the result matters in the application. The aha from Code Fragment 1 is that the threshold is not a model-score detail; it is the business rule translated into code. Once the false reject budget fixes the threshold, escape rate becomes the number that decides whether the system can ship.

8. Tools of the Trade by Application Beginner

The book's tools chapters teach the general stack. Table G.8 adds the domain packages that a practical team should know before reinventing utilities that the community already maintains. Skim this table only after choosing a domain; it is a shortcut list, not a reading assignment.

9. Capstone Variants Intermediate

The default capstone is an industrial defect-inspection system, but the same four-part arc can be specialized to other domains by changing the data contract, metric ladder, and failure modes. Table G.9 turns the map into a build plan by asking how each domain reshapes the path from pixels and geometry to learned models, generative support, and deployment-aware evaluation.