Resolution, Bit Depth & Dynamic Range

"The scene had twenty stops of dynamic range. I had eight bits. We compromised: I clipped."

A Permanently Clipped Highlight

Section 1.2 established the two discretizations: sampling and quantization. This section turns those abstractions into the three concrete specifications you will find on every camera datasheet and every dataset card, and adds the one that sampling and quantization together still miss: dynamic range, the question of what physical brightness interval your numbers span. The three budgets map cleanly onto the previous section's theory: resolution is your sampling budget, bit depth is your quantization budget, and dynamic range is where you choose to aim them.

1. Spatial Resolution: More Than Megapixels Beginner

Resolution colloquially means pixel count, but the useful engineering definition is resolving power: the finest real-world detail the whole system can distinguish. The two diverge constantly. As computed in Section 1.1, diffraction at f/8 blurs a point into a disk wider than three phone-sensor pixels; through that lens, extra megapixels sample blur, not detail. Optics, focus accuracy, motion, atmospheric shimmer, and the demosaicing interpolation all cap resolving power below the pixel count. Lens and system sharpness is measured by the modulation transfer function (MTF), which reports how much contrast survives at each spatial frequency; system MTF is the product of every stage's MTF, so the weakest stage dominates.

For scene understanding, the more actionable number is ground sample distance or its indoor cousin, pixels-per-object: how many pixels land on the thing you care about. Detection models do not need a 48-megapixel frame; they need enough pixels on target. A face recognizable to a human at roughly 30 pixels of eye-to-eye distance, a license plate readable at roughly 16 pixels of character height, a weld defect at 5 pixels of width: capacity planning for vision systems is the art of guaranteeing pixels-on-target, then choosing sensor and lens to deliver them. Meanwhile, classification networks routinely operate at 224×224 because, after the augmentation and resizing pipelines of Chapter 21, that is where their accuracy-compute tradeoff has historically settled.

2. Bit Depth: Levels, Headroom, and Windowing Intermediate

Bit depth sets the quantization ladder from Section 1.2: $2^b$ levels, roughly 6 dB of signal-to-noise per bit. Eight bits suffice for final display of a single rendered image. The case for more bits is headroom for computation. Every brightness adjustment, gamma change, or contrast stretch (the point operations of Chapter 2) re-spaces the levels; in an 8-bit pipeline, stretching a dim region multiplies its quantization gaps into visible bands. Editing in 16-bit or float and converting to 8-bit once, at the very end, avoids accumulating that damage. The same logic is why we recommended float32 as the working dtype in Chapter 0.

The clearest demonstration of bit depth as a budget is medical-style windowing. A CT scanner produces 12-bit data: 4096 levels spanning air to dense bone. Tissue differences a radiologist needs may be 20 levels apart, invisible when 4096 levels are crushed into 256. The fix is not more display bits; it is spending the 8 display bits only on the diagnostically relevant band. Code 1.3.1 builds a synthetic 12-bit image with a faint blob and shows the blob appear when windowed.

import numpy as np

rng = np.random.default_rng(seed=3)

# Synthetic 12-bit "scan": background tissue near level 2048 (sigma 60),
# plus a faint blob just +25 levels brighter, far below visual threshold
# after naive 8-bit conversion.
img12 = rng.normal(2048, 60, size=(256, 256))
yy, xx = np.mgrid[0:256, 0:256]
img12 += 25 * np.exp(-((xx - 170) ** 2 + (yy - 90) ** 2) / 400.0)
img12 = np.clip(img12, 0, 4095).astype(np.uint16)

blob_patch = (slice(80, 100), slice(160, 180))   # on the blob
back_patch = (slice(80, 100), slice(20, 40))     # plain background

# Naive 8-bit conversion: squeeze ALL 4096 levels into 256.
naive8 = (img12 // 16).astype(np.uint8)
print("naive  blob-background contrast:",
      round(float(naive8[blob_patch].mean() - naive8[back_patch].mean()), 1))

# Windowed conversion: spend all 256 levels on the 1900..2200 band.
lo, hi = 1900, 2200
win = np.clip((img12.astype(np.float32) - lo) / (hi - lo), 0, 1)
win8 = (win * 255).astype(np.uint8)
print("windowed blob-background contrast:",
      round(float(win8[blob_patch].mean() - win8[back_patch].mean()), 1))

naive  blob-background contrast: 1.6
windowed blob-background contrast: 21.2

3. Dynamic Range: The Span Between Noise and Clipping Intermediate

Dynamic range (DR) is the ratio between the brightest and darkest signals a system can represent usefully in the same image: bounded above by clipping (the full well of Section 1.1) and below by the noise floor. It is quoted as a ratio, in decibels, or most intuitively in stops (factors of two):

The engineering problem is a mismatch of spans, which Figure 1.3.1 lays out. A sunlit street with open shade spans roughly 17 to 20 stops. A good modern sensor captures 12 to 14 stops in one exposure; a phone sensor closer to 10. An 8-bit file holds 256 linear levels, about 8 stops between its smallest and largest codes (gamma encoding, covered in Chapter 2, spaces those codes perceptually but cannot widen the captured span). Something has to give: either highlights clip, shadows drown in noise, or the tones are compressed nonlinearly to fit.

4. HDR Capture: Widening the Bar Advanced

If one exposure cannot span the scene, take several. A short exposure preserves the highlights; a long one digs detail out of the shadows; intermediate ones cover the middle. High dynamic range (HDR) imaging merges the stack, using each pixel from whichever exposures rendered it best. Code 1.3.2 simulates a full bracket-and-merge round trip with no input files: it builds a synthetic radiance map spanning roughly 13 stops, "photographs" it at three exposure times through the clip-and-gamma pipeline of Section 1.1, and fuses the bracket with Mertens exposure fusion, which weights each pixel by how well exposed it is.

import cv2
import numpy as np

# Synthetic scene RADIANCE (linear light), spanning ~13 stops:
# a dim interior, a deep-shadow corner, and a blazing window.
h, w = 360, 480
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
radiance = 0.02 + 0.08 * (xx / w)              # interior, gently brightening
radiance[60:180, 300:440] = 40.0               # window: 500x the interior
radiance[200:320, 60:200] = 0.004              # shadow corner

def expose(rad, t):
    """One camera exposure: scale by time, clip the well, gamma-encode."""
    linear = np.clip(rad * t, 0.0, 1.0)
    return (255.0 * linear ** (1 / 2.2)).astype(np.uint8)

times = [1 / 64, 1.0, 16.0]                    # 10-stop bracket
stack = [cv2.cvtColor(expose(radiance, t), cv2.COLOR_GRAY2BGR)
         for t in times]

fused = cv2.createMergeMertens().process(stack)   # float32, ~[0, 1]
fused8 = np.clip(fused * 255, 0, 255).astype(np.uint8)

for t, s in zip(times, stack):
    print(f"t={t:>7}: clipped {(s == 255).mean():5.1%},"
          f"  near-black {(s <= 5).mean():5.1%}")
print(f"fused   : clipped {(fused8 == 255).mean():5.1%},"
      f"  near-black {(fused8 <= 5).mean():5.1%}")

t=0.015625: clipped  0.0%,  near-black  9.7%
t=    1.0: clipped  9.7%,  near-black  0.0%
t=   16.0: clipped 46.9%,  near-black  0.0%
fused   : clipped  0.0%,  near-black  0.0%

times_np = np.array(times, dtype=np.float32)
response = cv2.createCalibrateDebevec().process(stack, times_np)
hdr = cv2.createMergeDebevec().process(stack, times_np, response)  # float32 radiance
ldr = cv2.createTonemapReinhard(gamma=2.2).process(hdr)            # display version

5. The Budgets Meet Machine Learning Intermediate

Vision models inherit all three budgets through their training data. Resolution determines pixels-on-target and thus the detectability floor for small objects. Bit depth interacts with normalization: the standard practice of scaling images to $[0, 1]$ or standardizing with dataset statistics, which we will formalize in Chapter 21, silently assumes the input levels are perceptually spaced 8-bit values; feeding 12-bit linear data through the same constants wrecks the input distribution. Dynamic range determines what is in the data at all: a model trained on tone-mapped JPEGs has never seen the clipped-highlight failure mode of its deployment camera, and restoration methods from Chapter 7 cannot reconstruct what was never recorded. When you control the capture stack, record the widest range you can and compress late; when you do not, audit the data for clipping before you blame the model.

We now have the image's geometry (resolution), its level ladder (bit depth), and its physical span (dynamic range). The remaining unexplained axis of the array from Chapter 0 is the channel dimension: what the three numbers per pixel mean. That is color science, the subject of Section 1.4.