Digital Image Fundamentals

Chapter Overview

Every project in this book begins the same way: an image arrives, and code goes to work on it. Chapter 0 taught you to hold that image competently, as a NumPy array with a dtype, a channel order, and a set of conventions. This chapter asks the question that makes everything downstream make sense: what is that array, really? The answer is a story with a beginning (photons striking silicon), a middle (a chain of discretizations and encodings, each one a deliberate engineering compromise), and an end (a compressed file that preserves what a human viewer would miss least). Knowing this story is the difference between treating image data as a mysterious given and treating it as the output of a machine you understand, can reason about, and can debug.

The chapter follows the data's own path. We start inside the camera: lenses focus light, sensors count photons through a mosaic of color filters, and an image signal processor performs a dozen irreversible transformations before your code ever runs. We then formalize the two discretizations that turn a continuous optical image into numbers: sampling, whose failure mode is aliasing (detail that lies), and quantization, whose failure mode is banding (gradients that shatter). With those tools in hand we can read a camera datasheet critically, distinguishing the three budgets, resolution, bit depth, and dynamic range, and seeing which one actually limits a given application, including the high-dynamic-range capture tricks that widen the narrowest budget of all.

The last two sections explain the remaining mysteries of the array. The channel dimension gets its due in a tour of color science: why three numbers per pixel, what RGB actually encodes (and the gamma trap waiting inside it), and why the same color wears different coordinates in HSV, Lab, and YCbCr depending on whether the job is selection, measurement, or compression. Finally, the chapter reassembles all of its own ideas into the file formats you use daily: PNG's lossless contract, JPEG's perceptual gamble (chroma subsampling from the color section, quantization from the sampling section, frequency transforms prefiguring Chapter 4), and the modern WebP, AVIF, and learned codecs now replacing them. Along the way we meet PSNR and SSIM, the first members of an evaluation-metric family this book follows all the way to FID and beyond in Chapter 37.

A word on why this matters for AI specifically. Modern vision models are trained on millions of images that all passed through the machinery in this chapter, with its auto white balance, its tone curves, its 8-bit quantization, and its JPEG artifacts. The pipeline's choices become the model's silent assumptions, and the pipeline's failure modes (clipped highlights, aliased textures, compression-shifted embeddings) become the model's failure modes. The engineers who debug those failures fastest are invariably the ones who can look at a wrong prediction and ask not just "what did the model do?" but "what did the camera do?". This chapter makes you one of them.