Interpolation: Nearest, Bilinear, Bicubic & Lanczos

"People ask what lies between two pixels. I have four answers, ranked by how long you are willing to wait for them."

A Perfectionist Bicubic Interpolator

The previous section gave us matrices that move pixel coordinates anywhere we like, and therein lies the problem: "anywhere" is almost never an integer grid position. Rotate an image by 15 degrees and the pixel that should land at output position $(83, 41)$ comes from input position $(74.38, 52.91)$, where no sample exists. The image, as established in Chapter 1, is a grid of point samples of a continuous light field; between the samples there is officially nothing. Interpolation manufactures the missing value, and the way it does so determines how sharp, how smooth, and how honest your warped image is.

1. Interpolation Is Convolution Intermediate

Every method in this section fits one formula. To estimate the image value at fractional position $x$ (we work in 1D first; 2D is two nested applications), place a kernel $k$ at $x$ and take a weighted sum of the nearby samples $f[i]$:

This is exactly the convolution machinery of Chapter 3, evaluated at a non-integer location. The kernel's support (how many samples it touches) sets the cost; the kernel's shape sets the quality. Sampling theory from Chapter 4 tells us the ideal reconstruction kernel for a band-limited signal is the infinite sinc function; everything in practice is a cheap, finite approximation to that ideal, and each approximation fails in its own characteristic way.

Figure 5.3.1 plots the four kernels we cover. Keep it in view as you read: nearly every visual artifact in resized images can be diagnosed by looking at the shape of the kernel that produced it.

2. The Four Kernels Intermediate

2.1 Nearest neighbor: just copy

The box kernel assigns the value of whichever sample is closest: $\hat{f}(x) = f[\mathrm{round}(x)]$. It is the fastest possible method, introduces no new values, and looks terrible on photographs: diagonal edges become staircases and smooth gradients become bands. Yet it is the only correct choice for one important class of images: label maps and masks. A segmentation mask containing class IDs $\{0, 1, 2\}$ must never be bilinearly resized, because averaging class 0 (background) with class 2 (vehicle) produces class 1 (pedestrian) along every boundary, a phantom-class bug that has burned countless practitioners and will matter again when we resize masks in Chapter 24.

2.2 Bilinear: the workhorse

The tent kernel performs linear interpolation between the two bracketing samples; applied along both axes it reads four neighbors and blends them by area. If the fractional position within the $2 \times 2$ cell is $(a, b)$ with $a, b \in [0, 1)$:

Each sample's weight is the area of the sub-rectangle diagonally opposite it, a picture worth drawing once in your notebook and never forgetting. Bilinear is continuous (no blocking), cheap (4 reads, 3 multiply-adds per channel), and slightly blurry, since the tent kernel attenuates high frequencies noticeably. It is the default interpolation in most libraries and in nearly all real-time and GPU work; texture units in graphics hardware implement it in silicon.

2.3 Bicubic: the photographer's default

The cubic convolution kernel of Keys (1981) reads a $4 \times 4$ neighborhood and fits a smooth piecewise-cubic curve. The 1D kernel, parameterized by $a$:

The negative lobes visible in Figure 5.3.1 give bicubic its mild edge-enhancing quality: output values can overshoot the input range slightly, which reads as sharpness. The parameter $a$ tunes the lobe depth, and here hides a notorious cross-library gotcha: OpenCV uses $a = -0.75$ while Pillow uses $a = -0.5$ (the value Keys derived as optimal). "Bicubic" therefore names two different filters depending on which library resized your image, and the pixel differences are large enough to shift the metrics of trained models, as the research-frontier callout below recounts.

2.4 Lanczos: the archivist's choice

The Lanczos kernel is the ideal sinc multiplied by a window that tapers it to zero after $a$ lobes:

OpenCV's INTER_LANCZOS4 uses $a = 4$ (an $8 \times 8$ neighborhood); Pillow's LANCZOS uses $a = 3$. This is the closest practical approximation to the ideal reconstruction filter of Chapter 4, and it preserves fine detail best on upscales. The price is ringing: those oscillating negative lobes can paint faint halos around high-contrast edges, conspicuous around text and graphics. Photographic archives and astronomy pipelines accept the halos for the detail; user-interface code usually does not.

3. Implementing Bilinear From Scratch Intermediate

The bilinear sampler below is the heart of every warp we write in this chapter; Section 5.4 will call it on millions of coordinates at once, so we write it vectorized from the start. It accepts arrays of fractional coordinates and returns sampled values, grayscale for clarity:

import numpy as np

def bilinear_sample(img, xs, ys):
    """Sample a grayscale image at fractional coords (vectorized).

    img: (H, W) float array.  xs, ys: arrays of x and y coordinates.
    Returns an array shaped like xs with interpolated values.
    """
    h, w = img.shape
    x0 = np.clip(np.floor(xs).astype(int), 0, w - 2)   # left column
    y0 = np.clip(np.floor(ys).astype(int), 0, h - 2)   # top row
    a = np.clip(xs - x0, 0.0, 1.0)                     # horizontal fraction
    b = np.clip(ys - y0, 0.0, 1.0)                     # vertical fraction

    f00 = img[y0,     x0    ]                          # top-left
    f10 = img[y0,     x0 + 1]                          # top-right
    f01 = img[y0 + 1, x0    ]                          # bottom-left
    f11 = img[y0 + 1, x0 + 1]                          # bottom-right

    return ((1 - a) * (1 - b) * f00 + a * (1 - b) * f10
            + (1 - a) * b * f01 + a * b * f11)

# Sanity check on a tiny ramp image: value at (0.5, 0.5) should be
# the average of the four corner samples 0, 1, 10, 11 = 5.5
ramp = np.array([[0., 1.], [10., 11.]])
print(bilinear_sample(ramp, np.array([0.5]), np.array([0.5])))

[5.5]

small = cv2.resize(img, (640, 480), interpolation=cv2.INTER_AREA)     # downscale
big   = cv2.resize(img, None, fx=3, fy=3, interpolation=cv2.INTER_CUBIC)  # upscale

4. Comparing the Kernels in Practice Beginner

A fair comparison needs ground truth, so we use the classic protocol: shrink an image, blow it back up with each method, and measure fidelity to the original with PSNR (introduced in Chapter 1):

import cv2, time

img = cv2.imread("detail_rich.jpg", cv2.IMREAD_GRAYSCALE)  # 1024 x 1024
small = cv2.resize(img, None, fx=0.25, fy=0.25,
                   interpolation=cv2.INTER_AREA)           # clean 4x shrink

flags = {"nearest":  cv2.INTER_NEAREST,
         "bilinear": cv2.INTER_LINEAR,
         "bicubic":  cv2.INTER_CUBIC,
         "lanczos4": cv2.INTER_LANCZOS4}

for name, flag in flags.items():
    t0 = time.perf_counter()
    up = cv2.resize(small, img.shape[::-1], interpolation=flag)
    dt = (time.perf_counter() - t0) * 1000
    print(f"{name:9s}  PSNR {cv2.PSNR(img, up):5.2f} dB   {dt:5.1f} ms")

nearest    PSNR 26.94 dB     0.8 ms
bilinear   PSNR 30.11 dB     1.6 ms
bicubic    PSNR 31.07 dB     3.9 ms
lanczos4   PSNR 31.32 dB     9.7 ms

The pattern in Output 5.3.3a generalizes well: bilinear over nearest is a large, cheap win; bicubic over bilinear is a modest win photographers care about; Lanczos over bicubic is a subtle win that matters mostly for archival upscales and text. Table 5.3.1 summarizes the decision space, including the two special-purpose rows that the benchmark cannot show.

5. The Downscaling Trap Advanced

Everything so far assumed we are reading values between samples, upscaling or warping at roughly constant scale. Downscaling is a different physics problem. Shrinking an image by 8× means the new sampling grid is 8× coarser, and the sampling theorem from Chapter 4 is unambiguous: frequencies above the new, lower Nyquist limit must be removed before resampling, or they will fold back as aliasing. A bilinear or bicubic kernel evaluated at the new sample positions still only reads 2 or 4 input pixels per axis; at an 8× shrink it simply skips over 6 of every 8 pixels, and the skipped detail reappears as moiré patterns, shimmering textures, and crawling edges.

The correct procedure is to low-pass filter to the new band limit, then sample, and good libraries fuse the two: cv2.INTER_AREA averages the source pixels covered by each destination pixel, while torchvision's antialias=True stretches the reconstruction kernel by the scale factor. The same idea, executed octave by octave, is the Gaussian pyramid from Chapter 4. Whenever you shrink by more than about 2× with anything other than these, you are aliasing, whether you notice it or not.

6. Interpolation and Machine Learning Advanced

A modern wrinkle: resizing is now part of nearly every model's input contract, so interpolation choices silently shape what networks learn and how they are scored. Three concrete consequences are worth carrying forward. First, masks versus images: augmentation pipelines must resize images with bilinear or bicubic but masks with nearest, and frameworks let you specify both (a distinction we will rely on in Chapter 21). Second, train-serve skew: if training resized with Pillow bicubic ($a=-0.5$) and production resizes with OpenCV bicubic ($a=-0.75$), the model sees a slightly different image distribution in production; reproducible pipelines pin the resize implementation, not just the size. Third, evaluation: generative-model metrics compare statistics of resized images, so the resize kernel becomes part of the metric itself, a story told in the callout below and continued when we reach FID in Chapter 37.

One more perspective ties this section to the chapter's arc. We treated interpolation as reading a value at one fractional point, but a warp does this for every output pixel at once, and the pattern of fractional positions varies across the image: a perspective warp compresses some regions (locally a downscale, aliasing risk) while stretching others (locally an upscale, blur risk). Production warpers therefore sometimes blend strategies or warp through an image pyramid from Chapter 4. With values-at-fractional-coordinates solved, we are ready to assemble the full warping algorithm in Section 5.4.