Part I: Image Processing
Chapter 4: The Frequency Domain & Multi-Scale Analysis

Frequency-Domain Filtering: Low-Pass, High-Pass & Notch

"I deleted every frequency above the cutoff, exactly as designed. The image rang like a struck bell. Nobody warned me that being decisive has side effects."

A Heavy-Handed Ideal Low-Pass Filter
Big Picture

Filtering in the frequency domain is nothing more than multiplying the spectrum, element by element, by a designed mask called a transfer function, and the entire craft is choosing a mask whose shape does not create artifacts of its own. Every spatial kernel from Chapter 3 secretly is such a mask, by the convolution theorem. Working with the mask directly buys you three things: filters too large to be practical as kernels, exact control over which frequencies live and die, and operations like periodic-noise removal that no small spatial kernel can express cleanly.

Section 4.2 built the transform machinery; this section uses it as a workbench. The recipe never changes: transform the image, multiply by a transfer function $H(u, v)$, transform back. What changes is the shape of $H$, and the section's running lesson is that the shape of the cut matters as much as where you cut. We will watch the most intuitive design (a hard cutoff) produce the worst artifacts, understand exactly why through the Gibbs phenomenon from Section 4.1, and end with the one filter family that spatial convolution genuinely cannot replace: the notch.

1. Filtering as Spectrum Sculpting Beginner

The full pipeline, with the shift conventions made explicit, is:

$$g(x, y) = \mathcal{F}^{-1}\Big[\, H(u, v) \cdot \mathcal{F}\big[f(x, y)\big] \,\Big]$$

where $H$ is built in centered coordinates (so it must multiply a shifted spectrum, per Figure 4.2.1). Since most useful filters care only about how far a frequency is from the center, not its direction, we first compute a radial distance grid and then express every filter as a function of it:

import numpy as np
from skimage import data

img = data.camera().astype(np.float64)

def radial_distance(shape):
    """Distance of each (centered) frequency bin from the spectrum center."""
    h, w = shape
    yy, xx = np.ogrid[:h, :w]
    return np.sqrt((yy - h / 2) ** 2 + (xx - w / 2) ** 2)

def apply_filter(image, H):
    """Transform, multiply by the centered transfer function H, invert."""
    F = np.fft.fftshift(np.fft.fft2(image))
    return np.real(np.fft.ifft2(np.fft.ifftshift(F * H)))

D = radial_distance(img.shape)        # same shape as img, zero at center
Code 4.3.1: The reusable filtering scaffold: a radial distance grid D and an apply_filter helper that handles both shifts and returns the real part. Every filter in this section is one expression in D away.

Before designing new masks, note what this view says about old friends. The Gaussian blur of Chapter 3 has a transfer function that is itself a Gaussian (wide kernel, narrow $H$, and vice versa); the box filter's $H$ is a sinc function that dips negative, which is the frequency-domain explanation of the box filter's odd artifacts on fine patterns. A filter's spatial and spectral faces are two views of one object, and you can design on whichever side is clearer, then translate.

2. Three Low-Pass Designs: Ideal, Butterworth, Gaussian Intermediate

A low-pass filter keeps frequencies near the center (smooth content) and attenuates the rest (detail, edges, noise). The three classical designs differ only in how abruptly they say no:

$$H_{\text{ideal}}(u,v) = \begin{cases} 1 & D(u,v) \le D_0 \\ 0 & D(u,v) > D_0 \end{cases} \qquad H_{\text{btw}}(u,v) = \frac{1}{1 + \big[D(u,v)/D_0\big]^{2n}} \qquad H_{\text{gauss}}(u,v) = e^{-D^2(u,v) / 2 D_0^2}$$

Here $D_0$ is the cutoff radius and $n$ the Butterworth order, which dials its steepness anywhere between Gaussian-gentle ($n = 1$) and ideal-brutal ($n \to \infty$). Figure 4.3.1 plots the three profiles; the only visual difference is the sharpness of the shoulder, and that shoulder is the whole story.

D (distance from spectrum center) H(D) 1 0 D₀ (cutoff) vertical cliff here → ringing in the image Ideal (step) Butterworth, n = 2 Gaussian
Figure 4.3.1: Radial profiles of the three classical low-pass transfer functions at the same cutoff. The ideal filter's vertical cliff produces spatial ringing; the Butterworth order n trades steepness against ringing; the Gaussian declines smoothly and rings not at all.
D0 = 30.0                                       # cutoff radius, in frequency bins
H_ideal  = (D <= D0).astype(np.float64)
H_butter = 1.0 / (1.0 + (D / D0) ** (2 * 2))    # Butterworth, order n = 2
H_gauss  = np.exp(-(D ** 2) / (2 * D0 ** 2))

lp_ideal  = apply_filter(img, H_ideal)    # blur + ripples hugging every edge
lp_butter = apply_filter(img, H_butter)   # blur, only a whisper of ringing
lp_gauss  = apply_filter(img, H_gauss)    # pure blur, zero ringing
Code 4.3.2: Three low-pass filters at identical cutoff D0 = 30. Display the three results side by side: the ideal output shows ghostly contours rippling outward from every strong edge, the Butterworth shows almost none, the Gaussian none at all.

Run the code and look closely at the ideal-filtered image: every strong edge has ripples radiating from it, like a pond after a stone. This is the ringing artifact, and it is the Gibbs phenomenon of Section 4.1 in its adult form. The hard spectral cutoff is itself an edge, and an edge in one domain is an oscillation in the other: the ideal filter's spatial form is a 2D sinc whose decaying lobes stamp themselves around every feature they convolve with.

Key Insight: Sharp in One Domain, Ringing in the Other

The spatial and frequency domains are bound by a duality: a discontinuity in either domain forces slowly decaying oscillations in the other. A hard frequency cutoff rings in space (ideal LPF); a hard spatial edge demands frequencies out to infinity (the square wave of Section 4.1). Every practical filter design is a negotiated settlement with this duality, and the Gaussian, which is smooth in both domains at once, is the negotiation's most famous diplomat. When you see ripples around edges in any processed image, suspect that something, somewhere, applied a too-sharp spectral cut.

3. High-Pass Filtering and Sharpening Intermediate

Every low-pass design yields a high-pass twin for free: $H_{\text{HP}} = 1 - H_{\text{LP}}$. A pure high-pass result keeps only edges and texture floating on a mid-gray void (the DC term is gone, so the mean is zero), which is useful for inspection but rarely the goal. The practical sharpening tool is high-frequency emphasis: keep the whole image and add back an extra helping of its high frequencies,

$$H_{\text{hfe}}(u, v) = a + b \cdot H_{\text{HP}}(u, v)$$

with $a \approx 1$ preserving the original and $b$ controlling the boost. This is precisely the unsharp masking of Chapter 3 wearing frequency-domain clothes: there you computed "image plus $b$ times (image minus blur)", and expanding that expression gives exactly $1 + b(1 - H_{\text{blur}})$ as a spectral multiplier.

H_hp  = 1.0 - H_gauss                  # Gaussian high-pass: smooth, no ringing
H_hfe = 1.0 + 1.2 * H_hp               # a = 1 keeps the image, b = 1.2 boosts detail

sharp = np.clip(apply_filter(img, H_hfe), 0, 255)
# Edges and fine texture visibly crisper; flat regions untouched, because
# H_hfe equals 1.0 exactly at the spectrum center where smooth content lives.
Code 4.3.3: Sharpening by high-frequency emphasis built on the ringing-free Gaussian high-pass. The parameter b is the strength dial; values beyond about 2 start amplifying noise faster than detail, a trade-off explored properly in Chapter 7.
Note: Homomorphic Filtering, the Classic Variant

A venerable cousin worth recognizing on sight: take the log of the image first, then high-pass filter, then exponentiate. Because illumination (smooth, low-frequency) and surface reflectance (detailed, high-frequency) combine multiplicatively, the log turns their product into a sum the filter can separate, evening out lighting while boosting detail. It is a frequency-domain answer to the uneven-illumination problem that Chapter 2 attacked with adaptive histogram methods, and a preview of the illumination-reflectance reasoning that returns in Chapter 7.

4. Notch Filters: Surgical Removal of Periodic Noise Advanced

Now for the filter family that justifies the whole frequency-domain detour. Periodic interference, scanner banding, electrical hum coupled into a sensor, halftone dots from printed sources, screen-door moiré, is spread across every pixel in the spatial domain, hopelessly entangled with the signal. But in the frequency domain, a periodic pattern is a point: one conjugate pair of bright spikes at its frequency (visible in Figure 4.3.2). A notch filter zeroes a small disk around each spike and leaves everything else untouched. No spatial kernel of reasonable size can do this; the notch's spatial equivalent is an oscillating pattern as large as the image itself.

interference spike (+60 cycles) conjugate twin (always zero both) notch: H = 0 inside, H = 1 everywhere else image energy stays horizontal banding at 60 cycles → spike pair on the vertical frequency axis
Figure 4.3.2: Anatomy of a notch filtering job. Periodic banding concentrates into one conjugate pair of spectral spikes (red); the notch mask zeroes small disks around both (dashed blue) while the natural image energy near the center passes untouched.
h, w = img.shape
yy, xx = np.mgrid[0:h, 0:w]
noisy = img + 25 * np.sin(2 * np.pi * 60 * yy / h)   # 60-cycle horizontal banding

def notch_mask(shape, centers, radius=6):
    """All-ones mask with zeroed disks at given (row, col) offsets from center."""
    hh, ww = shape
    Y, X = np.ogrid[:hh, :ww]
    mask = np.ones(shape, dtype=np.float64)
    for dv, du in centers:
        cy, cx = hh / 2 + dv, ww / 2 + du
        mask[(Y - cy) ** 2 + (X - cx) ** 2 <= radius ** 2] = 0.0
    return mask

# The banding lives at (v, u) = (+60, 0) and its conjugate (-60, 0)
H_notch = notch_mask(img.shape, centers=[(60, 0), (-60, 0)])
clean = apply_filter(noisy, H_notch)
# The banding vanishes completely; the tiny residual difference from the
# original is the sliver of true image energy that shared those two bins.
Code 4.3.4: Synthetic 60-cycle banding removed by zeroing two 6-pixel disks out of 262,144 spectrum bins. The same recipe with detected spike locations handles scanner hum, halftone patterns, and screen moiré.

In production you rarely hard-code spike positions: detect them as local maxima of the log-magnitude spectrum beyond some radius (excluding the natural center blob and axis streaks), then notch each detection and its conjugate. Soft-edged notches (a small inverted Gaussian rather than a hard zero disk) avoid introducing miniature ringing of their own, the same lesson as Figure 4.3.1 at a smaller scale.

Practical Example: A Century of Newsprint, Minus the Halftone

Who: The imaging lead of a national library's newspaper digitization program, scanning roughly 1.2 million pages spanning 1880 to 1990.

Situation: Old newspapers print photographs as halftone dot screens. Scanned at archival resolution, the dot lattice survives, and it both degrades OCR on adjacent text and produces severe moiré when pages are rendered at reading sizes on the archive's website.

Problem: Gaussian blurring strong enough to erase the dots also erased small type. Median filtering helped the text but left blotchy artifacts in the photographs. Every spatial filter faced the same impossible trade, because the halftone occupies the same spatial neighborhoods as the content.

Decision: The team added an automated notch stage: detect the halftone's spectral peaks per scan (the dot screen's frequency varied by decade and printer), zero each peak pair with a soft Gaussian notch, and only then apply a mild low-pass tuned for the rendering size.

Result: OCR character error rate on photo-adjacent text fell from 8.9 to 3.6 percent across the validation sample, moiré complaints from the web archive's readers effectively stopped, and the stage added only milliseconds per page since it rode on FFTs already computed for quality checks.

Lesson: Interference that is periodic is a point defect in frequency space. Find the points, remove the points, and the spatial domain heals around them.

Library Shortcut: skimage.filters.butterworth

Our scaffold plus filter construction comes to roughly 30 lines once you add padding and color handling. scikit-image ships the whole pipeline as one call:

from skimage.filters import butterworth

low  = butterworth(img, cutoff_frequency_ratio=0.06, high_pass=False, order=2)
high = butterworth(img, cutoff_frequency_ratio=0.06, high_pass=True,  order=2)
Code 4.3.5: Butterworth low-pass and high-pass filtering through scikit-image's one-call API; the cutoff is specified as a fraction of the maximum representable frequency.

A 30-to-1 reduction for the standard cases. Internally it builds the squared Butterworth response, performs the forward and inverse FFTs, supports optional edge padding (npad) to suppress boundary artifacts from the DFT's implicit tiling, and handles color images via channel_axis. Notch filtering, being inherently bespoke, is the one design you will still assemble by hand from Code 4.3.4's parts.

Research Frontier: Learned and Scheduled Spectral Filters

Hand-designed transfer functions have learned descendants. Global Filter Networks (GFNet, NeurIPS 2021) make $H(u,v)$ a trainable parameter tensor: the network literally learns a bank of frequency-domain masks as its token mixer, and the learned masks turn out smooth and low-pass-heavy, echoing this section's designs. Fourier Domain Adaptation (FDA, CVPR 2020) filters between domains, transplanting low-frequency amplitude from real photos onto synthetic renders. And the idea of scheduling a low-pass cutoff over time now regulates 3D reconstruction: FreGS (CVPR 2024) anneals frequency content from coarse to fine while optimizing 3D Gaussian splats, preventing early overfitting to high frequencies, conceptually the same coarse-to-fine discipline that drives the pyramids of Section 4.5 and, one part later, the noise schedules of diffusion models, which destroy high frequencies first and regenerate them last.

Exercise 4.3.1: Filter Forensics Conceptual

You receive three blurred versions of the same photograph, produced by an ideal, a Butterworth (n = 2), and a Gaussian low-pass filter at the same cutoff, but the filenames are scrambled. Describe the visual evidence you would use to assign each image to its filter, and explain why examining regions near strong edges is more diagnostic than examining smooth regions.

Exercise 4.3.2: Automatic Notch Detection Coding

Extend Code 4.3.4 into an automatic pipeline: corrupt an image with two superimposed sinusoids of unknown (to your code) frequency and orientation, find the interference spikes as local maxima of the log-magnitude spectrum at radius greater than 20 bins from the center, place a soft Gaussian notch over each spike and its conjugate, and report the PSNR (Chapter 1's metric) of the restoration against the original. Compare against the PSNR achievable by the best Gaussian low-pass you can tune.

Exercise 4.3.3: The Ringing-Steepness Trade-off Analysis

For Butterworth orders n = 1, 2, 4, 8, 16 at fixed D0, filter a test image containing one strong vertical edge. For each n, measure (a) the amplitude of the largest ripple within 20 pixels of the edge and (b) the residual energy above the cutoff frequency. Plot (a) against (b) and identify the order you would choose for a medical-imaging viewer where false ripple structure is unacceptable. Justify with your plot.