The Canny Edge Detector, Step by Step

"I accept strong edges on the spot. Weak edges need a strong edge to vouch for them. It's not paranoia; it's hysteresis."

A Canny Detector With Trust Issues

The previous section ended with a measured indictment: a thresholded gradient map keeps thick ribbons instead of curves and shatters real contours no matter where the threshold sits. This section builds the machine that fixes both defects. We will walk the four stages in order, implement the whole detector from scratch in NumPy, verify it against OpenCV's, and finish with the one-line call and its tuning knobs.

1. What Should an Edge Detector Optimize? Intermediate

Canny's contribution begins before any code, with three criteria for judging any edge detector applied to a noisy step edge:

Good detection. Maximize the probability of marking real edges and minimize the probability of marking noise. Good localization. Marked points should sit as close as possible to the true edge position. Single response. One edge should produce one mark, not a cluster.

The criteria fight each other. Detection improves with more smoothing (noise dies faster than the step does), but localization worsens, because smoothing smears the step's position; this is the $\sigma$ trade-off that Section 9.1 flagged. Canny formalized the trade-off, solved for the 1D operator maximizing a product of the criteria, and found the optimum is approximated within about 20 percent by something we already own: the first derivative of a Gaussian, exactly the "smooth, then differentiate" composite of Chapter 3. The deep reassurance of the result is that the standard practice was not just convenient; it is close to provably the best a linear filter can do. What the optimal filter cannot do alone is satisfy criterion three and produce thin, connected curves; that requires the two nonlinear stages that make Canny's pipeline famous. Figure 9.2.1 lays out all four stages.

2. Stages 1 and 2: Smooth, Then Measure Intermediate

Nothing new is needed here, only a deliberate choice. Convolve with a Gaussian of scale $\sigma$, then apply Sobel kernels to get $I_x$, $I_y$, and from them magnitude and orientation. The single decision is $\sigma$, and it should be made consciously: $\sigma \approx 1$ to $1.4$ preserves fine texture edges; $\sigma \approx 2$ to $3$ is a sensible default for natural photographs; larger values are for finding only the dominant contours. One practical warning that surprises many users: cv2.Canny performs no internal Gaussian smoothing; it runs Sobel directly on whatever you pass it. Smoothing is your job (or use skimage.feature.canny, which takes an explicit sigma argument). Feeding a raw noisy image to cv2.Canny and wondering about speckle is among the most common edge-detection bugs in production code, a sibling of the denoise-first discipline from Chapter 7.

3. Stage 3: Non-Maximum Suppression Advanced

Stage 3 attacks the thick ribbons. The insight is geometric: crossing an edge perpendicularly, gradient magnitude rises, peaks at the true boundary, and falls. The ribbon exists because we kept the whole rise and fall; the boundary is just the crest. So at every pixel, look one step in the gradient direction and one step against it, and keep the pixel only if its magnitude is at least as large as both neighbors. The result is called non-maximum suppression (NMS): everything that is not a local maximum along the gradient direction is deleted, collapsing each ribbon to a one-pixel crest line exactly perpendicular to the local gradient, the geometry established in Figure 9.1.2 of Section 9.1.

On a discrete grid "one step in the gradient direction" needs care: a gradient at 37 degrees points between pixels. The classical implementation quantizes orientation into four bins (0, 45, 90, 135 degrees), each selecting one of the four neighbor pairs in the 3×3 neighborhood; finer implementations interpolate magnitude between the two pixels straddling the true direction. Figure 9.2.2 shows the quantized version that our from-scratch code implements.

4. Stage 4: Double Threshold and Hysteresis Advanced

Stage 4 attacks the broken contours, and it does so by refusing to use one threshold. Pick two, $T_{\text{high}}$ and $T_{\text{low}}$ (Canny suggested a ratio between 2:1 and 3:1). Pixels above $T_{\text{high}}$ are strong: accepted unconditionally. Pixels below $T_{\text{low}}$ are rejected unconditionally. The pixels in between are weak: accepted if and only if they connect, directly or through a chain of other weak pixels, to some strong pixel. Implementation is a flood fill seeded at the strong pixels and allowed to spread only through weak ones; readers of Chapter 6 will recognize this as morphological reconstruction of the weak map from strong markers.

Why this works is worth saying plainly. A real contour whose contrast dips for a stretch produces weak pixels precisely along the dip, sandwiched between strong segments, so the flood fill recovers it whole. An isolated noise response has the same magnitude as that dip but no strong sponsor anywhere nearby, so it dies. The decision uses geometry that no per-pixel threshold can see: membership in a connected structure.

5. The Whole Detector From Scratch Advanced

Forty lines of NumPy implement everything above. The code keeps each stage visibly separate so you can instrument any of them, and then checks itself against OpenCV's implementation.

import cv2
import numpy as np
from collections import deque

def canny_from_scratch(gray, sigma=1.4, low=40, high=90):
    # Stage 1: Gaussian smoothing (sigma chosen by the caller, deliberately)
    img = cv2.GaussianBlur(gray.astype(np.float32), (0, 0), sigma)

    # Stage 2: gradient magnitude and orientation
    gx = cv2.Sobel(img, cv2.CV_32F, 1, 0, ksize=3)
    gy = cv2.Sobel(img, cv2.CV_32F, 0, 1, ksize=3)
    mag = np.hypot(gx, gy)
    ang = np.rad2deg(np.arctan2(gy, gx)) % 180.0      # fold to [0, 180)

    # Stage 3: non-maximum suppression with 4-bin orientation quantization
    q = (((ang + 22.5) // 45).astype(int)) % 4         # 0,45,90,135 deg bins
    offs = [(0, 1), (-1, 1), (-1, 0), (-1, -1)]        # neighbor along gradient
    H, W = mag.shape
    nms = np.zeros_like(mag)
    for y in range(1, H - 1):
        for x in range(1, W - 1):
            dy, dx = offs[q[y, x]]
            m = mag[y, x]
            if m >= mag[y + dy, x + dx] and m >= mag[y - dy, x - dx]:
                nms[y, x] = m                          # crest pixels survive

    # Stage 4: double threshold + hysteresis (flood fill from strong seeds)
    strong = nms >= high
    weak = (nms >= low) & ~strong
    edges = strong.copy()
    queue = deque(zip(*np.nonzero(strong)))
    while queue:
        y, x = queue.popleft()
        for dy in (-1, 0, 1):
            for dx in (-1, 0, 1):
                yy, xx = y + dy, x + dx
                if 0 <= yy < H and 0 <= xx < W and weak[yy, xx] and not edges[yy, xx]:
                    edges[yy, xx] = True               # strong recruits weak
                    queue.append((yy, xx))
    return edges

gray = cv2.imread("street.jpg", cv2.IMREAD_GRAYSCALE)
mine = canny_from_scratch(gray, sigma=1.4, low=40, high=90)
ref = cv2.Canny(cv2.GaussianBlur(gray, (0, 0), 1.4), 40, 90) > 0

print(f"edge pixels (ours): {mine.mean():.2%}   (OpenCV): {ref.mean():.2%}")
print(f"pixelwise agreement: {(mine == ref).mean():.1%}")
# Representative output:
# edge pixels (ours): 3.08%   (OpenCV): 3.21%
# pixelwise agreement: 99.3%

The Python loops make this implementation a teaching instrument, not a production one: it runs in seconds where OpenCV takes about a millisecond. But every line corresponds to a sentence in this section, and instrumenting it (dump the nms array, count how many weak pixels hysteresis rescued) is the fastest way to make the pipeline's behavior concrete. On a typical street photograph, roughly a third of the final edge pixels arrive via rescue rather than direct strong acceptance, which is the broken-contour defect of Section 9.1 being repaired in bulk.

6. The One-Liner and Its Knobs Intermediate

In production you call the library, and the craft moves into choosing arguments. The most durable trick is deriving the thresholds from image statistics, in the spirit of Chapter 2's histogram methods, so that one configuration survives across exposures and scenes:

import cv2
import numpy as np

gray = cv2.imread("street.jpg", cv2.IMREAD_GRAYSCALE)
blur = cv2.GaussianBlur(gray, (0, 0), 1.4)     # cv2.Canny does NOT smooth

# Auto-thresholds from the median: robust to exposure changes
v = float(np.median(blur))
low, high = int(max(0, 0.66 * v)), int(min(255, 1.33 * v))

edges = cv2.Canny(blur, low, high,
                  apertureSize=3,      # Sobel kernel size: 3, 5, or 7
                  L2gradient=True)     # true Euclidean magnitude, not |gx|+|gy|

print(f"median {v:.0f} -> thresholds ({low}, {high}); "
      f"edge pixels: {(edges > 0).mean():.2%}")
# Representative output:
# median 118 -> thresholds (77, 156); edge pixels: 2.64%