Worked Example: A Document Scanner from Scratch

"You photographed an A4 page as a trapezoid at a 40-degree angle, in restaurant lighting, with your thumb in frame. I will fix it. I always fix it. This is my whole personality now."

A Long-Suffering Document Scanner

The previous sections built the theory in the order textbooks like: models, coordinates, interpolation, execution, estimation. Real projects run in the opposite direction, starting from a goal: the user photographs a paper at an angle; produce a clean, flat, high-contrast scan. This section walks the full distance from one to the other, including the unglamorous parts (corner ordering, resolution bookkeeping) where real implementations actually break. Figure 5.6.1 shows the route.

1. Stage 1: Find the Page Outline Intermediate

Our scanner's "correspondence problem" is friendlier than Section 5.5's general matching: we know the object of interest is a quadrilateral that contrasts with the background. The classical detection recipe is blur, edge detection, and contour analysis. We downscale first, both for speed and because edge detectors behave more consistently at a standard working resolution; the crucial bookkeeping is to remember the scale factor, because the final warp must run on the full-resolution original. Detection can be lossy; rectification must not be.

import cv2
import numpy as np

PROC_HEIGHT = 600.0          # standard working height for detection

def find_page_quad(image_bgr):
    """Return the page's 4 corners in FULL-RES coords, or None."""
    scale = PROC_HEIGHT / image_bgr.shape[0]
    small = cv2.resize(image_bgr, None, fx=scale, fy=scale,
                       interpolation=cv2.INTER_AREA)

    gray = cv2.cvtColor(small, cv2.COLOR_BGR2GRAY)
    gray = cv2.GaussianBlur(gray, (5, 5), 0)        # tame paper texture
    edges = cv2.Canny(gray, 75, 200)                # edge map
    edges = cv2.dilate(edges, np.ones((3, 3), np.uint8))  # bridge gaps

    contours, _ = cv2.findContours(edges, cv2.RETR_LIST,
                                   cv2.CHAIN_APPROX_SIMPLE)
    contours = sorted(contours, key=cv2.contourArea, reverse=True)[:5]

    for c in contours:                              # biggest first
        peri = cv2.arcLength(c, True)
        approx = cv2.approxPolyDP(c, 0.02 * peri, True)
        if len(approx) == 4 and cv2.isContourConvex(approx):
            if cv2.contourArea(approx) > 0.2 * small.size / 3:
                return approx.reshape(4, 2).astype(np.float64) / scale
    return None

Each ingredient earns its place. The Gaussian blur (from Chapter 3) suppresses paper grain and carpet texture that would otherwise fill the edge map with confetti. Canny, which we use here as a black box and dissect properly in Chapter 9, traces intensity discontinuities; the dilation closes one-pixel gaps in the page border so the contour is a single closed curve. approxPolyDP simplifies each candidate contour with the Douglas-Peucker algorithm at a tolerance of 2 percent of the perimeter: page outlines survive as exactly 4 vertices, while sleeves, mugs, and shadows rarely do. The convexity and minimum-area tests reject the rest.

2. Stage 2: Order the Corners Intermediate

The contour hands us four corners in an arbitrary cyclic order, but getPerspectiveTransform pairs source to destination points by index: if our first destination corner is "top-left", the first source corner had better actually be the page's top-left. Feed the points in a rotated or reflected order and you get a perfectly valid homography to an upside-down or mirror-imaged page. The classic ordering trick uses two scalar functions of each corner $(x, y)$: the sum $x + y$ is smallest at the top-left and largest at the bottom-right; the difference $y - x$ is smallest at the top-right and largest at the bottom-left.

def order_corners(pts):
    """pts: (4, 2) array in any order -> [tl, tr, br, bl]."""
    s = pts.sum(axis=1)            # x + y
    d = np.diff(pts, axis=1)[:, 0] # y - x
    return np.array([pts[np.argmin(s)],    # top-left
                     pts[np.argmin(d)],    # top-right
                     pts[np.argmax(s)],    # bottom-right
                     pts[np.argmax(d)]],   # bottom-left
                    dtype=np.float32)

3. Stage 3: Size the Output and Warp Intermediate

What size should the flattened page be? We measure the quadrilateral's edges in the photo and take the maximum of opposite sides as the output width and height. This preserves as much resolution as the photo captured and gets the aspect ratio approximately right. Only approximately: perspective foreshortening means the photographed side lengths are not the true paper proportions. Recovering the exact aspect ratio of a rectangle from one perspective view is possible, but it requires the camera's focal length, which belongs to the calibration story of Chapter 12. Production apps either do that or simply snap to known paper ratios (A4, Letter); we take the honest approximation.

def rectify(image_bgr, quad):
    """Warp the quadrilateral region into a flat, axis-aligned scan."""
    tl, tr, br, bl = order_corners(quad)

    W = int(max(np.linalg.norm(br - bl), np.linalg.norm(tr - tl)))
    H = int(max(np.linalg.norm(tr - br), np.linalg.norm(tl - bl)))

    src = np.array([tl, tr, br, bl], dtype=np.float32)
    dst = np.array([[0, 0], [W - 1, 0],
                    [W - 1, H - 1], [0, H - 1]], dtype=np.float32)

    M = cv2.getPerspectiveTransform(src, dst)   # 4 pairs -> 8 DoF, exact
    return cv2.warpPerspective(image_bgr, M, (W, H),
                               flags=cv2.INTER_LINEAR)

Pause on getPerspectiveTransform for a moment, because it closes a loop opened in Section 5.1: a homography has 8 degrees of freedom, each point pair supplies 2 equations, and 4 pairs make the system exactly determined, so the function solves a small linear system and returns the unique homography through our corners. No RANSAC is needed here, unlike Section 5.5, because we have exactly four correspondences and trust all of them; the robustness lives upstream in the contour tests. The warp call then runs the inverse-mapping gather of Section 5.4 with the bilinear kernel of Section 5.3.

4. Stage 4: Binarize Like a Scanner Beginner

A geometric rectangle of a photo still looks like a photo: gray paper, uneven lighting, a shadow from your hand. The "scanned document" look is a thresholding problem, and the right tool is the adaptive thresholding of Chapter 2, which computes a local threshold per neighborhood and therefore shrugs off illumination gradients that destroy any single global threshold:

def to_scan(image_bgr):
    gray = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2GRAY)
    return cv2.adaptiveThreshold(gray, 255,
                                 cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
                                 cv2.THRESH_BINARY,
                                 blockSize=21, C=10)

# The complete scanner, end to end:
image = cv2.imread("receipt_photo.jpg")
quad = find_page_quad(image)
if quad is None:
    raise SystemExit("No document found: check contrast with background")
flat = rectify(image, quad)
scan = to_scan(flat)
cv2.imwrite("scan.png", scan)
print(f"saved {scan.shape[1]}x{scan.shape[0]} scan")

saved 1187x1684 scan

And that is the entire scanner: roughly one hundred lines including comments, no machine learning, latency dominated by the single full-resolution warp. The binarized output often shows speckle noise from paper texture and dust; cleaning that up with a morphological opening is literally the first worked example of the next chapter, which picks up this exact image.

from imutils.perspective import four_point_transform
flat = four_point_transform(image, quad.reshape(4, 2))

5. Failure Modes and Hardening Advanced

Turning this demo into a product is mostly about the inputs that break it. Four failure classes account for nearly everything, and each maps to a specific upgrade path:

• Low edge contrast. White paper on a white desk gives Canny nothing. Mitigations: try multiple threshold pairs, run detection per color channel and on a saturation channel, or fall back to asking the user to tap the corners. The durable fix is replacing stage 1 with a segmentation model, the approach of Chapter 24, which is exactly what modern phone scanners do.
• Distractor quadrilaterals. Laptops, monitors, and floor tiles are large convex quads. Mitigations: prefer the quad containing the image center, score candidates by text-like high-frequency content inside, or track stability across video frames.
• Non-planar pages. Books near the spine, curled receipts. The homography is structurally wrong; see the practical example and research frontier.
• Extreme angles. Beyond roughly 60 degrees of tilt, the far edge's effective resolution collapses; the warp magnifies a few hundred captured pixels into a thousand output pixels of mush. Detect by comparing opposite side lengths and prompt for a re-shoot; no interpolation from Section 5.3 can manufacture detail the sensor never sampled.

6. What This Project Taught Beginner

Walk back through the hundred lines and notice how the chapter's sections each carried a stage: the hierarchy (5.1) told us a photographed plane needs exactly a homography, no more, no less; homogeneous coordinates (5.2) are why getPerspectiveTransform returns a 3×3 matrix and why the warp divides by $W$; interpolation (5.3) fills every output pixel from fractional source positions; inverse mapping (5.4) is the reason the output has no holes; and the four corners are a tiny, trusted correspondence set, the same currency 5.5 earned with feature matching and RANSAC. One pipeline, five ideas, each load-bearing.

The scanner also hands the book its next problem. Its output is a binary image, and binary images have their own algebra: erosion to strip speckle, dilation to heal broken strokes, connected components to find characters, shape descriptors to classify them. That algebra is Chapter 6: Morphology, Binary Images & Shape, and it begins exactly where scan.png ends.