Contours, Moments & Shape Descriptors

"Translate me, rotate me, shrink me to a thumbnail: my seven numbers will not budge. Hold me up to a mirror, though, and exactly one of them changes its mind."

A Defiantly Invariant Hu Moment

Section 6.5 collapsed shapes onto their centerlines; this final section walks their edges. We have met boundaries twice already: the morphological gradient of Section 6.3 produced a rim of boundary pixels, and Section 6.4's regionprops quietly consumed boundaries to report perimeters and orientations it did not explain. Both left the boundary as an unordered set. The upgrade in this section is order: a contour is the boundary traced in sequence, a closed polygon you can walk, simplify, integrate over, and compare. Order is what turns a rim into geometry, and geometry into recognition, completing the journey this chapter began when a threshold from Chapter 2 first split the world into foreground and background.

1. From Pixel Sets to Boundaries: Border Following Intermediate

The standard tracing algorithm is Suzuki and Abe's border following (1985), the engine inside cv2.findContours. It raster-scans the binary image for a foreground pixel whose left neighbor is background (a border start), then walks counterclockwise around the component, keeping foreground on one side, until it returns to the start. Crucially, it does this for every border in the image, outer boundaries and hole boundaries alike, and records who surrounds whom. The connectivity conventions of Section 6.1 are baked in exactly as Rosenfeld prescribed: 8-connected foreground borders, 4-connected holes, which is why a diagonal chain of pixels traces as one contour rather than many.

The "who surrounds whom" record is the contour hierarchy, and choosing how much of it to keep is the first API decision. cv2.RETR_EXTERNAL keeps only outermost boundaries (fastest, blind to holes); cv2.RETR_LIST returns every contour with no structure; cv2.RETR_CCOMP arranges them into exactly two levels, outer boundaries and their holes, which is the workhorse mode for parts on a tray; and cv2.RETR_TREE keeps the full nesting tree, which a washer lying inside a larger ring genuinely needs. The companion flag cv2.CHAIN_APPROX_SIMPLE compresses straight runs of boundary pixels to their endpoints (a rectangle becomes 4 points instead of hundreds) while cv2.CHAIN_APPROX_NONE keeps every boundary pixel. Figure 6.6.1 shows a three-deep scene and the tree that RETR_TREE builds from it, including the four-integer record [next, previous, first_child, parent] that OpenCV returns per contour.

The code below traces the scene of Figure 6.6.1 and walks the hierarchy, computing each contour's depth by following parent pointers. Depth parity then classifies every contour without any geometry at all, an idea Section 6.4's Euler number expressed as a single global count and the hierarchy now itemizes:

import cv2

bw = cv2.imread("plate_scene.png", cv2.IMREAD_GRAYSCALE)
contours, hierarchy = cv2.findContours(bw, cv2.RETR_TREE,
                                       cv2.CHAIN_APPROX_SIMPLE)
print(f"{len(contours)} contours found")
for i, (nxt, prev, child, parent) in enumerate(hierarchy[0]):
    depth, p = 0, parent
    while p != -1:                      # climb parent links to the root
        depth += 1
        p = hierarchy[0][p][3]
    kind = "object border" if depth % 2 == 0 else "hole border"
    print(f"C{i}: depth {depth} ({kind:13s}) "
          f"{len(contours[i]):4d} pts  child={child:2d} parent={parent:2d}")
# Output:
# 4 contours found
# C0: depth 0 (object border )  482 pts  child= 1 parent=-1
# C1: depth 1 (hole border   )  168 pts  child= 2 parent= 0
# C2: depth 2 (object border )   62 pts  child=-1 parent= 1
# C3: depth 0 (object border )  139 pts  child=-1 parent=-1

2. Contour Geometry: Perimeter, Area & Polygon Simplification Beginner

Once a boundary is an ordered polygon, classical geometry applies verbatim. cv2.contourArea evaluates the shoelace formula (Green's theorem specialized to polygons):

$$ A \;=\; \frac{1}{2} \,\Bigl|\, \sum_{i=0}^{n-1} \bigl( x_i\, y_{i+1} - x_{i+1}\, y_i \bigr) \Bigr|, $$

which is why its answer differs slightly from the pixel count Section 6.4 reported as area: one integrates a polygon threaded through pixel centers, the other counts squares. The discrepancy is worth a digit of caution on small blobs and nothing more. Perimeter via cv2.arcLength deserves more caution: summing pixel steps along a staircase systematically overestimates the smooth boundary length (a digital diagonal line of length $L$ measures up to about 8 percent long), so any spec written in millimeters of perimeter should be calibrated against parts of known size rather than trusted absolutely. Derived ratios inherit the bias; we return to this in Exercise 6.6.3.

The third primitive is simplification. The Douglas-Peucker algorithm inside cv2.approxPolyDP recursively keeps only vertices that deviate from a straight chord by more than a tolerance $\varepsilon$, typically set as a fraction of the perimeter. It converts a noisy 400-point boundary into the 4-point quadrilateral that the document scanner of Chapter 5 hunted for, or the 6-point hexagon that will identify a nut in Section 5 below. Together with the convex hull (cv2.convexHull) these produce the working measurements of contour analysis:

import cv2
import numpy as np

contours, _ = cv2.findContours(bw, cv2.RETR_EXTERNAL,
                               cv2.CHAIN_APPROX_SIMPLE)
c = max(contours, key=cv2.contourArea)            # largest object

area  = cv2.contourArea(c)                        # shoelace / Green's theorem
perim = cv2.arcLength(c, closed=True)
approx = cv2.approxPolyDP(c, 0.02 * perim, closed=True)
hull   = cv2.convexHull(c)

print(f"area {area:.0f} px^2, perimeter {perim:.1f} px")
print(f"vertices after Douglas-Peucker: {len(approx)}")
print(f"circularity {4*np.pi*area/perim**2:.3f}, "
      f"solidity {area/cv2.contourArea(hull):.3f}")
# Representative output for a slightly dog-eared rectangular label:
# area 50412 px^2, perimeter 916.4 px
# vertices after Douglas-Peucker: 4
# circularity 0.754, solidity 0.988

3. Moments & the Invariance Chain Intermediate

Geometry measures a shape as placed; recognition needs numbers that survive placement. The instrument is the family of image moments, borrowed directly from mechanics: treat the foreground as a sheet of unit mass and compute

$$ m_{pq} \;=\; \sum_{(x,\,y) \in A} x^p\, y^q . $$

The zeroth moment $m_{00}$ is the area; the first moments give the centroid $\bar x = m_{10}/m_{00}$, $\bar y = m_{01}/m_{00}$, the formula Section 6.4 promised we would derive. Second moments describe how mass spreads around the centroid, and from them comes the orientation that regionprops reports: the major axis of the best-fit ellipse sits at

$$ \theta \;=\; \tfrac{1}{2}\, \operatorname{atan2}\!\bigl( 2\mu_{11},\; \mu_{20} - \mu_{02} \bigr), $$

where the $\mu_{pq}$ are the central moments, the same sums computed about the centroid, $\mu_{pq} = \sum (x - \bar x)^p (y - \bar y)^q$. Centering is the first rung of a ladder that Figure 6.6.2 lays out in full. Central moments do not change when the object translates. Dividing by the right power of area, $\eta_{pq} = \mu_{pq} / m_{00}^{\,1 + (p+q)/2}$, yields normalized central moments that also ignore scale. The final rung is Hu's 1962 construction: seven polynomial combinations of the $\eta_{pq}$ up to third order that are additionally unchanged by rotation. The first two are

$$ \phi_1 = \eta_{20} + \eta_{02}, \qquad \phi_2 = (\eta_{20} - \eta_{02})^2 + 4\eta_{11}^2, $$

and five more follow in the same spirit. The seventh is special: it is a skew invariant whose magnitude survives reflection but whose sign flips when the shape is mirrored, because mirroring negates every central moment with an odd power of one coordinate, and $\phi_7$ alone is built from an odd combination. One bit of asymmetry survives all that quotienting, and Section 5's practical example will spend exactly that bit.

Hu invariants have a wide dynamic range ($\phi_1$ might be $10^{-3}$ while $\phi_7$ is $10^{-21}$), so in practice everyone compares them on a signed log scale, $h_i = -\operatorname{sign}(\phi_i)\,\log_{10}\lvert\phi_i\rvert$. The demonstration below puts the whole chain to the test: a wrench mask is rotated 37 degrees and shrunk to 60 percent, then mirrored, and the log-scale invariants are printed for all three. Watch the last column:

import cv2
import numpy as np

def log_hu(mask):
    """Seven Hu invariants on a signed log10 scale."""
    hu = cv2.HuMoments(cv2.moments(mask, binaryImage=True)).flatten()
    return -np.sign(hu) * np.log10(np.abs(hu) + 1e-30)

mask = cv2.imread("wrench_mask.png", cv2.IMREAD_GRAYSCALE)
h, w = mask.shape
M = cv2.getRotationMatrix2D((w/2, h/2), angle=37, scale=0.6)
moved    = cv2.warpAffine(mask, M, (w, h))    # rotated + scaled copy
mirrored = cv2.flip(mask, 1)                  # left-right reflection

for name, m in [("original", mask), ("rot 37, scale 0.6", moved),
                ("mirrored", mirrored)]:
    print(f"{name:18s}", np.round(log_hu(m), 2))
# Representative output (phi_1 .. phi_7, log scale):
# original           [ 2.95  6.61  9.69 10.31 20.43 13.66 -20.65]
# rot 37, scale 0.6  [ 2.95  6.60  9.70 10.33 20.47 13.66 -20.70]
# mirrored           [ 2.95  6.61  9.69 10.31 20.43 13.66  20.65]

4. A Descriptor Vector for Every Blob Beginner

Moments are one family of shape numbers; the geometry of Section 2 supplies the rest. In practice a small, interpretable descriptor vector covers an enormous range of industrial decisions. Circularity, $4\pi A / P^2$, is 1 for a perfect disk and falls as the boundary wrinkles or elongates. Aspect ratio from cv2.minAreaRect captures elongation without caring about orientation. Extent (area over bounding-box area) separates crosses from blobs; solidity (area over convex-hull area) drops the moment a bite, crack, or concavity appears; the Douglas-Peucker vertex count reads off polygonal class directly (4 means quadrilateral, 6 means hexagon); and the hierarchy's hole count from Section 1 is the topological feature no amount of boundary noise can fake. Each is one or two lines on top of code we have already written, and each has a failure mode you can state in a sentence, which is exactly what makes the vector auditable.

When classes are too subtle for thresholds on individual descriptors, the next step up is comparing whole invariant vectors against labeled templates, which is nearest-neighbor classification in descriptor space and the conceptual ancestor of everything in Chapter 16. OpenCV ships the comparison ready-made:

5. The Complete Shape Classifier Intermediate

Now the payoff the chapter overview promised: a complete shape classifier in a page of code, every stage of which this chapter built. Threshold with Otsu (Chapter 2), clean with an opening (Section 6.3), trace with hierarchy (Section 1), describe (Sections 2 to 4), and decide with rules a quality auditor can read aloud. The parts tray contains screws, washers, and hex nuts; topology and three descriptors are enough to tell them apart:

import cv2
import numpy as np

def describe(c):
    """Descriptor vector for one outer contour."""
    area  = cv2.contourArea(c)
    perim = cv2.arcLength(c, True)
    circularity = 4 * np.pi * area / perim**2 if perim > 0 else 0.0
    solidity = area / cv2.contourArea(cv2.convexHull(c))
    (w, h) = cv2.minAreaRect(c)[1]                  # oriented box sides
    aspect = max(w, h) / max(min(w, h), 1e-6)
    verts  = len(cv2.approxPolyDP(c, 0.02 * perim, True))
    return dict(area=area, circularity=circularity,
                solidity=solidity, aspect=aspect, vertices=verts)

def classify(d, has_hole):
    if has_hole and d["circularity"] > 0.80:        # round ring
        return "washer"
    if has_hole and 5 <= d["vertices"] <= 7:        # hexagonal ring
        return "nut"
    if not has_hole and d["aspect"] > 2.5:          # long and solid
        return "screw"
    return "reject"                                 # fail loud, not plausible

img = cv2.imread("tray.png", cv2.IMREAD_GRAYSCALE)
_, bw = cv2.threshold(img, 0, 255,
                      cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
se = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
bw = cv2.morphologyEx(bw, cv2.MORPH_OPEN, se)       # de-speckle first

contours, hier = cv2.findContours(bw, cv2.RETR_CCOMP,
                                  cv2.CHAIN_APPROX_SIMPLE)
counts = {}
for i, c in enumerate(contours):
    if hier[0][i][3] != -1:          # skip hole borders themselves
        continue
    if cv2.contourArea(c) < 200:     # dust gate
        continue
    has_hole = hier[0][i][2] != -1   # any child contour means a hole
    label = classify(describe(c), has_hole)
    counts[label] = counts.get(label, 0) + 1
print(counts)
# Representative output on a 24-piece kit tray:
# {'screw': 12, 'washer': 8, 'nut': 4}

Three habits in this code repay study. The reject class exists because a rule-based system should fail loudly on the unexpected, never shrug a foreign object into the nearest class. The hole feature comes from the hierarchy, not from any intensity heuristic, so grease stains cannot fake it. And every threshold (0.80, 2.5, 200) traces to a measurable property of golden samples, so recalibrating for a new part is a measurement session, not a retraining run. When classes finally overlap in every descriptor you can write down, that is the boundary of this method and the on-ramp to the learned features of Chapter 10 and the classifiers of Chapter 16.