Image Arithmetic, Blending & Compositing

"I am 30 percent mountain, 70 percent weather graphics, and 100 percent committed to this relationship."

A Partially Transparent Alpha Channel

With Section 2.1 through Section 2.4 behind us, we can curve, equalize, and binarize a single image. The natural next step is arithmetic between images: given two aligned frames $f_1$ and $f_2$, compute $f_1 + f_2$, $f_1 - f_2$, or $\alpha f_1 + (1-\alpha) f_2$ at every pixel. These operations stay within the point-operation family (each output pixel depends only on the input pixels at the same location), but the inputs now come from multiple images, which is where the first trap lies waiting.

1. The Arithmetic Trap: uint8 Overflow Basic

As established in Chapter 0, images arrive as uint8 arrays, and uint8 arithmetic in NumPy is modular: it wraps around at 256 rather than stopping at 255. Adding two bright pixels yields a dark one; subtracting a larger value from a smaller one yields a bright one. OpenCV's arithmetic functions instead saturate, clamping results into $[0, 255]$. The two-line demonstration below is worth running once in your life, because the bug it represents is endemic:

import numpy as np
import cv2

a = np.array([[200]], dtype=np.uint8)
b = np.array([[100]], dtype=np.uint8)

print(a + b)            # [[44]]   NumPy wraps: (200 + 100) % 256 = 44
print(cv2.add(a, b))    # [[255]]  OpenCV saturates: min(300, 255)

print(b - a)                 # [[156]]  NumPy wraps: (100 - 200) % 256
print(cv2.subtract(b, a))    # [[0]]    OpenCV saturates: max(-100, 0)
print(cv2.absdiff(a, b))     # [[100]]  |200 - 100|, always safe

The symptoms of wraparound in real pipelines are unforgettable: brightened skies turn black in blotches, difference images light up like neon. The rules that prevent it: use cv2.add, cv2.subtract, cv2.absdiff, and cv2.addWeighted for uint8 work; or convert to a wider type first (astype(np.float32) or np.int16), compute, then clip and convert back, exactly the float-clip-convert discipline from Section 2.1.

2. Blending: The Weighted Average Basic

The most useful two-image operation is the convex combination

$$g = \alpha f_1 + (1 - \alpha) f_2, \qquad \alpha \in [0, 1]$$

which crossfades between the images: $\alpha = 1$ shows pure $f_1$, $\alpha = 0$ pure $f_2$, and anything between is a dissolve. One call does it with saturation handled:

import cv2
import numpy as np

day   = cv2.imread("street_day.jpg")
night = cv2.imread("street_night.jpg")     # same size and dtype

# A 60-frame crossfade from day to night:
frames = []
for i in range(60):
    alpha = 1.0 - i / 59.0                  # 1.0 -> 0.0
    mix = cv2.addWeighted(day, alpha, night, 1.0 - alpha, 0.0)
    frames.append(mix)

# Watermarking is the same operation with a high alpha:
logo_bg = cv2.addWeighted(day, 0.92, np.full_like(day, 255), 0.08, 0.0)

Blending has a second life far from video editing: averaging $N$ aligned noisy frames divides the noise standard deviation by $\sqrt{N}$, the cheapest denoiser ever invented and a staple of astrophotography and microscopy (a theme developed properly in Chapter 7). And in deep learning, the mixup augmentation blends pairs of training images (and their labels) with a random $\alpha$, exactly this section's formula deployed as a regularizer, which we will meet among the training recipes of Chapter 21.

3. Difference Images: Seeing Change Intermediate

Subtracting two aligned images of the same scene isolates what changed between them. With a clean reference frame (an empty corridor, a bare conveyor belt), cv2.absdiff against the live frame highlights everything new, and a threshold from Section 2.4 turns the difference into a binary change mask. This three-line pattern is the ancestral form of motion detection and still runs in countless deployed systems:

import cv2

background = cv2.imread("empty_corridor.png", cv2.IMREAD_GRAYSCALE)
frame      = cv2.imread("corridor_now.png",  cv2.IMREAD_GRAYSCALE)

diff = cv2.absdiff(frame, background)              # |frame - background|
_, motion_mask = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)

changed_fraction = motion_mask.mean() / 255.0
if changed_fraction > 0.02:                        # >2% of pixels changed
    print(f"motion: {changed_fraction:.1%} of frame")   # e.g. motion: 7.3%

The static-reference assumption is, predictably, the weak point: lighting drifts, trees sway, and the "empty" scene never stays empty. Production systems therefore maintain a running statistical model of the background per pixel (OpenCV ships cv2.createBackgroundSubtractorMOG2, a per-pixel Gaussian mixture), and the full story of motion, including optical flow that estimates where each pixel went rather than just whether it changed, belongs to Chapter 15.

4. Alpha Compositing: The Over Operator Advanced

Blending with one global $\alpha$ dissolves whole images into each other. Compositing generalizes $\alpha$ into a per-pixel coverage map: a fourth channel, the alpha channel, where 1 means fully opaque, 0 fully transparent, and fractions mean partial coverage (the soft edge of hair, the translucency of glass, the antialiased boundary of a rendered logo). Porter and Duff's 1984 paper defined an algebra of twelve ways to combine two such images; the one that conquered the world is over. Placing foreground $F$ (color $C_f$, alpha $\alpha_f$) over background $B$ (color $C_b$, alpha $\alpha_b$):

$$\alpha_o = \alpha_f + \alpha_b (1 - \alpha_f), \qquad C_o = \frac{\alpha_f C_f + \alpha_b C_b (1 - \alpha_f)}{\alpha_o}$$

The structure is exactly what physical intuition demands: the foreground contributes in proportion to its coverage $\alpha_f$, and the background shows through only the remaining $(1 - \alpha_f)$. With an opaque background ($\alpha_b = 1$), the color formula collapses to the familiar $C_o = \alpha_f C_f + (1 - \alpha_f) C_b$. Figure 2.5.1 traces the operator's data flow.

The implementation below covers the overwhelmingly common case of an opaque background, where the division by $\alpha_o$ disappears. Watch the imread flag: OpenCV's default loader silently drops the alpha channel that the whole computation depends on.

import numpy as np
import cv2

def composite_over(fg_bgra, bg_bgr):
    """Porter-Duff 'over' for a BGRA foreground on an opaque BGR background."""
    fg = fg_bgra[..., :3].astype(np.float32)
    alpha = (fg_bgra[..., 3:4].astype(np.float32)) / 255.0   # (H, W, 1)
    bg = bg_bgr.astype(np.float32)
    out = alpha * fg + (1.0 - alpha) * bg       # broadcast over channels
    return np.clip(out, 0, 255).astype(np.uint8)

overlay = cv2.imread("score_banner.png", cv2.IMREAD_UNCHANGED)  # keeps alpha!
frame   = cv2.imread("match_frame.jpg")
print(overlay.shape)        # (H, W, 4): the 4th channel is alpha
shown = composite_over(overlay, frame)

from PIL import Image
out = Image.alpha_composite(bg_rgba, fg_rgba)    # both in RGBA mode

5. Masks and Bitwise Operations Intermediate

When transparency is all-or-nothing, the binary masks produced by the thresholding of Section 2.4 take the place of a fractional alpha channel, and compositing reduces to bitwise logic: AND to keep pixels where the mask is set, AND with the inverted mask to punch a hole, OR to merge the pieces. OpenCV's bitwise_* family accepts an 8-bit mask argument that gates the operation per pixel, giving the classic hard-edged logo overlay:

import cv2

frame = cv2.imread("match_frame.jpg")
logo  = cv2.imread("club_logo.png")            # opaque BGR logo on white

# Build a binary mask of the logo's own pixels (Otsu, inverted: logo is dark)
logo_gray = cv2.cvtColor(logo, cv2.COLOR_BGR2GRAY)
_, mask = cv2.threshold(logo_gray, 0, 255,
                        cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
mask_inv = cv2.bitwise_not(mask)

roi = frame[20:20 + logo.shape[0], 20:20 + logo.shape[1]]
hole  = cv2.bitwise_and(roi, roi, mask=mask_inv)   # background minus logo area
inked = cv2.bitwise_and(logo, logo, mask=mask)     # logo pixels only
frame[20:20 + logo.shape[0], 20:20 + logo.shape[1]] = cv2.add(hole, inked)

This bitwise idiom is the workhorse of region-of-interest processing throughout the rest of the book: masks select which pixels an operation may touch, whether that operation is a histogram (the mask argument of cv2.calcHist in Section 2.2), a color correction, or a generative edit. The masks themselves will get vastly smarter, produced by morphology in Chapter 6 and by segmentation networks later, but the compositing algebra they plug into is the one on this page. Two previews of where this leads: blending with a soft mask across a seam is done far better at multiple scales with the pyramids of Chapter 4, and the modern generative version of "put this object into that photo" appears in Chapter 35.