Library Landscape: OpenCV, scikit-image, Pillow & SciPy ndimage

"OpenCV insists I am BGR, Pillow swears I am RGB, and scikit-image quietly converted me to float64 while nobody was looking. I just lie here, contiguous in memory, and let them argue."

A Diplomatically Neutral NumPy Array

Throughout Part I we reached for whichever library had the cleanest API for the moment: Pillow to load files in Chapter 0, OpenCV for warps in Chapter 5, scikit-image for denoisers in Chapter 7. This section finally puts the four side by side: where each came from, what data model it assumes, how to translate between them safely, and a decision procedure for choosing. Treat it as the reference you return to whenever an import decision feels arbitrary.

1. Four Libraries, Four Philosophies Beginner

OpenCV (cv2) is a C++ library wearing a Python costume. Born at Intel in 1999, it grew into the largest open-source vision codebase in existence: thousands of functions spanning image processing, video I/O, calibration, tracking, and legacy machine learning. Its Python bindings hand you NumPy arrays, but the semantics underneath are C++: preallocated outputs, in-place options, integer-first dtypes, and the famous BGR channel order it inherited from early Windows bitmap conventions. When raw speed on a CPU matters, OpenCV is usually the answer; Section 8.2 quantifies why.

scikit-image (skimage) is the scientific community's counterproposal: pure-Python-plus-Cython, NumPy-native, with an API designed to read like the textbook. Functions are named after the algorithm and its author (filters.frangi, restoration.richardson_lucy), default to float images in $[0, 1]$, and favor correctness and documentation over raw throughput. Its docs double as a small image processing course, and many functions cite the paper they implement.

Pillow (PIL) is the elder of the family, the maintained fork of the 1995 Python Imaging Library. Its center of gravity is file handling: it reads and writes dozens of formats, understands EXIF orientation, color profiles, animated GIFs, and 16-bit TIFFs, and exposes a friendly Image object with methods like resize and crop. It is the default tool of web backends and the loader underneath torchvision, which is why it reappears in Chapter 18.

SciPy ndimage (scipy.ndimage) is the quiet generalist. It implements filters, morphology, interpolation, and measurements for arrays of any dimensionality, which makes it the only choice in this lineup when your "image" is a 3D microscopy stack or a 4D fMRI volume. Its API is spartan, but several scikit-image functions delegate to it internally, and its map_coordinates is the engine behind many custom warps. Figure 8.1.1 places all four on the common NumPy foundation.

Table 8.1.1 compresses the comparison into the form you will actually use: a row to scan before you type import.

2. The Data-Model Trap: dtype, Range & Channel Order Intermediate

The libraries agree on the container and disagree on the contents. As established in Chapter 1, an image is an array plus an interpretation, and each library bakes in a different interpretation. Three axes of disagreement cause essentially all of the trouble:

• Channel order. OpenCV stores color as BGR; everyone else uses RGB. Display an OpenCV-loaded photo with Matplotlib and faces turn avatar-blue.
• Dtype and range. OpenCV and Pillow think in uint8 with values 0 to 255; scikit-image converts to float64 in $[0, 1]$ at the first opportunity. Pass a float image where uint8 is expected and thresholds, gamma curves, and saturation arithmetic silently misbehave.
• Coordinate conventions. NumPy indexes [row, col]; every OpenCV geometric API speaks $(x, y)$ points. Chapter 5 already scarred us with this one.

The defensive pattern is a small, boring conversion layer at the borders of your code, written once and tested once. Code 8.1.1 is a version you can paste into any project.

import numpy as np
import cv2
from PIL import Image

def pil_to_cv(im: Image.Image) -> np.ndarray:
    """Pillow (RGB Image) -> OpenCV (BGR uint8 ndarray)."""
    return cv2.cvtColor(np.asarray(im.convert("RGB")), cv2.COLOR_RGB2BGR)

def cv_to_pil(arr: np.ndarray) -> Image.Image:
    """OpenCV (BGR uint8) -> Pillow (RGB Image)."""
    return Image.fromarray(cv2.cvtColor(arr, cv2.COLOR_BGR2RGB))

def cv_to_skimage(arr: np.ndarray) -> np.ndarray:
    """OpenCV (BGR uint8) -> scikit-image convention (RGB float64 in [0, 1])."""
    rgb = cv2.cvtColor(arr, cv2.COLOR_BGR2RGB)
    return rgb.astype(np.float64) / 255.0

def skimage_to_cv(arr: np.ndarray) -> np.ndarray:
    """scikit-image (RGB float in [0, 1]) -> OpenCV (BGR uint8), with safe rounding."""
    u8 = np.clip(np.rint(arr * 255.0), 0, 255).astype(np.uint8)
    return cv2.cvtColor(u8, cv2.COLOR_RGB2BGR)

from skimage import util

f = util.img_as_float(u8_image)    # uint8 [0,255]  -> float64 [0,1]
u = util.img_as_ubyte(float_img)   # float [0,1]    -> uint8, clipped + rounded
w = util.img_as_uint(float_img)    # float [0,1]    -> uint16 [0, 65535]

3. A Rosetta Stone: One Operation, Four APIs Intermediate

Seeing the same operation written four ways teaches more about the libraries' personalities than any prose. Code 8.1.2 applies a Gaussian blur with $\sigma = 3$ (the workhorse smoother from Chapter 3) in each library, then measures how much the results differ.

import numpy as np, cv2
from PIL import Image, ImageFilter
from scipy import ndimage
from skimage import filters, util

rng = np.random.default_rng(8)
img = rng.integers(0, 256, size=(512, 512), dtype=np.uint8)  # synthetic test card

# OpenCV: ksize=(0,0) lets sigma determine the kernel size; uint8 in, uint8 out
b_cv = cv2.GaussianBlur(img, ksize=(0, 0), sigmaX=3.0)

# SciPy ndimage: dtype-agnostic, n-dimensional; we feed floats for precision
b_nd = ndimage.gaussian_filter(img.astype(np.float64), sigma=3.0)

# scikit-image: float-in-[0,1] convention, channel_axis for color images
b_sk = filters.gaussian(util.img_as_float(img), sigma=3.0) * 255.0

# Pillow: its own Image object; radius plays the role of sigma (approximately)
b_pil = np.asarray(Image.fromarray(img).filter(ImageFilter.GaussianBlur(radius=3.0)),
                   dtype=np.float64)

print("max |OpenCV - ndimage|     :", np.abs(b_cv - b_nd).max().round(2))
print("max |ndimage - skimage|    :", np.abs(b_nd - b_sk).max().round(2))
print("max |ndimage - Pillow|     :", np.abs(b_nd - b_pil).max().round(2))

max |OpenCV - ndimage|     : 1.42
max |ndimage - skimage|    : 0.0
max |ndimage - Pillow|     : 3.87

The output is the lesson: four correct implementations of "the same" blur disagree by up to a few gray levels, because kernel truncation radius, border policy (the Chapter 3 border modes again), and internal precision all differ. For most applications this is irrelevant; for reproducing a paper's numbers it is decisive, a theme Section 8.3 revisits with SSIM. Table 8.1.2 extends the Rosetta stone to the operations Part I used most.

Two patterns jump out of the table. First, OpenCV and scikit-image cover nearly everything from Chapter 2 through Chapter 6, so most projects standardize on one of them and call the others at the edges. Second, the gaps are informative: Pillow has no morphology or components because it was never meant for analysis, and ndimage has no Otsu or file I/O because thresholding policy and disk formats are not n-dimensional math.

4. Choosing: A Decision Guide Intermediate

The decision procedure this book uses, distilled from the trade-offs above:

1. Loading and saving files, EXIF, web formats? Pillow first. It is the I/O specialist, and np.asarray(im) hands the pixels to everyone else.
2. Real-time or video pipeline, CPU-bound? OpenCV. Nothing else in the lineup touches its throughput (Section 8.2 measures it), and cv2.VideoCapture has no peer here.
3. Research code, papers to reproduce, readability reviewed? scikit-image. The float convention removes a whole class of overflow bugs, and the docstrings cite their sources.
4. Volumes, stacks, anything not (H, W, 3)? SciPy ndimage, which never assumed two dimensions in the first place.
5. The operation will eventually live inside a neural network? Start thinking in PyTorch/Kornia tensors now; Chapter 18 makes the move official.

Mixing is normal; pick one primary convention per project (most teams choose either "OpenCV everywhere" or "skimage everywhere, OpenCV for hotspots"), write the border layer once, and never convert ad hoc in the middle of an algorithm.