Understanding the Low NMI Issue in RGB and Thermal Image Registration

When registering multi-modal images, such as visible spectrum (RGB) and infrared/thermal (IR) imagery, Normalized Mutual Information (NMI) is frequently used to evaluate alignment quality. However, practitioners often encounter a puzzling phenomenon: despite an accurate physical alignment (e.g., landmark errors under 5 pixels), the NMI score reported by skimage.metrics.normalized_mutual_information remains near the theoretical minimum (~1.02 to 1.05, where 1.0 means no shared information and 2.0 means identical images).

Why Does NMI Fail on Raw RGB-Thermal Pairs?

The formula for NMI is defined as NMI = (H(A) + H(B)) / H(A, B), where H represents Shannon entropy. NMI drops close to 1.0 for two primary reasons:

  • Continuous Floating-Point Noise: Fine sensor noise creates unique continuous pixel values. Without explicit histogram binning, almost every pixel pair (A[i, j], B[i, j]) becomes unique in joint space. This skyrockets the joint entropy H(A, B) to nearly H(A) + H(B), driving NMI down toward 1.0.
  • Complex Cross-Modality Relationships: RGB images capture surface reflectance, whereas thermal cameras capture emitted thermal radiation. A cold object might appear bright white in visible light but dark in thermal imagery. Because NMI relies on consistent joint probability distributions between pixel intensities, complex or non-monotonic relationships dilute the metric.

Solution 1: Apply Explicit Histogram Binning

The most immediate fix when using scikit-image is controlling the joint histogram bin count via the bins parameter in normalized_mutual_information. Reducing the number of bins groups continuous floating-point values into discrete range buckets, eliminating high-frequency noise dispersion.

import numpy as np
from skimage.metrics import normalized_mutual_information as nmi_fn

# Calculate NMI with discrete binning (e.g., 32 or 64 bins)
score_binned = nmi_fn(img_a, img_b, bins=32)
print(f"Binned NMI: {score_binned:.4f}")

Solution 2: Compute NMI on Gradient and Edge Maps

Because pixel intensity correlations between thermal and RGB images are frequently non-linear, geometric structure (such as object boundaries) provides a significantly stronger correlation signal. Extracting gradient magnitudes prior to calculating NMI allows you to compare shared edges rather than raw intensities.

from skimage.filters import sobel
from skimage.metrics import normalized_mutual_information as nmi_fn

# Extract structural edge maps
grad_a = sobel(img_a)
grad_b = sobel(img_b)

# Compute NMI on edge maps with binning
edge_nmi = nmi_fn(grad_a, grad_b, bins=32)
print(f"Edge-based NMI: {edge_nmi:.4f}")

Solution 3: Use Specialized Multi-Modal Metrics

If intensity-based NMI remains insensitive even after binning, consider adopting structural metrics tailored specifically for multi-modal image alignment:

  • MIND (Modality Independent Neighborhood Descriptor): Calculates local self-similarity descriptors around each pixel, rendering it invariant to cross-modal intensity transformations.
  • Normalized Cross-Correlation (NCC) on Edge Features: Computing standard NCC on Sobel or Canny edge maps typically yields a sharp metric peak directly at the ideal registration point.

Summary

Near-minimum NMI scores (~1.03) on visually aligned RGB-IR images are expected when using default continuous settings due to floating-point sensor noise and complex modality differences. Specifying explicit histogram binning (such as bins=32) or evaluating NMI over edge-filtered gradient maps restores the metric's sensitivity to spatial alignment.