OpenCV Image Stitching

Image stitching is an important application in computer vision. It can stitch multiple images with overlapping regions into a larger image.

Common application scenarios include panorama generation, satellite image stitching, etc.

OpenCV is a powerful computer vision library that provides rich tools for image stitching.

This article will detail how to use OpenCV for image stitching, focusing on feature point detection and matching techniques.

Application Scenarios

  • Panorama generation: stitch multiple images into a panorama.
  • Map stitching: stitch multiple map images into a larger map.
  • Medical image processing: stitch multiple medical images into a complete image.

Basic Process of Image Stitching

The basic process of image stitching can be divided into the following steps:

  1. Image Reading: Read the images to be stitched.
  2. Feature Point Detection: Detect keypoints (feature points) in each image.
  3. Feature Point Matching: Match these feature points between different images.
  4. Compute Transformation Matrix: Compute the transformation matrix between images based on the matched feature points.
  5. Image Blending: Stitch the images according to the transformation matrix, and perform blending to eliminate stitching artifacts.

Next, we will explain the implementation of each step in detail.

1. Image Reading

First, we need to read the images to be stitched. OpenCV provides thecv2.imread()function to read images.

Example

import cv2

# Read images
image1 = cv2.imread('image1.jpg')
image2 = cv2.imread('image2.jpg')

# Check whether the images were read successfully
if image1 is None or image2 is None:
    print("Error: Unable to read image")
    exit()

2. Feature Point Detection

Feature point detection is a key step in image stitching. OpenCV provides a variety of feature point detection algorithms, such as SIFT, SURF, ORB, etc. Here we use SIFT as an example.

Example

# Create SIFT detector
sift = cv2.SIFT_create()

# Detect feature points and descriptors
keypoints1, descriptors1 = sift.detectAndCompute(image1, None)
keypoints2, descriptors2 = sift.detectAndCompute(image2, None)

detectAndCompute()The function returns two values: keypoints and descriptors. Keypoints are salient points in the image, and descriptors are descriptions of these keypoints, used for subsequent matching.

3. Feature Point Matching

After detecting feature points, we need to match these feature points between different images. OpenCV provides theBFMatcherorFlannBasedMatcherto perform feature point matching.

Example

# Create BFMatcher object
bf = cv2.BFMatcher()

# Use KNN matching
matches = bf.knnMatch(descriptors1, descriptors2, k=2)

# Apply ratio test to filter out good matches
good_matches = []
for m, n in matches:
    if m.distance < 0.75 * n.distance:
        good_matches.append(m)

knnMatch()The function returns the two best matches for each feature point. We use the ratio test (Lowe's ratio test) to filter out good matching points.

4. Compute Transformation Matrix

After obtaining good matching points, we can use these points to compute the transformation matrix between images. A common transformation matrix is the homography matrix, which can map points from one image to another.

Example

# Extract coordinates of matching points
src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches]).reshape(-1, 1, 2)
dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches]).reshape(-1, 1, 2)

# Compute homography matrix
H, _ = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)

findHomography()The function returns a 3x3 homography matrixH, which can mapimage1points in ... toimage2...

5. Image Blending

Finally, we use the computed homography matrix to stitch the images, and perform blending to eliminate stitching artifacts.

Example

# Get image dimensions
h1, w1 = image1.shape[:2]
h2, w2 = image2.shape[:2]

# Compute the size of the stitched image
pts = np.float32([[0, 0], [0, h1], [w1, h1], [w1, 0]]).reshape(-1, 1, 2)
dst = cv2.perspectiveTransform(pts, H)
[x_min, y_min] = np.int32(dst.min(axis=0).ravel() - 0.5)
[x_max, y_max] = np.int32(dst.max(axis=0).ravel() + 0.5)

# Compute the translation matrix
translation_matrix = np.array([[1, 0, -x_min], [0, 1, -y_min], [0, 0, 1]])

# Apply the translation matrix for image stitching
result = cv2.warpPerspective(image1, translation_matrix.dot(H), (x_max - x_min, y_max - y_min))
result[-y_min:h2 - y_min, -x_min:w2 - x_min] = image2

# Display the stitching result
cv2.imshow('Result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

warpPerspective()The function, based on the homography matrix,HPairimage1performs a perspective transformation and combines it withimage2to stitch.

Application Implementation

The following is the complete code for image stitching using feature point detection and matching:

Example

import cv2
import numpy as np

# 1. Load images
image1 = cv2.imread("path/to/image1.jpg")
image2 = cv2.imread("path/to/image2.jpg")

# 2. Convert to grayscale
gray1 = cv2.cvtColor(image1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(image2, cv2.COLOR_BGR2GRAY)

# 3. Feature point detection
sift = cv2.SIFT_create()
keypoints1, descriptors1 = sift.detectAndCompute(gray1, None)
keypoints2, descriptors2 = sift.detectAndCompute(gray2, None)

# 4. Feature point matching
matcher = cv2.BFMatcher()
matches = matcher.knnMatch(descriptors1, descriptors2, k=2)

# 5. Filter matching points
good_matches = []
for m, n in matches:
    if m.distance < 0.75 * n.distance:
        good_matches.append(m)

# 6. Compute homography matrix
if len(good_matches) > 10:
    src_pts = np.float32([keypoints1[m.queryIdx].pt for m in good_matches]).reshape(-1, 1, 2)
    dst_pts = np.float32([keypoints2[m.trainIdx].pt for m in good_matches]).reshape(-1, 1, 2)
    H, _ = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
else:
    print("Not enough matches found.")
    exit()

# 7. Image transformation
height1, width1 = image1.shape[:2]
height2, width2 = image2.shape[:2]
warped_image = cv2.warpPerspective(image1, H, (width1 + width2, height1))

# 8. Image stitching
warped_image[0:height2, 0:width2] = image2

# 9. Display result
cv2.imshow("Stitched Image", warped_image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Other Extensions