Object tracking using Homography – OpenCV 3.4 with python 3 Tutorial 34

We’re going to learn in this tutorial how to track an object using the Feature matching method, and then finding the Homography.

This detection method works only to track two identical objects, so for example if we want to find the cover of a book among many other books, if we want to compare two pictures.

I’m going to take the cover of a book from google and then I will try to detect the same book on my hand.

How do we match the images of the Book?

What approach are we going to use to detect the book that I’m holding on my hand, once we already have the image of the cover (the image above)?

First, we’re going to use the Feature matching approach, that I’ve already explained in this post.

We load the image of the book (queryimage), and then we load the camera.

import cv2
import numpy as np

img = cv2.imread("ultimo_sopravvissuto.jpg", cv2.IMREAD_GRAYSCALE)  # queryiamge
cap = cv2.VideoCapture(0)

We then load the SIFT algorythm (or another feature detection algorythm).
On line 8 we get the keypoints and descriptors of the Queryimage.
On line 12 we load the flann algorythm which we are going to use to find the matching features.

# Features
sift = cv2.xfeatures2d.SIFT_create()
kp_image, desc_image = sift.detectAndCompute(img, None)
# Feature matching
index_params = dict(algorithm=0, trees=5)
search_params = dict()
flann = cv2.FlannBasedMatcher(index_params, search_params)

Then we detect the features and descriptors of the frame (train image) from the webcam, and we compare them with the ones of the query image.

_, frame = cap.read()
grayframe = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # trainimage
kp_grayframe, desc_grayframe = sift.detectAndCompute(grayframe, None)
matches = flann.knnMatch(desc_image, desc_grayframe, k=2)
good_points = []
for m, n in matches:
    if m.distance < 0.6 * n.distance:
        good_points.append(m)

And we get this result:

How do we detect the Book and its Homography?

Once we have the matches between queryimage and trainimage, most of the work is done.
What we only need to do is to find its homography, so the object with its perspective. For example as you can notice in the video tutorial, when I move the book in different angle, its perspective changes.

To get the homography, we need first to obtain the matrix and we do it with the function findHomography.

query_pts = np.float32([kp_image[m.queryIdx].pt for m in good_points]).reshape(-1, 1, 2)
train_pts = np.float32([kp_grayframe[m.trainIdx].pt for m in good_points]).reshape(-1, 1, 2)
matrix, mask = cv2.findHomography(query_pts, train_pts, cv2.RANSAC, 5.0)
matches_mask = mask.ravel().tolist()

Finally we do the perspective transform using the points and the matrix.

# Perspective transform
h, w = img.shape
pts = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2)
dst = cv2.perspectiveTransform(pts, matrix)

And at the end we can show the result on the screen.

homography = cv2.polylines(frame, [np.int32(dst)], True, (255, 0, 0), 3)
cv2.imshow("Homography", homography)