Skip to content

Beginners Opencv · Tutorials

Object tracking using Homography – OpenCV 3.4 with python 3 Tutorial 34

June 5, 2018 Sergio Canu 3 minute read
Access community, courses and source codes
Logo

AI Vision Academy

Access the code of this tutorial, computer vision courses and an exclusive community on AI Vision Academy

  • Access to over 50+ source codes from Pysource.com/blog
  • Dedicated video courses about computer vision
  • Access to an exclusive community of professionals
  • Real-World AI Projects – Get hands-on experience building practical AI Computer Vision solutions with a structured path.
  • Monthly Coaching Calls – get support and any of your questions answered

Subscribe to our newsletter to learn more

We’re going to learn in this tutorial how to track an object using the Feature matching method, and then finding the Homography.

This detection method works only to track two identical objects, so for example if we want to find the cover of a book among many other books, if we want to compare two pictures.

I’m going to take the cover of a book from google and then I will try to detect the same book on my hand.

Book Cover

How do we match the images of the Book?

What approach are we going to use to detect the book that I’m holding on my hand, once we already have the image of the cover (the image above)?

Book detection

First, we’re going to use the Feature matching approach, that I’ve already explained in this post.

We load the image of the book (queryimage), and then we load the camera.

import cv2
import numpy as np

img = cv2.imread("ultimo_sopravvissuto.jpg", cv2.IMREAD_GRAYSCALE)  # queryiamge
cap = cv2.VideoCapture(0)

We then load the SIFT algorythm (or another feature detection algorythm).
On line 8 we get the keypoints and descriptors of the Queryimage.
On line 12 we load the flann algorythm which we are going to use to find the matching features.

# Features
sift = cv2.xfeatures2d.SIFT_create()
kp_image, desc_image = sift.detectAndCompute(img, None)
# Feature matching
index_params = dict(algorithm=0, trees=5)
search_params = dict()
flann = cv2.FlannBasedMatcher(index_params, search_params)

Then we detect the features and descriptors of the frame (train image) from the webcam, and we compare them with the ones of the query image.

_, frame = cap.read()
grayframe = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # trainimage
kp_grayframe, desc_grayframe = sift.detectAndCompute(grayframe, None)
matches = flann.knnMatch(desc_image, desc_grayframe, k=2)
good_points = []
for m, n in matches:
    if m.distance < 0.6 * n.distance:
        good_points.append(m)

And we get this result:

Feature matching

How do we detect the Book and its Homography?

Once we have the matches between queryimage and trainimage, most of the work is done.
What we only need to do is to find its homography, so the object with its perspective. For example as you can notice in the video tutorial, when I move the book in different angle, its perspective changes.

To get the homography, we need first to obtain the matrix and we do it with the function findHomography.

query_pts = np.float32([kp_image[m.queryIdx].pt for m in good_points]).reshape(-1, 1, 2)
train_pts = np.float32([kp_grayframe[m.trainIdx].pt for m in good_points]).reshape(-1, 1, 2)
matrix, mask = cv2.findHomography(query_pts, train_pts, cv2.RANSAC, 5.0)
matches_mask = mask.ravel().tolist()

Finally we do the perspective transform using the points and the matrix.

# Perspective transform
h, w = img.shape
pts = np.float32([[0, 0], [0, h], [w, h], [w, 0]]).reshape(-1, 1, 2)
dst = cv2.perspectiveTransform(pts, matrix)

And at the end we can show the result on the screen.

homography = cv2.polylines(frame, [np.int32(dst)], True, (255, 0, 0), 3)
cv2.imshow("Homography", homography)
Homography

16 comments

    1. Hi Sergio Canu,
      Is it work on vehicles detection on roads????
      I mean that if you trying apply this method with cars detection on roads?
      Plz show your opinion???
      Thanks.

  2. Hello~ Sir,
    I cannot find “cv2.drawKeypoints()”.
    I has install opencv-python_3.4.4.19 in PyCharm Community 2018.3
    What should i do for this problem,
    Thanks for your reply.

    BTW,
    Thanks a lot for your tutorial

  3. I tried copying and pasting this onto my python but i keep getting this error.. Please Help! :

    File “C:/Users/Arsla/.spyder-py3/ttest.py”, line 14, in
    kp_image, desc_image = sift.detectAndCompute(img, None)

    error: OpenCV(3.4.1) C:\Miniconda3\conda-bld\opencv-suite_1533128839831\work\opencv_contrib-3.4.1\modules\xfeatures2d\src\sift.cpp:1121: error: (-5) image is empty or has incorrect depth (!=CV_8U) in function cv::xfeatures2d::SIFT_Impl::detectAndCompute

    1. Wait i changed the picture to my own but now i get this error any luck?? :

      File “C:/Users/Arsla/.spyder-py3/ttest.py”, line 21, in
      grayframe = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # trainimage

      error: OpenCV(3.4.1) C:\Miniconda3\conda-bld\opencv-suite_1533128839831\work\modules\imgproc\src\color.cpp:11147: error: (-215) scn == 3 || scn == 4 in function cv::cvtColor

    2. Hi, I noticed that with some images the SIFT algorythm has some problem.

      You can solve it loading the image using an external library. Instead of loading the image using cv2.imread(“image.png”) just install the library Pillow

      pip install Pillow

      and then add this code:

      from PIL import Image

file = Image.open(infilename)
file.load()
img = np.asarray(file, dtype="int32")

  4. I am getting error:-

    error Traceback (most recent call last)
    in
    3 grayframe = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # trainimage
    4 kp_grayframe, desc_grayframe = sift.detectAndCompute(grayframe, None)
    —-> 5 matches = flann.knnMatch(desc_image, desc_grayframe, k=2)
    6 good_points = []
    7 for m, n in matches:

    error: OpenCV(3.4.1) C:\Miniconda3\conda-bld\opencv-suite_1533128839831\work\modules\flann\src\miniflann.cpp:315: error: (-210) type=0 in function cv::flann::buildIndex_

    1. I also have same error
      Tried this :
      -> matches = flann.knnMatch(np.asarray(desc_image, np.float32), np.asarray(desc_gray_frame, np.float32), 2)
      After this I got a new error of knnMatch:
      cv2.error: OpenCV(3.4.2) C:\projects\opencv-python\opencv\modules\flann\src\miniflann.cpp:487: error: (-215:Assertion failed) (size_t)knn size() in function ‘cv::flann::runKnnSearch_’

  6. I am getting error:-
    ift = cv2.xfeatures2d.SIFT_create()
    AttributeError: ‘module’ object has no attribute ‘xfeatures2d’

    BTW,
    Thanks a lot for your tutorial

Join the discussion