Optical Flow with Lucas-Kanade method – OpenCV 3.4 with python 3 Tutorial 31
How work Optical Flow?
Optical flow is the vector who describe the moviment of the object betweent two consecutive frames.
If I want to track an object what can I do? There is an algorithm called Lucas-Kanade, let’s take this toothbrush as an example, I indicate a point that I want to trace in a specific area. The algorithm checks the object in the previous area and if it does not find it it assumes that it has moved and then selects a new point.
For more details on how the function works, I suggest you read the official OpenCv guide: Optical Flow with Lucas-Kanade method
The Lucas-Kanade algorithm needs some conditions to work. For example, the object must move really close, no matter if it is fast because the algorithm analyzes frame by frame.
Optical flow with Lucas-Kanade example code
The first step is to call the usual OpenCV and Numpy in our code, then we have to write the code to retrieve the data frame by frame from the camera. I recommend you also read my article on the subject: Control webcam with servo motor and raspberry pi – Opencv with Python which could help you with your project.
Select the point
The first necessary step is to select a point. To do this you need a mouse callback function
# Mouse function def select_point(event, x, y, flags, params): global point, point_selected, old_points if event == cv2.EVENT_LBUTTONDOWN: point = (x, y) point_selected = True old_points = np.array([[x, y]], dtype=np.float32) cv2.namedWindow("Frame") cv2.setMouseCallback("Frame", select_point) point_selected = False point = () old_points = np.array([])
We need to initialize the arrays and remember that to be used with OpenCV they must be converted to nparray.
Now we can use the calcOpticalFlowPyrLK () tracking function. The first thing to do is to convert the frame to grayscale format.
_, frame = cap.read() gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
We therefore consider the parameters to be used in the optical-flow function:
- old_gray : old frame array
- gray_frame : current frame
- old_point : array with old point
- None : empy param
- **lk_params : some configuration about the function
# Lucas kanade params lk_params = dict(winSize = (15, 15), maxLevel = 4, criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03)) ... new_points, status, error = cv2.calcOpticalFlowPyrLK(old_gray, gray_frame, old_points, None, **lk_params) old_gray = gray_frame.copy() old_points = new_points x, y = new_points.ravel()
I have tried to make this as simple as possible and surely you can take more points as a reference and everything can be improved.