Facial Landmarks Detection | with Opencv, Mediapipe and Python
In this tutorial, we will see how to find 468 facial landmarks easily using a library called mediapipe , extract the X and Y coordinates so we can use them as we like. We will see how to do it first on an image and then on the videos.
There are a lot of applications for this type of function. Here are some examples on the site:
- Face swapping (explained in 8 steps) – Opencv with Python
- Pig’s nose (Instagram face filter) – Opencv with Python
- Press a key by blinking eyes – Gaze controlled keyboard with Python and Opencv p.8
- Face detection using Haar Cascades – OpenCV 3.4 with python 3 Tutorial 37
I made this tutorial to make using the library as easy as possible. For those wishing to see the official documentation on this part, you can look at this link: Face mesh .
Mediapipe installation for facial landmarks detection
The installation is very simple, first we need to install opencv with this command:
pip install opencv-python
and then the mediapipe library:
pip install mediapipe
Facial landmarks whit python on a image
If the installation was successful we are ready to recall the libraries and load the image from our folder.
import cv2 import mediapipe as mp image = cv2.imread("person.jpg")
After that we need to load face mesh and create an object for that.
# Face Mesh mp_face_mesh = mp.solutions.face_mesh face_mesh = mp_face_mesh.FaceMesh()
We must see the result but first if a fundamental step: convert the color format. Opencv uses BGR instead of RBG.
rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # Facial landmarks result = face_mesh.process(rgb_image)
Now we need to get the image information: height, width, which will serve as a reference to indicate the points. We have to use a for click for the extraction and apply the extracted points by drawing a circle.
height, width, _ = image.shape for facial_landmarks in result.multi_face_landmarks: for i in range(0, 468): pt1 = facial_landmarks.landmark[i] x = int(pt1.x * width) y = int(pt1.y * height) cv2.circle(image, (x, y), 5, (100, 100, 0), -1)
Finally we can show the image with the opencv cv2.imshow() command.
This is the result
Facial landmarks whit python on a video
This part is very simple because we know that with OpenCV, the video is a sequence of frames, so we just change a few things to the code to make everything work correctly.
Obviously, we have to recall the video instead of the image.
cap = cv2.VideoCapture("video.mp4")
and all the rest of the code must be put in a while loop
while True: # Image ret, image = cap.read() if ret is not True: break height, width, _ = image.shape ... cv2.imshow("Image", image) cv2.waitKey(1)
The result is more appreciable in the video, but this image gives the idea.