AI Vision Academy

Access the code of this tutorial, computer vision courses and an exclusive community on AI Vision Academy

• Access to over 50+ source codes from Pysource.com/blog
• Dedicated video courses about computer vision
• Access to an exclusive community of professionals
• Real-World AI Projects – Get hands-on experience building practical AI Computer Vision solutions with a structured path.
• Monthly Coaching Calls – get support and any of your questions answered

Subscribe to our newsletter to learn more

❌

In this tutorial, we will see how to find 468 facial landmarks easily using a library called mediapipe , extract the X and Y coordinates so we can use them as we like. We will see how to do it first on an image and then on the videos.

There are a lot of applications for this type of function. Here are some examples on the site:

• Face swapping (explained in 8 steps) – Opencv with Python
• Pig’s nose (Instagram face filter) – Opencv with Python
• Press a key by blinking eyes – Gaze controlled keyboard with Python and Opencv p.8
• Face detection using Haar Cascades – OpenCV 3.4 with python 3 Tutorial 37

I made this tutorial to make using the library as easy as possible. For those wishing to see the official documentation on this part, you can look at this link: Face mesh .

Mediapipe installation for facial landmarks detection

The installation is very simple, first we need to install opencv with this command:

pip install opencv-python

and then the mediapipe library:

pip install mediapipe

Facial landmarks whit python on a image

If the installation was successful we are ready to recall the libraries and load the image from our folder.

import cv2
import mediapipe as mp

image = cv2.imread("person.jpg")

After that we need to load face mesh and create an object for that.

# Face Mesh
mp_face_mesh = mp.solutions.face_mesh
face_mesh = mp_face_mesh.FaceMesh()

We must see the result but first if a fundamental step: convert the color format. Opencv uses BGR instead of RBG.

rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Facial landmarks
result = face_mesh.process(rgb_image)

Now we need to get the image information: height, width, which will serve as a reference to indicate the points. We have to use a for click for the extraction and apply the extracted points by drawing a circle.

height, width, _ = image.shape

for facial_landmarks in result.multi_face_landmarks:
    for i in range(0, 468):
        pt1 = facial_landmarks.landmark[i]
        x = int(pt1.x * width)
        y = int(pt1.y * height)

        cv2.circle(image, (x, y), 5, (100, 100, 0), -1)

Finally we can show the image with the opencv cv2.imshow() command.

cv2.imshow("Image", image)

This is the result

Facial landmarks whit python on a video

This part is very simple because we know that with OpenCV, the video is a sequence of frames, so we just change a few things to the code to make everything work correctly.

Obviously, we have to recall the video instead of the image.

cap = cv2.VideoCapture("video.mp4")

and all the rest of the code must be put in a while loop

while True:
    # Image
    ret, image = cap.read()
    if ret is not True:
        break
    height, width, _ = image.shape

    ...

    cv2.imshow("Image", image)
    cv2.waitKey(1)

The result is more appreciable in the video, but this image gives the idea.