Eye Gaze detection 2 – Gaze controlled keyboard with Python and Opencv p.4
We’re going to learn in this tutorial how to detect the gaze, and more specifically to detect if the eyes are looking to the left, the right or the center.
To detect the gaze we need first to understand how the eye appears when it’s looking in different directions.
I’ve discussed this into details in another tutorial, so if you want to know more about it, please check the tutorial about Eye motion tracking.

What at a first glance is really clear, by looking at the image above, it’s that the sclera (white part of the eye) fills the right part of the eye when the eye is looking at the left, the opposite happens when it’s looking to the right and when it’s looking to the center the white is well balanced between left and right.
Eye gaze ratio
Keeping in mind what i’ve explained before, we need now to focus on the detection of the white part.
The idea is to split the eye in two parts and to find out in which of the two parts there is more sclera visible.

If the sclera is more visible on the right part, so the eye is looking at the left (our left) like in this case.
Technically to detect the sclera we convert the eye into grayscale, we find a treshold and we count the white pixels.
gray_eye = eye[min_y: max_y, min_x: max_x] _, threshold_eye = cv2.threshold(gray_eye, 70, 255, cv2.THRESH_BINARY) height, width = threshold_eye.shape left_side_threshold = threshold_eye[0: height, 0: int(width / 2)] left_side_white = cv2.countNonZero(left_side_threshold) right_side_threshold = threshold_eye[0: height, int(width / 2): width] right_side_white = cv2.countNonZero(right_side_threshold)
We divide the white pixels of the left part and those of the right part and we get the gaze ratio.
gaze_ratio = left_side_white / right_side_white
Detecting the gaze
The gaze ratio tells use where a specific eye is looking.
Normally both the eyes look in the same direction, so if we correctly detect the gaze of a single eye, we detect the gaze of both eyes.
Only if we want to be more precise we could detect the gaze of both the eyes and use both values to detect the gaze ratio.
# Gaze detection gaze_ratio_left_eye = get_gaze_ratio([36, 37, 38, 39, 40, 41], landmarks) gaze_ratio_right_eye = get_gaze_ratio([42, 43, 44, 45, 46, 47], landmarks) gaze_ratio = (gaze_ratio_right_eye + gaze_ratio_left_eye) / 2
And once we have the gaze ratio we can display on the screen the result.
I found out that the if the gaze ratio is smaller than 1 when looking to the right side and greater than 1.7 when the eyes are looking to the left side.
if gaze_ratio <= 1: cv2.putText(frame, "RIGHT", (50, 100), font, 2, (0, 0, 255), 3) new_frame[:] = (0, 0, 255) elif 1 < gaze_ratio < 1.7: cv2.putText(frame, "CENTER", (50, 100), font, 2, (0, 0, 255), 3) else: new_frame[:] = (255, 0, 0) cv2.putText(frame, "LEFT", (50, 100), font, 2, (0, 0, 255), 3)
Related posts:
6 Comments
Leave a Reply Cancel reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Gaze controlled keyboard
Most Read:
-
Train YOLO to detect a custom object (online with free GPU)
-
YOLO object detection using Opencv with Python
-
Detecting colors (Hsv Color Space) – Opencv with Python
-
How to install Python 3 and Opencv 4 on Windows
-
How to install Dlib for Python 3 on Windows
-
Check if two images are equal with Opencv and Python
-
Feature detection (SIFT, SURF, ORB) – OpenCV 3.4 with python 3 Tutorial 25
-
Eye motion tracking – Opencv with Python
Hello Sergio!
I follow your webite since a long a time, I started a master in computer vision and robotics, and this is like heaven for studies for me!
This specific eye blinking and gaze detection got my attention because of the many applications.
I was wondering, that can it be used to detect if the person is ‘probabily’ sleeping? Like… if the blinking lasts for more than a certain period of time it would show that it is sleeping!
If this can be one, can you help me with a source of research?
Hi Fabio,
It will be possible the approach is really similar to the one used in this tutorial with the only difference that you need to change the time, so instead of blinking it will be mostly to detect if the eyes are closed at all and not just blinking.
I suggest to use an infrared camera so that you can do the detection even at night when the light is off.
Hi what method can i use to detect if the eye is looking UP or DOWN
Hi sir what method can i use to detect face and eyes and protect to recognize from picture
Reply
what if i want to check when the eye is looking up or looking down?
I tried to run this code on anaconda and i received this error:
gaze_ratio = (gaze_ratio_right_eye + gaze_ratio_left_eye) / 2
TypeError: unsupported operand type(s) for +: ‘NoneType’ and ‘NoneType’