Eye Gaze detection 2 – Gaze controlled keyboard with Python and Opencv p.4

We’re going to learn in this tutorial how to detect the gaze, and more specifically to detect if the eyes are looking to the left, the right or the center.

To detect the gaze we need first to understand how the eye appears when it’s looking in different directions.
I’ve discussed this into details in another tutorial, so if you want to know more about it, please check the tutorial about Eye motion tracking.

What at a first glance is really clear, by looking at the image above, it’s that the sclera (white part of the eye) fills the right part of the eye when the eye is looking at the left, the opposite happens when it’s looking to the right and when it’s looking to the center the white is well balanced between left and right.

Eye gaze ratio

Keeping in mind what i’ve explained before, we need now to focus on the detection of the white part.

The idea is to split the eye in two parts and to find out in which of the two parts there is more sclera visible.

If the sclera is more visible on the right part, so the eye is looking at the left (our left) like in this case.

Technically to detect the sclera we convert the eye into grayscale, we find a treshold and we count the white pixels.

gray_eye = eye[min_y: max_y, min_x: max_x]
_, threshold_eye = cv2.threshold(gray_eye, 70, 255, cv2.THRESH_BINARY)
height, width = threshold_eye.shape
left_side_threshold = threshold_eye[0: height, 0: int(width / 2)]
left_side_white = cv2.countNonZero(left_side_threshold)

right_side_threshold = threshold_eye[0: height, int(width / 2): width]
right_side_white = cv2.countNonZero(right_side_threshold)

We divide the white pixels of the left part and those of the right part and we get the gaze ratio.

gaze_ratio = left_side_white / right_side_white

Detecting the gaze

The gaze ratio tells use where a specific eye is looking.
Normally both the eyes look in the same direction, so if we correctly detect the gaze of a single eye, we detect the gaze of both eyes.

Only if we want to be more precise we could detect the gaze of both the eyes and use both values to detect the gaze ratio.

# Gaze detection
gaze_ratio_left_eye = get_gaze_ratio([36, 37, 38, 39, 40, 41], landmarks)
gaze_ratio_right_eye = get_gaze_ratio([42, 43, 44, 45, 46, 47], landmarks)
gaze_ratio = (gaze_ratio_right_eye + gaze_ratio_left_eye) / 2

And once we have the gaze ratio we can display on the screen the result.
I found out that the if the gaze ratio is smaller than 1 when looking to the right side and greater than 1.7 when the eyes are looking to the left side.

if gaze_ratio <= 1:
	cv2.putText(frame, "RIGHT", (50, 100), font, 2, (0, 0, 255), 3)
	new_frame[:] = (0, 0, 255)
elif 1 < gaze_ratio < 1.7:
	cv2.putText(frame, "CENTER", (50, 100), font, 2, (0, 0, 255), 3)
else:
	new_frame[:] = (255, 0, 0)
	cv2.putText(frame, "LEFT", (50, 100), font, 2, (0, 0, 255), 3)