Write using your Eyes – Gaze controlled keyboard with Python and Opencv p.10

This is the final tutorial of the project “Gaze controlled keyboard with Python and Opencv”.

The code is approximately 300 lines. I added the missing parts to the code, I improved the graphic of the keyboard and I made a few adjustments. I’m not going to do any coding in this tutorial, instead I will show you the result of the final project and on the video I’ll do a quick review of the entire code.

How does this work?

Once we run the script we see see three different windows containing respectively: the keyboard, a white board and the capture from the webcam.

The keyboard is divided in two parts, and before pressing each letter we need to select the part.
The left side contains the letters: Q, W, E, R, T, A, S, D, F, G, Z, X, C, V.
The right side contains the letters: Y, U, I, O, P, H, J, K, L, V, B, N, M and spacebar.

1) Choose keyboard’s side:

The webcam is controlling the position of your eyes, so if you want to select the left side you need to look on the left for around a second, for the opposite side you need to look on the right.

Keyboard side

2) Press a key

Let’s supposed that we choosed the Right side.
Now we will see the keys contained on the right side.
To press a key we need to wait that the key lights up, and then we blink our eyes. We keep them closed until we hear a sound. That sound means that the key has been pressed.

3) Repeat step 1 and 2 to press a new key.

Each time we press a key, we need to repeat the step 1 and 2.

Each time the key is pressed, it is added to the white board. For example I typed the word “Hello” and below you see the result.

Final considerations:

This project was realised with the only purpose to teach you how to detect the eyes, theyr gaze and the blinking.

A few parts regarding the eye detection have been hardcoded to work specifically with my eyes and with a specific lightening condition and with a certain webcam.
This means that if some of these thress variables is different for you, the code might not work properly or might not work at all.

It’s of course possible to improve it for production environment.
If you’re the owner of a company and you want to develop this project further, feel free to contact me.