I played checkers vs a Robot | AI and Computer Vision

In this video, I will build artificial intelligence to play checkers with computer vision and a robotic arm.

The purpose of this project is to show you how to integrate computer vision into the real world and make it interact with a robotic arm. In this way, you can verify and evaluate, in perspective, the potential of computer vision in real applications.

I will tell you how I built the project, what hardware and strategies were used, and also the errors and problems that occurred during its development.

Hardware for computer vision checkers

The hardware I used for the development of the computer vision checkers project is quite basic: obviously the checkers game, a USB video camera, and a robotic arm for prototypes.

By combining computer vision, artificial intelligence, and hardware, the computer vision checkers project can be created as in the diagram of the image below.

Following the diagram, I have prepared everything as in the image below in a real scenario.

Checkers and artificial intelligence software

I looked for a checkers game that was written in python, with artificial intelligence, and without graphics in order to integrate it easily with my project. On GitHub, I found the code from Dimitrije Karanfilović, checkers.

As you can see from the image, the operation is very simple, just enter the number of the column and row of the pawn position and then indicate the new position. Then the artificial intelligence will make its move in less than 5 seconds.

Checkers computer vision part

The only goal of the computer vision is to identify the pieces on the board and their location, so I placed a camera over the board and processed the images in real-time. The first thing to do is to have a good overhead view of the game. I did it with a simple USB camera.

I tried a couple of solutions to identify the checkers and their position.

1. Computer vision checkers identify the pieces by colors [didn’t work]

I tried to identify the pieces with color recognition but it wasn’t stable. By changing the light or the position the checker detection was missed quite often, so it wasn’t reliable at all.

As you can see from the image below something was not properly identified.

The color detection method works very well when on a light controlled environment (which I didn’t have in this specific case). If you want to know more about how to recognize colors you can take a look at the article Simple Color recognition with Opencv and Python

For these reasons I immediately discarded this method and switched to object detection with deep learning.

2. Identify the pieces with AI [working method]

As you can see from the image below everything is correctly identified. Each checker is surrounded by a bounding box that identify its location.

I used object detection with deep learning. It takes longer to build an object detector with deep learning because you have to teach the AI what checkers are and where they are.

The object detection method I used to build this project is taught inside the course Object Detection with OpenCV and Deep Learning.

Robot arm integration with computer vision checkers

Using Dimitrije’s Checkers project we define the position of the checker by putting the row number and the column number. An example in the image below is circled in red.

I then convert the positions of the robotic arm from x and y to row and column based on the physical chessboard. As you can see in the image below if I write 4,3 the arm moves to the desired position. Note that the count starts at 0.

Checkers AI VS Person

Everything is ready and I can finally play.

I make my move

The software works out its move

The robotic arm replicates it in the real chessboard

Conclusions: How complex was it?

Even though to a beginner this seems quite a complex project, it took me only 3 days to build it from scratch.

This is possible because the field of Computer Vision is growing very fast and there are a lot of sources and libraries that can be freely used to build any project you have in mind.

If you want to dive deeper into Computer Vision and Object Detection, I recommend you to start with my free Workshop: Computer Vision blueprint, where you will learn how to detect and track any object in the fastest and most efficient way.