How Artificial Intelligence counts people and vehicles from CCTV cameras
Is it simple to do the Vehicle and People counting on CCTV ?
I made this video to explain, in 5 steps, what are the tools needed to start automating the image analysis of CCTV cameras, using deep learning and AI.
Several companies have contacted me and it is common to ask for information analysis such as counting the number of people entering a store and knowing how long they stay or figure out how to improve the efficiency if you have a gas station.
The answer to these questions is not always trivial. For example, someone would think that any object detection method is enough to know how much a person remains in the store, in reality, if there is more than one person the system goes into a crisis for not considering the use of more cameras. This is why it must always be considered a broader project and with this article, I will try to clarify at least the basics.
Here’s what you will find:
- Object detection
- Object tracking
- Select ROI (Region of Interest)
- Detecting ID’s crossing ROI
- Display ID’s total count
1. Object detection
For object detection, I always recommend using a deep learning model because it is extremely more precise than using OpenCV alone. In this way, we can always have a bounding box of the object so we get the exact position and we can define the class. For example, we can know if it is a car, person, truck, or other.
Often for object recognition, we talk about the famous YOLO deep learning model and consequently, the question I get most often is: Which version should I use for my project?
It always depends on many factors and often even a different or specially designed deep learning model must be chosen. First of all, you have to analyze the type of project, hardware in which it is to be used, budget, and required accuracy. It may be enough to change a few parameters or you have to completely change the deep learning model to use.
In this case, I have used YOLOv4 and this is the result of the first start on the video images of a security camera. As a first start, it is a very good result.
As you can see there are many recognized objects but they are things that we do not need. We need to recognize buses, cars, trucks, and people. What we need to understand is that it is not important that it is accurate in each frame but only that the object is identified in a precise area.
After selecting the classes of interest, our program already begins to have what we need.
2. Object tracking
Object tracking allows us to assign an ID per recognized object and track it in each frame for the duration of the video. We will not go into the details of the operating technique but commonly SORT or Deep SORT is used.
I suggest you see my previous article for more details on object tracking: Object Tracking with Opencv and Python In this example, we will use Deep SORT and see how each object in the image has its own ID.
3. Select ROI (Region of Interest)
ROI indicates the area of interest, this means selecting specific areas in order to save the data when an object passes over it and then count it.
In this case that the CCTV is fixed it is enough to simply identify the coordinates and if we want to show the areas it is possible to do it with OpenCV. As you can see from the image I have drawn the yellow area for cars and the purple area for people.
When an object passes in the yellow or purple polygon, detect the ID.
I used a similar process in this project too: Agriculture plant analysis with the drone and Artificial Intelligence
4. Detecting ID’s crossing ROI
Once the area of interest has been determined and thanks to the standard functions of OpenCV we can easily understand when to count the object. In my example, when the most extreme point, which I have marked with a red balloon, crosses at least one side of the polygon, the count can be activated.
To count each object uniquely we must:
– Understanding which category it belongs to, for example, if a car or a person
– Add it to a data list and in this case set() which allows you to count only once
– Make sure the object is identified with a unique ID
5. Detecting ID’s crossing ROI
Now we have everything and we just have to show the counted objects on the screen. Taking the Python lists and using the len() function we get the number of machines and people for both lists.
How can i develop my project?
If you have the knowledge you can take this article as a starting point to design your video surveillance system with Artificial Intelligence and Deep Learning.
If you want a step by step guide on ‘Object Detection, Object Tracking, and the basics of OpenCV necessary to operate at best I recommend my course: Build COMPUTER VISION PROJECTS, even though you only have basic programming knowledge.
You will also find an entire project on drone footage in a broccoli plantation.