Real-time defect identification of products on a conveyor belt

In this video, we will see how to build Real-time defect identification software for plastic bottles. Obviously, this is an operation that can be performed manually perhaps by taking samples of a production batch. Large companies already have sophisticated systems to check every single bottle, in this tutorial we will see how Real-time defect identification software can be developed in a simplified way.

Real-time defect identification description

For the realization of Real-time defect identification software, we will consider the defects of a typical plastic bottle that could be damaged during a production cycle. With image processing, if the bottle is damaged, the defect is classified and entered into the database. To have more accurate control of the damage and verify via software, each bottle is associated with an ID and photographed to be inserted in a folder.

In particular, we are interested in knowing the following:
1. Water level
2. Bottle defect, bottle defects or damaged plastic
3. Problem with the label, damaged or missing
4. Different bottles or general problems

In the image below there is an example of recognition of a labeling defect of a bottle. The software recognizes it and simultaneously saves all the data in order to precisely identify the bottle.

If required, physical activity can be added, for example activating a gate to throw away the defective bottle.

How does Real-time defect identification software work?

To create this prototype I used a basic approach to computer vision: object detection combined with object tracking. The level of this proof of concept was sufficient but if you want to learn more about the subject I suggest you read this article: Research and implementation of machine vision technologies for empty bottle inspection systems.

Object detection on Real-time defect identification

Object detection consists of a bounding box around the identified object, in this way we can obtain the exact position of the object and classify it, for example, a bottle or label. In order to identify the bottle with deep learning it is necessary to teach the software what a bottle is, this is done by training.

As I explained above Build your OBJECT DETECTION SOFTWARE – Crash course I used the object detection method with deep learning to recognize the defects and identify the bottles.

Object tracking

To prevent each bottle from being double-checked several times or in any case keeping track of it accurately, it is necessary to have a track to assign it a unique ID.

If you want to learn more about the principles of object tracking in this article I explain Object tracking from scratch – OpenCV and python but if you want to develop a complete project I advise you to evaluate my course Object Detection (Opencv & Deep learning).

Database

Once the defects have been identified and an id assigned to every single bottle, we proceed with saving the data in an SQL database. In this way, everything will be verifiable and it will be possible to estimate the percentage of errors present in a batch and generally monitor production.

Hardware

Large resources are not needed to carry out this project but it is necessary to carefully evaluate the type of hardware to be used that is compatible with our needs.

If the processing is not particularly heavy, a simple Nvidia jetson nano or Jetson Xavier could be enough.

Otherwise, you could opt for a common gaming laptop that has an Nvidia graphics card or evaluates more performing solutions.