Calculate the Area of an object | with Mask R-CNN and Python

In this tutorial, we will see how to Calculate the Area of an object with Mask R-CNN. We will get the value in square centimeters and this is very useful in various sectors, for example when we have to check the defects of an object, if the surface is different it means that it has a problem. Not only the surface of an entire object can be identified but also parts of it, such as the logo in a car wheel (as in the image below).

For simplicity and for educational use we will do all this with common lego bricks. We will calculate the area in square centimeters and show the value on the screen. We will be using Google Colab for the entire process, so there will be no tedious installation required.

Goals to be achieved to calculate the Area of an object

There are two specific requirements to complete the project:

1. You have to be able to Detect and segment the object, in this case a lego brick
2. Define a ratio to be able to correctly convert the measurements into square centimeters. Because the same object if close to the camera has many more pixels than a farther object.

Write the code on the Google Colab notebook

Now we can start with the Google Colab notebook, and all the material is ready to use, you don’t have to load anything.

Installation

As soon as you open the notebook you will need to install Mask R-CNN, just click on the “start” button to install. As in the image below

Load images and detect legos for Calculate the Area of an object

To make the job faster I have already done the training with Mask R-CNN and you just run the Colab notebook to get started. The result you get should look like this in the image below.

The first step is to identify the object and draw bounding boxes. To facilitate the operations I have already written the code to use Mask R-CNN for this purpose, we only have to use OpenCV to extract the coordinates and do the other processing

import cv2
from mrcnn_colab_engine import detect_contours_maskrcnn, draw_mask

img = cv2.imread("lego_image_sample.jpg")

# 1. Get objects mask with Mask RCNN
class_ids, boxes, masks = detect_contours_maskrcnn(lego_model, img)
for class_id, box, object_contours in zip(class_ids, boxes, masks):
    # Box
    y1, x1, y2, x2 = box
    cv2.rectangle(img, (x1, y1), (x2, y2), colors[class_id], 15)

By applying the code to the image we obtain this result, it is the first step but to calculate the area is not enough.

By offering all the coordinates we can also draw the mask with OpenCV to see if everything works correctly. Just add these two lines of code.

    cv2.polylines(img, [object_contours], True, colors[class_id], 10)
    img = draw_mask(img, [object_contours], colors[class_id])

and this is the result

Calculate the Area of an object

Calculating the area is very simple, just pass the points to the OpenCV function cv2.contourArea(object_contours) and we get the area. The only problem is that the unit of measurement is the pixel, so we have to do a little conversion calculation.

Using a tool to calculate pixels such as GIMP and a reference image, in this case, a ruler, we can understand how many pixels correspond to a centimeter, and based on this we can do a conversion ratio operation.

As you can see from the image, we now know that 78 pixels are 1cm. Based on this data, here is the code to convert pixels to square centimeters.

RATIO_PIXEL_TO_CM = 78 # 78 pixels are 1cm
RATIO_PIXEL_TO_SQUARE_CM = 78 * 78

...

    # 2. Calculate area
    area_px = cv2.contourArea(object_contours)
    area_cm = round(area_px / RATIO_PIXEL_TO_SQUARE_CM, 2)

    cv2.putText(img, "A: {}cm^2".format(area_cm), (x1, y1), cv2.FONT_HERSHEY_PLAIN, 3, colors[class_id], 5)

By adding the text to each lego in the image we can see the area of each individual object.

Train easily your model with Mask R-CNN

For this tutorial, I have already prepared the Mask R-CNN model to identify the legos but if you need a custom model I suggest you take a look at my mini-course on Train Mask R-CNN to detect any custom object.

In this mini-course I have simplified the training process of Mask R-CNN as much as possible, you can do it on Google Colab and in just 3 steps:

1. Annotate the Images (Image Dataset)
2. Upload Dataset into Google Drive
3. Run Python Notebook (on Google Colab)