How to code an OpenCV project from a phone (with Google Colab)

In this tutorial, we will see how to make an entire OpenCV project from a phone project entirely from your phone. It doesn’t matter which phone you have, all you need is an internet connection and a browser.

You can take a picture and upload it to your notebook to identify objects and get a report. For example, how many people are in the photo, how many computers, etc.

How to start OpenCV project from phone?

We have to do everything with Google Colab in order to take advantage of the computing power made available by Google video cards. So make sure you have a Google account you generally get it by making a Gmail account, now open your browser and enter this address:

colab.research.google.com

As soon as you enter and after logging in you will see a page similar to the photo below.

Now choose one or let’s create a new one and get started. By clicking on the menu at the top left we can extend the list of options

For example, to upload photos from your phone select Show file browser and you will see your file listed as in the photo below on the right.

• 

• 

Show the image on google colab

Before proceeding with the OpenCV project from the phone, let’s do a test to see if the image we upload is displayed correctly. With this operation, we also import the OpenCV library which is generally already installed.

In the first cell enter this code and press play

import cv2

img= cv2.imread("office.jpg)

from google.colab.patches import cv2.imread

cv2_imshow(img)

So it is like the photo shows

Create an object detection project on the phone

Now we are going to create an object detection similar to a project already carried out on this site: Train YOLO to detect a custom object (online with free GPU). For the phone, I have already prepared all the material for a quick installation and simplified use. Let’s go step by step

1. Install the library of OpenCV project from phone

Open a new cell and run this command to install all required libraries to the object detector.

!git clone https://github.com/pysource7/simpledetection

2. Download DNN model

Everything has been foreseen in the installation so it is sufficient to call the libraries in a new cell. This model allows you to recognize 80 of the most common objects.

# Object detection
from simpledetection.objectdection import ObjectDetection

od = ObjectDetection()

Running the python code will return confirmation that it has downloaded the model. In the photo you can see how the result should be.

2. Run Object detector

Nuvo step and finally we can do the object detection on our photo. The library installed allows us to do it with a very simple code.

class_ids, scores, bboxes = od.detect(img)

for class_id, score, box in zip(class_ids, scores, bboxes)
    
    (x,y,w,h) = box
    cv2.rectangle(img, (x,y), (x + w, y + h), (255,0,0),3)

cv2_imshow(img)

Before showing the result here is a quick explanation of what we pulled out. Each identified image has 3 associated variables:

1. Class_id: indicates which class the object corresponds to. Each id represents a type of object.
2. Score: the percentage associated with the identification of the object.
3. Box: returns the coordinates of the top left point and the bottom right point. With these points you can draw a box that contains the object.

With this information, there is everything you need to identify objects and obtain coordinates. This is the result

3. Associate the object with its class

By modifying a little the code we have seen previously we can assign a different color for each type of object and write what it is on it. For example, people will have a brown rectangle and it will have “Person” written on it

class_ids, scores, bboxes = od.detect(img)

for class_id, score, box in zip(class_ids, scores, bboxes)
    
    (x,y,w,h) = box

    class_name = od.classes[class_id[0]]
    color = od.ccolors[class_id[0]]
    cv2.puText(img,class_name, (x, y -5), 0, 1, color, 2 )

    cv2.rectangle(img, (x,y), (x + w, y + h), color,3)

cv2_imshow(img)

This is the result. Even though it looks very small, each object category has its own writing on it

4. Generate reports of things found

We modify the code further to make it so that if it finds people, laptops or books, it increments the variables.

#count specific object
person = 0
laptop = 0
book = 0

class_ids, scores, bboxes = od.detect(img)

for class_id, score, box in zip(class_ids, scores, bboxes)
    
    (x,y,w,h) = box
    class_name = od.classes[class_id[0]]

    if class_name == "person":
        person +=1
    elif class_name == "laptop":
        laptop +=1
    elif class_name == "book":
        book +=1
    color = od.ccolors[class_id[0]]
    cv2.puText(img,class_name, (x, y -5), 0, 1, color, 2 )

    cv2.rectangle(img, (x,y), (x + w, y + h), color,3)

cv2_imshow(img)

Now let’s restart everything to make sure that the changes are applied and proceed with a new cell. At this moment, you only need to insert the variables in a text file.

print(person)
print(laptop)
print(book)

with open("summary.txt", "w") as fo:
    fo.write("In this picture there are:")
    fo.write(str(laptop) + " laptops")
    fo.write(str(book) + " books")
    fo.write(str(person) + " people")

This you see in the picture is the result after running the code

If we look in the list of files we find “summary.txt” which is exactly the file we created.

Downloading the file, this is the result