You will learn:

  1. How to Improve the classifier to find the filter automatically
  2. Increase the difficulty of the detection by adding new images and with bigger size
  3. Test the classifier with new images

Find the filter automatically

In the previous lesson we learnt how to create a really simple image classifier that was able to detect if the image was a horizontal line or a vertical line.
The drawback of that classifier was that we needed to create manually the filter to classify the images. So if we wanted to classify different type of images, we would need again to create manually a new filter.

We’re going to see right now how to let the computer do this operation for us, so that it all will become more automatic.

Preprocess the image

First of all we create a function to preprocess the image before applying the classifier, so that instead of dividing each single image by 255 and flattening it, we will get in return this operation aleady executed by the function.

The function preprocess_img will take the image path and give in return the image ready to bre processed by the classifier.

def preprocess_img(path_img):
    img = cv2.imread(path_img, cv2.IMREAD_GRAYSCALE)

    # Simplify the image
    img = img / 255

    # Flatten the image
    flattened_image = img.flatten()
    return flattened_image


flattened_vertical = preprocess_img("images/vertical.png")
flattened_horizontal = preprocess_img("images/horizontal.png")

Find the filter

Once we have the images loaded, we can move forward and look for the filter.

How do we know if the filter is good?
The filter is good if it after the convolution operation and the sum, returns a different value for the two images.
If it would return the same result, would be a bad filter because no matter if you give a vertical line or a horizontal line you get the same result and there is no distinction.

How do we create the filter?
We create the filter putting random numbers from -1 to 1. There is a numpy function that does that: np.random.randint().
The size must be the same size of the images we want to multiply the filter with, so in this case is 9 numbers.

filter_images = np.random.randint(-1, 2, size=9)

What we have to do is the convolution operation (multyply the image with the filter and sum the result) for both images.

Then we compare the result of the convolution for both images. If the result of the two images is different, we can say it’s a good filter and we can use that one. Otherwise it’s a bad filter and we want to discard it.

We will put everything into a loop, and keep looping until we find a good filter.

count = 0
while True:
    count += 1
    filter_images = np.random.randint(-1, 2, size=9)

    convolution_vertical = sum(flattened_vertical * filter_images)
    convolution_horizontal = sum(flattened_horizontal * filter_images)

    if convolution_horizontal != convolution_vertical:
        print("Good filter")
        print("Filter: ", filter_images)
        print(convolution_vertical)
        print(convolution_horizontal)
        print("Attempts: ", count)
        break

2. Add more images

If we were able to easily and quickly detect a filter that would work having just one image per class, so one image for the “Vertical” and one for the “Horizontal”, how would this method perform if we had more images per class?

We’re going to to that right now, but to make the detection more challenging, we will try this time with more difficult images of size 5×5.

I prepared 5 images containing squares and 5 images containing crosses.

Squares:

Crosses:

Now the task gets more challenging.
We don’t need to load just two images, but more images and the condition after the convolution must satisfy more images for the same class.

Let’s start by taking the path of all the 10 images, so we can easily load them. For the squares I have 5 images and their name starts with square, so I take all the images with square on their name. Same operation I do for the crosses.

squares = glob.glob("images/square*.png")
crosses = glob.glob("images/cross*.png")

We then need to run a while loop. We need the while loop to keep looking for a filter until we get a good one. This time we also add a switch button because it won’t be so easy to break the loop as we did on the previous code with just the break operator.

Inside the loop we generate a random filter.

Then we load the 5 squares and 5 crosses and we make the convolution for each single image.

In order to consider the filter a good filter, we set two requirements: all the squares need to be bigger or equal to 0, so if the result is different we break the for loop and we go back to the while loop and start everything until all the 5 squares satisfy the condition to be greater or equal to 0.

The same is for the crosses, if they’re not less than 0, we break the for loop.

How do we know when all the 5 squares and 5 crosses satisfy the requirements?
We know it if we’re able to get through all the images without breaking the loop. So we set the final condition, if count is 4 it means we were able to loop through all the 5 images, so the filter is good.
At this point we set good_filter to True and the while loop will stop running.

count = 0
good_filter = False
while good_filter is False:
    count += 1

    # random filter
    filter_images = np.random.randint(-1, 2, size=25)

    for i in range(5):
        flattened_img = preprocess_img(squares[i])
        convolution = sum(flattened_img * filter_images)

        flattened_img2 = preprocess_img(crosses[i])
        convolution2 = sum(flattened_img2 * filter_images)

        if convolution < 0:
            break

        if convolution2 >= 0:
            break

        if i == 4:
            good_filter = True


print("Filter:", filter_images)
print("Attempts:", count)

How does the code perform?
Eventually using this code you will be able to find a good filter. It might take from a few seconds to a few minutes, during which the computer will be able to make several attempts to find a good filter.
In my case it took more than 100 thousands attempt to find a good filter.

3. Test the classifier on a new Image

Once we find the filter, we can be sure that tha image classifier will work with the 5 images we used to find the filter, as it was tested exactly for that.

Let’s adjust a bit the classifier using the filter I’ve just generated. You might have a different filter than the one I have, so you can use the one you got if you prefer.

The filter this time is 25 numbers, as the images are of size 5×5 (which is a total of 25 pixels).

def classify_image(img):
    # 1) Simplify image by dividing 255
    img = img / 255

    # 2) Flatten the image
    img_flattened = img.flatten()

    # 3) Filter
    filter = [ 1,  1, -1,  1,  0,  0, -1,  0,  1, -1,  0,  0, -1,  0, -1, -1, -1,  1,  0,  1,  1, -1, -1,  0, 1]

    # 4) Multiply filter * flattened image
    convolution = img_flattened * filter

    # 5) Sum
    sum_convolution = sum(convolution)

    # 6) Condition
    if sum_convolution < 0:
        return "Cross"
    else:
        return "Square"

But what if we try it with some image that we never used before?
I tested it with a few images placing the crosses and the squares in different positions and it did work decently.
For example, the image below which is a square, with the filter I have is detected as a cross.
Your filter might perform differently, but still most likely it won’t be so good, as neither it’s mine.