We’re going to learn in this tutorial how to subtract the background on a video.

The concept of background subtraction is really simple. On the video we take the first frame, and we find the absolute difference with another frame.

Let’s see an example where we take the first frame and the frame 100 and we compute the absolute difference. The result will a mask where in black are the parts that are images in both the images and white the parts that are different.

In this way is possible to distinguish the stable background from the objects that are moving. Keep in mind that this method only works with a stable camera and a stable background.

On opencv we have two ways to subtract the background:

  1. A manual way which consists on taking the first frame and from that one subtricting each time the following frames from the first one.
  2. The SubtractorMOG2 which has more advanced features, like for example keeping the history of last number of frames and detecting shadows.

Manual subtraction from first frame:

We import the libraries and load the Video.

import cv2
import numpy as np

cap = cv2.VideoCapture("highway.mp4")

We then take the first frame from the video, convert it to gray scale and apply Gaussian Blur to remove some noise.

_, first_frame = cap.read()
first_gray = cv2.cvtColor(first_frame, cv2.COLOR_BGR2GRAY)
first_gray = cv2.GaussianBlur(first_gray, (5, 5), 0)

We run now the while loop so we load one by one the following frames.

while True:
    _, frame = cap.read()
    gray_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray_frame = cv2.GaussianBlur(gray_frame, (5, 5), 0)

And now we get to the core part of the background subtraction where we compute the absolute difference between the first frame and the current frame we are in inside the loop.

    difference = cv2.absdiff(first_gray, gray_frame)
    _, difference = cv2.threshold(difference, 25, 255, cv2.THRESH_BINARY)

At the end we show everything on the screen.

    cv2.imshow("First frame", first_frame)
    cv2.imshow("Frame", frame)
    cv2.imshow("difference", difference)
   
    key = cv2.waitKey(30)
    if key == 27:
        break

cap.release()
cv2.destroyAllWindows()

Subtraction using Subtractor MOG2:

The Subtractor MOG2, is already built in Opencv and it’s simpler to use than the manual mode.

The SubtractorMog2 has the advantage of working with a frame history, it works by default with the last 120 frames, but you can change it and we will see later how).

Why is working with a frame history an advantage?

Let’s consider that you take the first frame of a high way during the day, after a few hours when the sun goes down, computing the difference from the first frame taken a few hours before would not work, the images on the day and night will be completely different.
Working with the frame history you don’t have this problem.

We load the libraries and the video.

import cv2
import numpy as np

cap = cv2.VideoCapture("highway.mp4")

Then we load the subtractor.

Inside the parenthesis we can change the value of the subtractor. History is the number of the last frame that are taken into consideretion (by default 120).
The threshold value is the value used when computing the difference to extract the background. A lower threshold will find more differences with the advantage of a more noisy image.
Detectshadows is a function of the algorythm that can remove the shadows if enabled.
There are no right or wrong values, you need to try different settings to see what best fits your need.

subtractor = cv2.createBackgroundSubtractorMOG2(history=100, varThreshold=50, detectShadows=True)

We then run the while loop to get the frames from the video.

while True:
    _, frame = cap.read()

Once we have the frames we can use the subtractor to find get the background.

    mask = subtractor.apply(frame)

Finally we show everything on the screen.

    cv2.imshow("Frame", frame)
    cv2.imshow("mask", mask)

    key = cv2.waitKey(30)
    if key == 27:
        break

cap.release()
cv2.destroyAllWindows()