OpenCV background subtraction from a still image

Question

I'm working on an application that will work with an inside mounted camera on the ceiling. The purpose is for it to keep track of objects on a surface.

I need to remove the background, so that I can get the contours of the "diff" that's there, but using BackgroundSubtractorMOG gets frustrating, as I find that its only application is for video.

What I need is to provide a single image that will be the background, and then calculate on each frame from a stream what has changed.

Here's what I have:

#include <libfreenect/libfreenect_sync.h>
#include <opencv2/opencv.hpp>
#include <iostream>
#include <vector>

const char *kBackgroundWindow = "Background";
const char *kForegroundWindow = "Foreground";
const char *kDiffWindow = "Diff";

const cv::Size kCameraSize(cv::Size(640, 480));

int main(int argc, char **argv) {
  uint8_t *raw_frame = (uint8_t *)malloc(640 * 480 * 3);
  uint32_t timestamp;

  // First, we show the background window. A key press will set the background
  // and move on to object detection.
  cvNamedWindow(kBackgroundWindow);
  cv::Mat background(kCameraSize, CV_8UC3, cv::Scalar(0));
  for(;;) {
    freenect_sync_get_video((void **)&raw_frame, &timestamp, 0, FREENECT_VIDEO_RGB);
    background.data = raw_frame;
    cv::cvtColor(background, background, CV_BGR2RGB);

    cv::imshow(kBackgroundWindow, background);
    if(cv::waitKey(10) > 0)
      break;
  }

  // Create two windows, one to show the current feed and one to show the difference between
  // background and feed.
  cvNamedWindow(kForegroundWindow);


  // Canny threshold values for the track bars
  int cannyThresh1 = 20;
  int cannyThresh2 = 50;
  cvNamedWindow(kDiffWindow);
  cv::createTrackbar("Canny Thresh 1", kDiffWindow, &cannyThresh1, 5000, NULL);
  cv::createTrackbar("Canny THresh 2", kDiffWindow, &cannyThresh2, 5000, NULL);


  // Start capturing frames.
  cv::Mat foreground(kCameraSize, CV_8UC3, cv::Scalar(0));
  cv::Mat diff(kCameraSize, CV_8UC3, cv::Scalar(0));

  cv::BackgroundSubtractorMOG2 bg_subtractor(101, 100.0, false);
  bg_subtractor(background, diff, 1);

  for(;;) {
    freenect_sync_get_video((void **)&raw_frame, &timestamp, 0, FREENECT_VIDEO_RGB);
    foreground.data = raw_frame;
    cv::cvtColor(foreground, foreground, CV_BGR2RGB);
    // Calculate the difference between the background
    // and the foreground into diff.
    bg_subtractor(foreground, diff, 0.01);

    // Run the Canny edge detector in the resulting diff
    cv::Canny(diff, diff, cannyThresh1, cannyThresh2);

    cv::imshow(kForegroundWindow, foreground);
    cv::imshow(kDiffWindow, diff);

    cv::waitKey(10);
  }
}

How can I change this so that it doesn't "learn" about the new background, but just uses the static image stored in background?

Thanks!

Solution

If you truly only want a static image as the background, you can simply subtract the background image from the foreground image:

cv::Mat diff;
cv::absdiff(foreground, background, diff);

As a side note, I think your calls to cv::cvtColor() are unnecessary. OpenCV's native image format is BGR, so imshow() will show the red and blue channels swapped if you convert to RGB beforehand.

Comments

[changelog]

The background is still, but the feed has some grain that doesn't quite match 100%. Thanks for the cv::cvtColor tip tho!

[Aurelius]

@changelog That's probably because of noise. You could run both background and foreground through a median filter to reduce the noise.

[changelog]

Sorry, I didn't make it clear enough. I meant the video feed. It's from a crappy camera (Kinect)