Understanding Rect in OpenCV C++

Question

I am trying to mark pixels inside a rectangle. The rectangle to be marked is decided in an if-else construct. I am getting the following error on using rectangles which are defined in either of the if-else blocks: "Assertion failed (0 <= roi.x && 0 < = roi.width && roi.x + roi.width <=m.cols && 0 <=roi.height && roi.y +roi.height <= m.rows)"

Here is the code snippet I am using:

if (faces.size() != 0){r = faces[0];}
if (eyes.size()!=0){r2 = eyes[0];}

markers(image.rows,image.cols)
if(faces.size() == 0){
    cout << "No Face found";
    rectangle_face = rectangle_old;
    rectangle_inner = rectangle_inner_old;
    rectangle_outer = rectangle_outer_old;
}
else {
    pt1.x = r.x;
    pt1.y = r.y;
    pt2.x = pt1.x + r.width;
    pt2.y = pt2.y + r.height;

    pt1_inner.x = r.x + (r.width)/3;
    pt1_inner.y = r.y + (r.height)/3;
    pt1_outer.x = pt1.x;
    pt1_outer.y = pt1.y;

    pt2_inner.x = pt2.x - (r.width)/3;
    pt2_inner.y = pt2.y - (r.height)/3;
    pt2_outer.x = pt2.x;
    pt2_outer.y = image.rows;

    rectangle_face = r; 
    rectangle_inner = Rect(pt1_inner,pt2_inner); 
    rectangle_outer = Rect(pt2_outer,pt2_outer);
}


//rectangle_inner = Rect(pt1_inner,pt2_inner); 
//rectangle_outer = Rect(pt2_outer,pt2_outer);

rectangle_old = rectangle_face;
rectangle_outer_old = rectangle_outer;
rectangle_inner_old = rectangle_inner;



// Setting all pixels to possible background first
markers.setTo(cv::GC_PR_BGD);

//It get stuck at the following two lines
cv::Mat1b fg_seed_inside_face = markers(rectangle_inner);
//Marking pixels in order. Note: Order is important here.
cv::Mat1b Prfg_seed_FaceExtended = markers(rectangle_outer);

'faces' is vector of rectangles returned from the detectMultiScale. What baffles me is that if I declare the rectangle_inner and rectangle_outer outside of the if-else blocks (as is commented out in the code-snippet, right below the else condition), the code works fine for the case where the 'if' condition is not true. So basically I am expected to declare my rectangle_inner and rectangle_outer outside the if-else blocks, which doesn't make sense. Is there a work-around?

Solution

When you compute the (x,y) coordinates of your inner and outer points in order to build your rectangles, you have to check that the coordinates you get by adding or subtracting to pt1 or pt2 r.weight or r.height don't get out of the image. This is a fairly common occurrence when your face is close to the edge of your image.

It can be done easily by doing something like

pt1.x = r.x;
pt1.y = r.y;
pt2.x = pt1.x + r.width;
if(pt2.x >= image.cols) //check that coordinate pt2.x doesn't get out of the image
    pt2.x = image.cols;

pt2.y = pt2.y + r.height;
if(pt2.y >= image.rows) //check that coordinate pt2.y doesn't get out of the image
    pt2.y = image.rows;

And repeat the check for each coordinate you compute. When you subtract just check that coordinates are >= 0 and set them to 0 if they aren't.

Hope it helps..

Comments

[TheBlueNotebook]

Hello, I suspected the same and did a check on it. Co-ordinates going out of range is not a problem. I printed the co-ordinates of the rectangle_inner and rectangle_outer, and both of them were within the range. The problem seems to be something with the fact that I am defining rectangle_inner and rectangle_outer inside a block. If I make the exact same definitions outside the if/else blocks, the code seems to work correctly!

[powder]

So if you checked the coordinates in the else I'm guessing the problem can be in the first if(faces.size() == 0){..have you checked rectangle_inner_old and rectangle_outer_old values? There shouldn't be any difference in assigning the values in or out the if block so the problem must be elsewhere..

[TheBlueNotebook]

I have checked those as well. That's why it is baffling me. I know from a previous program that for the first few frames, in the video I am using, faces.size()!=0. Also, when I uncomment the declaration outside the else block, the program works well for the first few frames, where there is a detection of the face. After that it gives the same error.

[TheBlueNotebook]

I am sorry. I was wrong. There is a problem with the range of values itself. But I can't fix it yet. For some reason, the face detector is returning a rectangle whose top-right point itself is outside the frame. I will apply the check you suggested next.

[Micka]

To adjust your rect you can do something like this: cv::Rect imageRect(0,0, imWidth,imHeight); adjustedRect = yourRect & imageRect;