How to merge contours in opencv?

Question

Ok guys I have been working on this project for quite some time now.

I am building this bot that plays the chrome dinosaur game. So I tried other methods to detect the characters like matchTemplate and even made my own algorithm to locate the objects, but I like this one (findcontours) the most.

Here's what I have:

Can anyone help me find out how I should merge the two rectangles of the cacti?

img = screen_cap()
roi = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(roi,127, 255, 0)
im2, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
first = True
for cnt in contours:
    area = cv2.contourArea(cnt)
    if area > 200: #filtering contours
        x,y,w,h = cv2.boundingRect(cnt)
        if w/h < 4: # filtering even more
            cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)

Solution 1

This is an old question and seems like it has not been answered properly yet (apologies to fellow SOers who did it partially in comments though). To me it seems like the questioner has a two-part question:

1. Is there a opencv function that merges two rectangles?

Answer to this question is yes and no. Let me be clear; yes if you are using opencv C++ bindings. Simple & can be used to take a union and | for intersection of two rects. But Python bindings lack those functions.

2. How to then do it in Python?

   def union(a,b):
       x = min(a[0], b[0])
       y = min(a[1], b[1])
       w = max(a[0]+a[2], b[0]+b[2]) - x
       h = max(a[1]+a[3], b[1]+b[3]) - y
       return (x, y, w, h)
   
   def intersection(a,b):
       x = max(a[0], b[0])
       y = max(a[1], b[1])
       w = min(a[0]+a[2], b[0]+b[2]) - x
       h = min(a[1]+a[3], b[1]+b[3]) - y
       if w<0 or h<0: return () # or (0,0,0,0) ?
       return (x, y, w, h)
       # Please remember a and b are rects.
   

Source Code Credit: OpenCV union and intersaction on rects

Solution 2

Sorry to be a little late to the party. However if I google "merge opencv contours" I find this; and I think there should be an answer.

You can merge any two contours by one of those recipes:

• Get a list of points of each contour
• append them
• force them into cv2 contour format
• get cv2.convexHull of that if you don't care too much about the details.

If you don't like the result of convexHull because the concave parts of the contours are important, then follow this recipe instead:

• Get a list of points of each contour
• append them
• get a common center
• sort all the points in clockwise order around the center
• force them into cv2 contour format

If the two contours have a lot of concave shapes in them, this could yield a zigzag-pattern as the recipe goes through both contours disregarding their original structure. If that's the case, you need to follow a third recipe:

• Get a list of points of each contour
• get a common center
• delete points of each contour which are inside the other contour
• find the point which is closest to the common center in each contour.
• go through the first contour as it is listed until you hit the closest point.
• Then switch to the other list, starting from the closest point you go clockwise through the other contour until it is used up
• switch back to the first contour and append the rest of their points.
• force them into cv2 contour format

The next-more-complicated case is if you have multiple intersections between the contours and you want to preserve the holes between the two. Then it's better to make a black image and draw the contours in white via cv2.fillPoly(); then get the contours back out via cv2.findContours()

I sketched some steps for the first two recipes here

get a list of points of each contour:

import cv2
list_of_pts = [] 
for ctr in ctrs_to_merge
    list_of_pts += [pt[0] for pt in ctr]

order points clockwise

I use the function of this really great posting of MSeifert to order the points in clockwise order

class clockwise_angle_and_distance():
    '''
    A class to tell if point is clockwise from origin or not.
    This helps if one wants to use sorted() on a list of points.

    Parameters
    ----------
    point : ndarray or list, like [x, y]. The point "to where" we g0
    self.origin : ndarray or list, like [x, y]. The center around which we go
    refvec : ndarray or list, like [x, y]. The direction of reference

    use: 
        instantiate with an origin, then call the instance during sort
    reference: 
    https://stackoverflow.com/questions/41855695/sorting-list-of-two-dimensional-coordinates-by-clockwise-angle-using-python

    Returns
    -------
    angle
    
    distance
    

    '''
    def __init__(self, origin):
        self.origin = origin

    def __call__(self, point, refvec = [0, 1]):
        if self.origin is None:
            raise NameError("clockwise sorting needs an origin. Please set origin.")
        # Vector between point and the origin: v = p - o
        vector = [point[0]-self.origin[0], point[1]-self.origin[1]]
        # Length of vector: ||v||
        lenvector = np.linalg.norm(vector[0] - vector[1])
        # If length is zero there is no angle
        if lenvector == 0:
            return -pi, 0
        # Normalize vector: v/||v||
        normalized = [vector[0]/lenvector, vector[1]/lenvector]
        dotprod  = normalized[0]*refvec[0] + normalized[1]*refvec[1] # x1*x2 + y1*y2
        diffprod = refvec[1]*normalized[0] - refvec[0]*normalized[1] # x1*y2 - y1*x2
        angle = atan2(diffprod, dotprod)
        # Negative angles represent counter-clockwise angles so we need to 
        # subtract them from 2*pi (360 degrees)
        if angle < 0:
            return 2*pi+angle, lenvector
        # I return first the angle because that's the primary sorting criterium
        # but if two vectors have the same angle then the shorter distance 
        # should come first.
        return angle, lenvector

center_pt = np.array(list_of_pts).mean(axis = 0) # get origin
clock_ang_dist = clockwise_angle_and_distance(origin) # set origin
list_of_pts = sorted(list_of_pts, key=clock_ang_dist) # use to sort

force a list of points into cv2 format

import numpy as np
ctr = np.array(list_of_pts).reshape((-1,1,2)).astype(np.int32)

merge them with cv2.convexHull instead

If you use this then there is no need to order the points clockwise. However, the convexHull might lose some contour properties because it does not preserve concave corners of your contour.

# get a list of points
# force the list of points into cv2 format and then
ctr = cv2.convexHull(ctr) # done.

I think the function to merge two contours should be content of the opencv library. The recipe is quite straightforward and it is sad that many programmers using opencv will have to boilerplate code this.

Comments

[Abdelsalam Hamdi]

what if it was multiple contours and they are very close to each other, is there a way to merge them into one contour ?

[Knight Forked]

@abss, you just need to iterate over them to merge them. Alternately you can use morphological dilation on your binary image itself before finding contours, depending on what works best for your use case.

[Abdelsalam Hamdi]

Thanks for you answer, The problem is that, my contours that I want to merge, they are so close that they look like they are one, but there's a very small gab between them, and there's a bit of noise around them so using dilation will include the noise into the contours, I have already filtered the image as much as I can, but I can't remove them completely

[Knight Forked]

@abss, it would be hard to tell you what to do to process the images you have until you post them for others to have a look. If your foreground contours are larger than a certain size as compared to noise then you can filter out smaller contours using cv2.ContourArea() function, redrawing the foreground contours and then running morphological dilation.

[Abdelsalam Hamdi]

yea I understand, I'm currently still trying to do some filtering, if I run out of idea I will post my question with the image. Thanks for your reply

[alkasm]

@abss the usual process is to use morphological closing which dilates the masks so they merge, then erodes them so they shrink back to their original size (but now with the holes/gaps filled).

[fermi]

Thank you Anderas, this is quite helpful!