How to extract a specific section of an image using OpenCV in Python?

Question

I am trying to extract a portion of an image by performing Canny edge detection. I have succesfully created a mask of that object. But when I perform a bitwise_and operation with the original image, to extract the foreground section, I am getting the following error .

OpenCV Error: Assertion failed ((mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1)) in cv::binary_op, file C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp, line 241
    Traceback (most recent call last):
      File "C:\Users\Boudhayan Dev\Desktop\extraction.py", line 37, in <module>
        new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)
    cv2.error: C:\projects\opencv-python\opencv\modules\core\src\arithm.cpp:241: error: (-215) (mtype == CV_8U || mtype == CV_8S) && _mask.sameSize(*psrc1) in function cv::binary_op

My code is as follows -

import cv2
import numpy as np

img_rgb = cv2.imread("3.jpg")
cv2.namedWindow("Original Image",cv2.WINDOW_NORMAL)

img = cv2.cvtColor(img_rgb,cv2.COLOR_RGB2HSV)
img = cv2.bilateralFilter(img,9,105,105)
r,g,b=cv2.split(img)
equalize1= cv2.equalizeHist(r)
equalize2= cv2.equalizeHist(g)
equalize3= cv2.equalizeHist(b)
equalize=cv2.merge((r,g,b))

equalize = cv2.cvtColor(equalize,cv2.COLOR_RGB2GRAY)

ret,thresh_image = cv2.threshold(equalize,0,255,cv2.THRESH_OTSU+cv2.THRESH_BINARY)
equalize= cv2.equalizeHist(thresh_image)


canny_image = cv2.Canny(equalize,250,255)
canny_image = cv2.convertScaleAbs(canny_image)
kernel = np.ones((3,3), np.uint8)
dilated_image = cv2.dilate(canny_image,kernel,iterations=1)


new,contours, hierarchy = cv2.findContours(dilated_image, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
contours= sorted(contours, key = cv2.contourArea, reverse = True)[:10]
c=contours[0]
print(cv2.contourArea(c))
final = cv2.drawContours(img, [c], -1, (255,0, 0), 3)



mask = np.zeros(img_rgb.shape,np.uint8)
new_image = cv2.drawContours(mask,[c],0,255,-1,)
new_image = cv2.bitwise_and(img_rgb,img_rgb,mask=mask)


cv2.namedWindow("new",cv2.WINDOW_NORMAL)
cv2.imshow("new",new_image)

cv2.imshow("Original Image",img)
cv2.waitKey() 

NOTE :- The code works fine if I try to perform the bitwise_and with a grayscale version of the image. However RGB,HSV or any other color spaces give the error above.

Please help.

---

EDIT 1 - The image in question is this -

EDIT 2-

The following is the result after using Numpy method. As you can see, the extracted image is the same size as the orange but it does not contain the orange instead the mask itself.

EDIT 3- @DanMašek and @lightalchemist , I could finally extract any foreground image.

Thank you

Solution 1

I used the code provided above but only altered the line where the cv2.bitwise_and() is used:

new_image = cv2.bitwise_and(img_rgb, img_rgb, mask = equalize)

This is what I got and what you expected (I guess):

EDIT

I get it, you want to mask your image with an image of contour having the greatest area. In the following additional snippet I have binarized the image containing the contour of greatest area to be used as mask.

new_image = cv2.drawContours(mask,[c], -1, (255,255,255), -1)
new_image_gray = cv2.cvtColor(new_image, cv2.COLOR_BGR2GRAY)
ret, thresh1 = cv2.threshold(new_image_gray, 100, 255, cv2.THRESH_BINARY)
final = cv2.bitwise_and(img_rgb, img_rgb, mask = thresh1)

This is what I got:

Compared to the image above you do not see those holes inside the object of interest.

Solution 2

The error is telling you that you are trying to perform the bitwise_and operation on matrices where the entries are not integers. I believe the matrices also have to have the same number of channels. This is why it works with your grayscale image but not with the HSV image.

Instead of using bitwise_and, it is easier and more flexible to just use numpy matrix vectorization to perform the masking like so:

mask = np.zeros_like(img_rgb, dtype=np.uint8)

# I believe you want to draw the filled in contour on the mask
# You code actually assigns the resulting mask to new_image
# But that does not affect things as drawContours modifies mask in place
mask = cv2.drawContours(mask, [c] ,0, 255, -1) 

new_image = img_rgb.copy()
new_image[mask < 255] = 0  # Set values not masked to be 0

Note that if your mask is a single channel matrix instead of a 3 channel matrix, you would have to modify the code to be

new_image[mask < 255, :] = 0

Comments

[Boudhayan Dev]

I think the values are not integers mostly because of the equalization step performed earlier. Your code work, but the problem is the new image has the mask as the output. Instead of the mask I wanted the foreground image. I'll edit the answer to explain what is happening.

[Boudhayan Dev]

It works for this specific image. For other images, there is trace of unwanted background image here and there. That happens because the ` equalize ` variable stores edges besides the object edge as well. It is only after eliminating them via finding largest contour, that we end up with foreground image edge only. And that's why equalize is not used as mask in my code. But thanks anyway.

[Jeru Luke]

@BoudhayanDev I have edited the answer. Hope it suits your expectation.