How to remove the background from a picture in OpenCV python

Question

Because I am new to computer vision. I would like also to ask how can I delete the whole background of this image and keep only the pills untouched. I tried different things like to change the background color but still, there are some small edges and also noise.

Or if it's possible for all the white background to be a neutral color, without the line between the circle.

Solution 1

Here is one way in Python/OpenCV. Threshold the image on white. Then apply some morphology to clean it up a bit. Then invert it to make a mask. Then apply the mask to the input. I note that your pills overlap the ring. So this method does not remove the ring.

Input:

import cv2
import numpy as np

# Read image
img = cv2.imread('pills.jpg')
hh, ww = img.shape[:2]

# threshold on white
# Define lower and uppper limits
lower = np.array([200, 200, 200])
upper = np.array([255, 255, 255])

# Create mask to only select black
thresh = cv2.inRange(img, lower, upper)

# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (20,20))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# invert morp image
mask = 255 - morph

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask)


# save results
cv2.imwrite('pills_thresh.jpg', thresh)
cv2.imwrite('pills_morph.jpg', morph)
cv2.imwrite('pills_mask.jpg', mask)
cv2.imwrite('pills_result.jpg', result)

cv2.imshow('thresh', thresh)
cv2.imshow('morph', morph)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Threshold image:

Morphology cleaned image:

Mask image:

Result:

Solution 2

Here is another way to do that in Python/OpenCV removing the ring. But it will remove parts of the pills that overlap the ring.

• Read the input
• Threshold on white
• Apply morphology close to remove the center strip
• Get the contours
• Draw the contours as white filled on black background
• Get the convex hull of the white filled contours
• Fit an ellipse to the convex hull
• Print the ellipse shape to make sure it is close to a circle
• Draw the convex hull outline in red on the input to check if fits the white region
• Draw a circle using the average ellipse radii and center as white filled on black background
• Erode the circle a little to avoid leaving a partial white ring
• Combine the inverted morph image and the circle image to make a final mask
• Apply the final mask to the input
• Save the results

import cv2
import numpy as np

# Read image
img = cv2.imread('pills.jpg')
hh, ww = img.shape[:2]

# threshold on white
# Define lower and uppper limits
lower = np.array([200, 200, 200])
upper = np.array([255, 255, 255])

# Create mask to only select black
thresh = cv2.inRange(img, lower, upper)

# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (20,20))
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)

# get contours
contours = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]

# draw white contours on black background as mask
mask = np.zeros((hh,ww), dtype=np.uint8)
for cntr in contours:
    cv2.drawContours(mask, [cntr], 0, (255,255,255), -1)

# get convex hull
points = np.column_stack(np.where(thresh.transpose() > 0))
hullpts = cv2.convexHull(points)
((centx,centy), (width,height), angle) = cv2.fitEllipse(hullpts)
print("center x,y:",centx,centy)
print("diameters:",width,height)
print("orientation angle:",angle)

# draw convex hull on image
hull = img.copy()
cv2.polylines(hull, [hullpts], True, (0,0,255), 1)

# create new circle mask from ellipse 
circle = np.zeros((hh,ww), dtype=np.uint8)
cx = int(centx)
cy = int(centy)
radius = (width+height)/4
cv2.circle(circle, (cx,cy), int(radius), 255, -1)

# erode circle a bit to avoid a white ring
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (6,6))
circle = cv2.morphologyEx(circle, cv2.MORPH_ERODE, kernel)

# combine inverted morph and circle
mask2 = cv2.bitwise_and(255-morph, 255-morph, mask=circle)

# apply mask to image
result = cv2.bitwise_and(img, img, mask=mask2)

# save results
cv2.imwrite('pills_thresh2.jpg', thresh)
cv2.imwrite('pills_morph2.jpg', morph)
cv2.imwrite('pills_mask2.jpg', mask)
cv2.imwrite('pills_hull2.jpg', hull)
cv2.imwrite('pills_circle.jpg', circle)
cv2.imwrite('pills_result2.jpg', result)

cv2.imshow('thresh', thresh)
cv2.imshow('morph', morph)
cv2.imshow('mask', mask)
cv2.imshow('hull', hull)
cv2.imshow('circle', circle)
cv2.imshow('mask2', mask2)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Threshold image:

Morphology image:

Filled contours image:

Convex hull on input:

Circle image:

Final mask image:

Result:

Comments

[Renos Bardhis]

Yes indeed very helpful and well explained. However, I tried to apply to another picture, and the picture was not so great.

[Renos Bardhis]

I add another image, I tried to apply your code to the second image but were not so good.

[Renos Bardhis]

I fix the issue with the other picture. Just need to adjust the lower and upper array

[fmw42]

Post the other picture. Does it have a white center circle? Is the white pure white? You may have to adjust the inRange values.

[nathancy]

@fmw42 To remove the ring and just isolate the pills, you do this continuation process. Starting from your morphology cleaned image, find the minimum enclosing circle then completely fill this onto a new mask. From there dilate this new mask and then bitwise-or to isolate only the added border section. Next we bitwise-and this with the morphology cleaned image. This should "separate" the pills from the ring so finally we just findContours to isolate all the pills :)

[fmw42]

@nathancy. Thanks. But is that not similar to what I posted below. I did a bit more round about method using the convex hull. Your minEnclosingCircle is a good idea. I had forgot about that. I wish OpenCV had a fitCircle in addition to the fitEllipse. Is there some way to use fitEllipse for force it to find a circle with one radius?

[rv.kvetch]

+0. +1 for the idea at the start about using cv.inRange to threshold between a range, but that's about all I'm using. -1 because the morphology is complicated and i have no idea how it works, and furthermore it does not work, at least for generic images where I'm trying to capture the background (for example where a person standing in front of a wall of any color or hue). The morphology fails completely to detect foreground/background in most of these generic cases. Note: as it evens out, I have not voted on this answer.