How to detect and extract signature from an image with OpenCV?

Question

I am importing the attached image. After importing the image, I want to remove horizontal lines, detect the signature and then extract it, create rectangle around signature, crop the rectangle and save it. I am struggling to identify entire region of a signature as one contour or a group of contours.

I have already tried findcontour and then various ways to detect signature region. Please refer the code below.

Python Script:

imagePath

#read image
image = cv2.imread(imagePath,cv2.COLOR_BGR2RGB)

#Convert to greyscale
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) # grayscale

#Apply threshold
ret,thresh1 = cv2.threshold(gray, 0, 255,cv2.THRESH_OTSU|cv2.THRESH_BINARY_INV)

plt.imshow(thresh1,cmap = 'gray')


#preprocessing
rect_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (15,15))
dilation = cv2.dilate(thresh1, rect_kernel, iterations = 1)
plt.imshow(dilation,cmap = 'gray')


#Detect contours
contours, hierarchy = cv2.findContours(dilation, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contours[0]

height, width, _ = image.shape
min_x, min_y = width, height
max_x = max_y = 0   
for contour, hier in zip(contours, hierarchy):
    (x,y,w,h) = cv2.boundingRect(contour)
    min_x, max_x = min(x, min_x), max(x+w, max_x)
    min_y, max_y = min(y, min_y), max(y+h, max_y)
    if w > 80 and h > 80:
        cv2.rectangle(frame, (x,y), (x+w,y+h), (255, 0, 0), 2)

if max_x - min_x > 0 and max_y - min_y > 0:
    fin=cv2.rectangle(image, (min_x, min_y), (max_x, max_y), (255, 0, 0), 2)

 

plt.imshow(fin)



final=cv2.drawContours(image, contours,-1,(0,0,255),6)

plt.imshow(final,cmap = 'gray')

Final objective is to create rectangle around entire signature

Trying to generalize on the other image:

Solution

Instead of removing the horizontal lines, it may be easier to perform HSV color thresholding. The idea is to isolate the signature onto a mask and then extract it. We convert the image to HSV format then use a lower/upper color threshold to generate a mask

lower = np.array([90, 38, 0])
upper = np.array([145, 255, 255])
mask = cv2.inRange(image, lower, upper)

Mask

To detect the signature, we can get the combined bounding box for all of the contours with np.concatenate() then use cv2.boundingRect() to obtain the coordinates

Now that we have the bounding box coordinates, we can use Numpy slicing to crop and extract the ROI

import numpy as np
import cv2

# Load image and HSV color threshold
image = cv2.imread('1.jpg')
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
lower = np.array([90, 38, 0])
upper = np.array([145, 255, 255])
mask = cv2.inRange(hsv, lower, upper)
result = cv2.bitwise_and(image, image, mask=mask)
result[mask==0] = (255, 255, 255)

# Find contours on extracted mask, combine boxes, and extract ROI
cnts = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
cnts = np.concatenate(cnts)
x,y,w,h = cv2.boundingRect(cnts)
cv2.rectangle(image, (x, y), (x + w, y + h), (36,255,12), 2)
ROI = result[y:y+h, x:x+w]

cv2.imshow('result', result)
cv2.imshow('mask', mask)
cv2.imshow('image', image)
cv2.imshow('ROI', ROI)
cv2.waitKey()

Note: The lower/upper color ranges were obtained from choosing the correct upper and lower HSV boundaries for color detection with cv::inRange (OpenCV)

Comments

[user3203657]

Thanks..one query..Using this approach, i would not be able to exttact only signatute..how to identify starting and ending point for a signature and crop only that part of the image? can u pls help on that part?

[nathancy]

Check the update, you can get the bounding box for all the contours then use Numpy to find the top left and bottom right coordinates. From there, you can extract the ROI

[user3203657]

I am not able to generalize this solution for some of the other images. Tried the same approach on one more image, but it's not working. I have uploaded the image above. Can you please help?

[nathancy]

Try modifying the number of iterations in cv2.morphologyEx. There is more noise in the 2nd image so you need to erode then dilate to clean up the small particles. I'm able to get good results setting the opening iterations to 4

[user3203657]

Yes..tried some morphological transformations and it worked really well...Thnks nathancy..

[Rajneesh071]

Hi, I want to do this in swift any suggestions ?

[Tecnologia da Net]

@nathancy do you know how i can do this extraction using android studio with opencv?