How does cv2.merge((r,g,b)) works?

Question

I am trying to do a linear filter on an image with RGB colors. I found a way to do that is by splitting the image to different color layers and then merge them.

i.e.:

cv2.split(img)
Sobel(b...)
Sobel(g...)
Sobel(r...)
cv2.merge((b,g,r))

I want to find out how cv2.merge((b,g,r)) works and how the final image will be constructed.

Solution 1

cv2.merge takes single channel images and combines them to make a multi-channel image. You've run the Sobel edge detection algorithm on each channel on its own. You are then combining the results together into a final output image. If you combine the results together, it may not make sense visually at first but what you would be displaying are the edge detection results of all three planes combined into a single image.

Ideally, hues of red will tell you the strength of the edge detection in the red channel, hues of green giving the strength of the detection for the green channel, and finally blue hues for the strength of detection in the blue.

Sometimes this is a good debugging tool so that you can semantically see all of the edge information for each channel in a single image. However, this will most likely be very hard to interpret for very highly complicated images with lots of texture and activity.

What is more usually done is to actually do an edge detection using a colour edge detection algorithm, or convert the image to grayscale and do the detection on that image instead.

As an example of the former, one can decompose the RGB image into HSV and use the colour information in this space to do a better edge detection. See this answer by Micka: OpenCV Edge/Border detection based on color.

Solution 2

This is my understanding. In OpenCV the function split() will take in the paced image input (being a multi-channel array) and split it into several separate single-channel arrays.

Within an image, each pixel has a spot sequentially within an array with each pixel having its own array to denote (r,g and b) hence the term multi channel. This set up allows any type of image such as bgr, rgb, or hsv to be split using the same function.

As Example (pretend these are separate examples so no variables are being overwritten)

b,g,r = cv2.split(bgrImage)
r,g,b = cv2.split(rgbImage)
h,s,v = cv2.split(hsvImage)

Take b,g,r arrayts for example. Each is a single channel array contains a portion of the split rgb image.

This means the image is being split out into three separate arrays:

rgbImage[0] = [234,28,19]

r[0] = 234
g[0] = 28
b[0] = 19

rgbImage[41] = [119,240,45]
    
    r[41] = 119
    g[14] = 240
    b[14] = 45

Merge does the reverse by taking several single channel arrays and merging them together:

newRGBImage = cv2.merge((r,g,b))

the order in which the separated channels are passed through become important with this function.

Pseudo code:

cv2.merge((r,g,b)) != cv2.merge((b,g,r))

As an aside: Cv2.split() is an expensive function and the use of numpy indexing is must more efficient.

For more information check out opencv python tutorials