Detecting LED object status from Image

Question

My question is similar to OpenCV: Detect blinking lights in a video feed openCV detect blinking lights

I want to detect LED on/off status from any image which will have LED object. LED object can be of any size ( but mostly circle ). It is important to get the location of all the LEDs in that image although it can be ON or OFF. First of all I would like to get the status and position of LEDs which are ON only. Right now image source is static for my work out but it must be from video of any product having glowing LEDs. So there is no chance of having template image to substract the background.

I have tried using OpenCV (new to OpenCV) with threshold, Contours and Circles methods but not found sucessful. Please share if any source code or solution. The solution can be anything not only using OpenCV which would give result for me. It would be greatly appreciated.

The difference from other two question is that I want to get the number of LEDs in the image whether it can be ON or OFF and status of all LEDs. I know this is very complex. First of all I was trying to detect glowing LEDs in the image. I have implemented the code which i shared below. I had different implementations but below code is able to show me the glowing LEDs just by drawing the contours but number of contours are more than the glowing LEDs. So I am not able to get the total number of LEDs glowing atleast. Please suggest me your inputs.

int main(int argc, char* argv[])
{

IplImage* newImg = NULL;
IplImage* grayImg = NULL;
IplImage* contourImg = NULL;
float minAreaOfInterest = 180.0;
float maxAreaOfInterest = 220.0;
//parameters for the contour detection
CvMemStorage * storage = cvCreateMemStorage(0);
CvSeq * contours = 0;
int mode = CV_RETR_EXTERNAL;
mode = CV_RETR_CCOMP; //detect both outside and inside contour
cvNamedWindow("src", 1);
cvNamedWindow("Threshhold",1);
//load original image
newImg = cvLoadImage(argv[1], 1);

IplImage* imgHSV = cvCreateImage(cvGetSize(newImg), 8, 3);
cvCvtColor(newImg, imgHSV, CV_BGR2HSV);

cvNamedWindow("HSV",1);
cvShowImage( "HSV", imgHSV );
IplImage* imgThreshed = cvCreateImage(cvGetSize(newImg), 8, 1);

cvInRangeS(newImg, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
cvShowImage( "src", newImg );

cvShowImage( "Threshhold", imgThreshed );

//make a copy of the original image to draw the detected contour
contourImg = cvCreateImage(cvGetSize(newImg), IPL_DEPTH_8U, 3);
contourImg=cvCloneImage( newImg );
cvNamedWindow("Contour",1);
//find the contour
cvFindContours(imgThreshed, storage, &contours, sizeof(CvContour), mode, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
int i = 0;
for (; contours != 0; contours = contours->h_next)
{
    i++;
    //ext_color = CV_RGB( rand()&255, rand()&255, rand()&255 ); //randomly coloring different contours      
    cvDrawContours(contourImg, contours, CV_RGB(0, 255, 0), CV_RGB(255, 0, 0), 2, 2, 8, cvPoint(0,0));
}
printf("Total Contours:%d\n", i);
cvShowImage( "Contour", contourImg );

cvWaitKey(0);
cvDestroyWindow( "src" ); cvDestroyWindow( "Threshhold" );
cvDestroyWindow( "HSV" );
cvDestroyWindow( "Contour" );
cvReleaseImage( &newImg ); cvReleaseImage( &imgThreshed ); 
cvReleaseImage( &imgHSV );
cvReleaseImage( &contourImg );



}

Solution

I has some time yesterday night, here is a (very) simple and partial solution that works fine for me. I created a git repository that you can directly clone :

git://github.com/jlengrand/image_processing.git

and run using Python

$ cd image_processing/LedDetector/
$ python leddetector/led_highlighter.py

You can see the code here

My method :

• Convert to one channel image
• Search for brightest pixel, assuming that we have at least one LED on and a dark background as on your image
• Create a binary image with the brightest part of the image
• Extract the blobs from the image, retrieve their center and the number of leds.

The code only takes an image into account at this point, but you can enhance it with a loop to take a batch of images (I already provides some example images in my repo.) You simply have to play around a bit with the center found for LEDs, as they might not be one pixel accurate from one image to another (center could be slightly shifted).

In order to get the algorithm more robust (know whether there is a LED on or not, find an automatic and not hard coded margin value), you can play around a bit with the histogram (placed in extract_bright). I already created the function for that you should just have to enhance it a bit.

Some more information concerning the input data : Opencv does only accept avi files for now, so you will have to convert the mp4 file to avi (uncompressed in my case). I used this, that worked perfectly. For some reason, the queryframe function caused memory leaks on my computer. That is why I created the grab_images functions, that takes the avi file as input and creates a batch of jpg images that you can use easier.

Here is the result for an image :

Input image :

Binary image :

Final result :

Hope this helps. . .

EDIT :

Your problem is slightly more complex if you want to use this image. The method I posted could still be used, but needs to be a bit complexified.

You want to detect the leds that display 'an information' (status, bandwidth, . . . ) and discard the design part.

I see three simple solutions to this :

• you have a previous knowledge of the position of the leds. In this case, you can apply the very same method, but on a precise part of the whole image (using cv.SetImageROI).
• you have a previsous knowledge of the color of the leds (you can see on the image that there are two different colors). Then you can search the whole image, and then apply a color filter to restrain your choice.
• you have no previous knowledge. In this case, things get a bit more complex. I would tend to say that leds that are not useful should all have the same color, and that status leds usually blink. This means that by adding a learning step to the method, you might be able to see which leds actually have to be selected as useful.

Hope this brings some more food for thoughts

Comments

[jlengrand]

What did not work for you? No work is rather vague, explain whether you had implementation problems or why your tests dit nos succeed (at least your opinion about it). Also, what is different for you from the two other questions ?

[jkstar]

I have edited my question and added the code.

[maximus]

Can you put here what kind of images you are working with? Several examples.

[jkstar]

As I am a new user, I am not allowed to upload images. Please find the below link for a sample image. link

[jlengrand]

Are you searching for leds of only one color? Do you know this color before starting the algorithm? Do you have a batch of images from the very same place (or is your camera moving over time ? )?

[jkstar]

Thank you so much. This helped me a lot. I got few issues when i checked your code. I tried with an image in the following link The Result image shows the non glowing LED objects and other bright objects. How do we eliminate such that it detects only glowing LED?

[jkstar]

Thanks again for your response. First and second solution is not applicable in my case. In third case, can you please explain more in details. I mean I dont understand some terms. For example, what is learning step? It would be great if you could enlarge your thoughts.

[jlengrand]

Well, this basically means that if you have no previous knowledge, the only solution is to fond a clever way to 'guess' where to search for. I would tend to record images over time, and search for objects that blink. I would remove the still (in your pictures, the blue part) as theuy do not move with time, and all have the same color

[jkstar]

Thanks. You have mentioned that we need to play around with histogram to make the algorithm robust to know whether LED is ON or not. Can you please explain how it could be used here.

[jlengrand]

You can see on your image that in grayscale, you have two dominants : a very dark part, corresponding to the background of the image; and a very bright part where LEDs shine. I expect the grey histogram to thus have two spikes. Search for the highest value spike should help you find the leds; and get the algorithm more robust

[GrantU]

Link no longer works? please re-post this looks very helpful? :)

[Bob9710]

Hi. I have the same problem. Would like to detect the LED in the image. But would like to use ROS as well. So when detect the LED in the image like to publish the pixels image coordinate in ROS as a topic .. Any help The application is for Visual Servoying

[Bob9710]

So the whole code must be in ROS. I can use cv_bridge. But than how to adapt the code for ROS?