How to get luminosity of Mat image from opencv?

Question

I'm doing an image processing on iOS using OpenCV in c++. Sometime the ambient light is not enough and the image cannot be processed properly. I did not found any suitable way to detect the ambient light so trying to detect the luminosity of the image instead.

This answer suggests to use the luma value or Y in YUV format. So I follow this answer to access the pixel in formation from Mat image after convert the image to YUV format.

- (void)processImage:(Mat&)image
{    
    Mat frame_yuv;
    cvtColor( image, frame_yuv, CV_BGR2YUV );

    uint8_t* pixelPtr = (uint8_t*)frame_yuv.data;
    int cn = frame_yuv.channels();
    for(int i = 0; i < frame_yuv.rows; i+=50)
    {
        for(int j = 0; j < frame_yuv.cols; j += 50)
        {
            Scalar_<uint8_t> yuvPixel;
            yuvPixel.val[0] = pixelPtr[i*frame_yuv.cols*cn + j*cn + 0]; // Y
            yuvPixel.val[1] = pixelPtr[i*frame_yuv.cols*cn + j*cn + 1]; // U
            yuvPixel.val[2] = pixelPtr[i*frame_yuv.cols*cn + j*cn + 2]; // V

            // do something with YUV values...
            std::cout << "Y: " << yuvPixel.val[0] << endl;
            std::cout << "U: " << yuvPixel.val[1] << endl;
            std::cout << "V: " << yuvPixel.val[2] << endl;
        }
    }
}

Here are some lines of log which contains some weird results to me and the values of "Y", "U", or "V" should not be this much in my understanding. Please suggest me what I missed or what did wrong? Thank you very much.

Y: P
U: \204
V: \206
Y: Q
U: \201
V: \207
Y: K
U: \205
V: \211
Y: H
U: \203
V: \210
Y: G
U: \202
V: \210
Y: H
U: \201
V: \210
Y: H
U: \202
V: \211
Y: \326
U: \200
V: \204
Y: \377
U: \200
V: \200
Y: \377
U: \200
V: \200
Y: \377
U: \200
V: \200
Y: \376
U: |
V: \201
Y: \313
U: x
V: \210
Y: \231
U: ~
V: \204
Y: \214
U: ~
V: \204
Y: \205
U: |
V: \204
Y: \221
U: 
V: \202

Solution 1

The reason for your log output is that your pixel values are of type uint8_t, which will be interpreted as unsigned char (on iOS, at least) when you try to output it to the console. So the log output is the character represented by that pixel value. To output the corresponding numerical value, cast the value to an int, like so:

std::cout << "Y: " << (int)yuvPixel.val[0] << std::endl;
std::cout << "U: " << (int)yuvPixel.val[1] << std::endl;
std::cout << "V: " << (int)yuvPixel.val[2] << std::endl;

Also, your posted code was outputting the same element three times, and bgrPixel was undefined. I have corrected those errors in this snippet as well.

If you are only interested in the luminance information, you could instead call

cv::cvtColor(image, frame_yuv, CV_BGR2GRAY);

Which does the same color conversion as the Y channel of YUV, but will be more efficient.

Solution 2

Here's how you can do it pretty simply, works for me!

Also posted here on PasteBin: https://pastebin.com/tA1R8Qtm

// -----------------------------------------------------
// ---------- FIND AVG LUMINENCE OF FRAME --------------
// -----------------------------------------------------

// Assuming you have a cv::Mat to start with called "input_mat"
cv::Mat grayMat; 
cv::cvtColor(input_mat, grayMat, CV_BGR2GRAY);

int Totalintensity = 0;
for (int i=0; i < grayMat.rows; ++i){
    for (int j=0; j < grayMat.cols; ++j){
        Totalintensity += (int)grayMat.at<uchar>(i, j);
    }
}

// Find avg lum of frame
float avgLum = 0;
avgLum = Totalintensity/(grayMat.rows * grayMat.cols);

Comments

[Protocole]

Sorry for my mistake, I corrected it. Thank you for your suggestions. One more thing, do you have any idea about the suitable number of pixels to be calculated in rows for(int i = 0; i < frame_yuv.rows; i+=50) and columns for(int j = 0; j < frame_yuv.cols; j += 50).

[Aurelius]

I'm afraid I don't quite understand what you are asking here. And why is it that you are incrementing by 50 each time?

[Protocole]

I mean calculating on every pixels causes bad performance so I tried to sample just 1 pixel per 50 pixels to calculate to not affect much on performance but still have enough accuracy. I'm looking for this optimal number.

[Aurelius]

I don't think there's a canonical way to do this. You could try different sampling steps and compare them against the true average (or median) and then choose the largest step that results in an acceptable error.

[Brad Larson]

@Protocole - If I may suggest a much faster way of calculating average luminance, I have a GPU-accelerated approach that I describe in this answer: stackoverflow.com/a/14612913/19679

[Alexander Belokon]

This should be the accepted answer, that is actually the only clear implementation of luminosity calculation with C++ and Opencv I could find.