OpenCV image comparison in Android

Question

[EDIT] I have devised some code for image comparison. The matching part is still a bit flawed and I would love some assitance. The project can be found at - GitHub.

I have these two images Img1 and Img2:

When I use the following command in openCV

Mat img1 = Highgui.imread("mnt/sdcard/IMG-20121228.jpg");
Mat img2 = Highgui.imread("mnt/sdcard/IMG-20121228-1.jpg");

try{
    double l2_norm = Core.norm( img1, img2 );
    tv.setText(l2_norm+"");
} catch(Exception e) {
    //image is not a duplicate
}

I get a double value for l2_norm. This double value varies for duplicate image pairs. But if the images are different, then an exception is thrown. Is this how I identify duplicate images? Or is there a better method? I've Googled extensively and couldn't find a really convincing answer. I would like the code and explanation as to how I'd compare two images and get a boolean value of true or false depending upon the images.

EDIT

Scalar blah= Core.sumElems(img2);
    Scalar blah1=Core.sumElems(img1);

    if(blah.equals(blah1))
    {
        tv.setText("same image");
    }
    }

I've tried this, but the if condition is never satisfied. I'm assuming there are a few differences, but there is no compare function for Scalar. What do I do?

EDIT

try{
    Scalar blah= Core.sumElems(img2);
    Scalar blah1=Core.sumElems(img1);
    String b=blah.toString();
    String b1=blah1.toString();
    System.out.println(b+" "+b1);
    double comp=b.compareTo(b1);
    tv.setText(""+comp);
    }

This method is again flawed. Although it can be used to compare images with a decent accuracy, it fails when images are of different sizes.

When images are of different sizes and I print the scalar values I get this:

[9768383.0, 1.0052889E7, 1.0381814E7, 0.0] [1.5897384E7, 1.6322252E7, 1.690251E7, 0.0]

The variation between the second and third numbers although not much is quite large compared to when the images of same size are compared. The first number however suffers the most change.

What would be the best fastest way to compare the contents of two images?

[EDIT]

I'm using the code I found here.

What I'm not able to figure out is how to initialize the MatOfKeyPoint variables keypoints and logoKeypoints. Here's my code snippet:

           FeatureDetector detector = FeatureDetector.create(FeatureDetector.SURF);
        //FeatureDetector detector = FeatureDetector.create(FeatureDetector.FAST);
        //Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGBA2RGB);
        //Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGBA2RGB);

        DescriptorExtractor SurfExtractor = DescriptorExtractor
        .create(DescriptorExtractor.SURF);


        //extract keypoints
        MatOfKeyPoint keypoints, logoKeypoints;
        long time= System.currentTimeMillis();
        detector.detect(img1, keypoints);
        Log.d("LOG!", "number of query Keypoints= " + keypoints.size());
        detector.detect(img2, logoKeypoints);
        Log.d("LOG!", "number of logo Keypoints= " + logoKeypoints.size());
        Log.d("LOG!", "keypoint calculation time elapsed" + (System.currentTimeMillis() -time));

        //Descript keypoints
        long time2 = System.currentTimeMillis();
        Mat descriptors = new Mat();
        Mat logoDescriptors = new Mat();
        Log.d("LOG!", "logo type" + img2.type() + "  intype" + img1.type());
        SurfExtractor.compute(img1, keypoints, descriptors);
        SurfExtractor.compute(img2, logoKeypoints, logoDescriptors);
        Log.d("LOG!", "Description time elapsed" + (System.currentTimeMillis()- time2));

I obviously can't initialize the variables keypoints and logoKeypoints to null cuz I'll receive a null pointer exception then. How do I initialize them?

Solution 1

You should understand that this is not a simple question and you have different concepts you could follow. I will only point out two solution without source-code.

1. Histogram comparison: You could convert both images into grey-scale make a histogram in the range of [0,...,255]. Every pixel-value will be counted. Then use both histograms for comparison. If the distribution of pixel-intensities equals or is above some treshold (perhaps 90% of all pixels), you could consider this images as duplicates. BUT: This is one of the simplest solutions and it isn't stable if any picture has an equal distribution.
2. Interest-Point-Detectors/-Descriptors: Take a look at SIFT/SURF image-detectors and descriptors. A detector will try to determine unique keypoits of intensities in an image. A descriptor will be computed at this location I(x,y). A normal matcher with a bruteforce-approach and euclidean distance can match these images using their descriptors. If an image is a duplicate the rate of given matches should very high. This solution is good to implement and there could be enough tutorials regarding this topic.

I'll hope this helps. Please ask if you have questions.

[UPDATE-1] A C++-tutorial: http://morf.lv/modules.php?name=tutorials&lasit=2#.UR-ewKU3vCk

Some JavaCV-tutorials: http://code.google.com/p/javacv/w/list

[UPDATE-2] Here is an example with SIFT-Detector and SIFT-Descriptor using default parameters. RANSAC-Threshold for homography is 65, reprojection-error (epsilon) is 10, cross-validation enabled. You could try to count the matched. If the Inliner-Outlier-Ratio is too high you could see this pair as duplicates. For example: These images produce 180 keypoints in IMG1 and 198 in IMG2. The matched descriptors are 163 of which only 3 are outliers. So this gives a really good ratio which only could mean that these images could be duplicates.

[UPDATE-3] I don't understand why you can initialize the MatOfKeypoints. I've read the API and there's a public constructor. AND: You can use the Mat of the image you want to analyse. This is very nice. =)

MatOfKeyPoint reference = new MatOfKeyPoint(matOfReferenceImage);

For Matching use a BRUTEFORCE_SL2 Descriptor-Matcher cause you will need the euclidean distance for SURF or SIFT.

Solution 2

Use cv2.absDiff to compute the difference between the pictures and cv2.sumElems to get the sum of all pixels differences.

Then invent a threshold by which you judge wether two images are similar or not.

Comments

[harism]

This OpenCV tutorial hopefully provides some information on the topic; goo.gl/gwN6e .

[sschrass]

try-catch is not the same as if-else! If an exception was thrown (catch block), something went totally wrong!

[Karthik Balakrishnan]

@SatelliteSD - I know that. That's why I'm asking if there's a better method.

[Emile]

I don't believe you need to, MatOfKeyPoint keypoints, logoKeypoints; is fine. The method detector.detect(img1, keypoints); will then populate your keypoints with the found features. Otherwise try MatOfKeyPoint keypoints= new MatOfKeyPoint();

[Karthik Balakrishnan]

@Emile Could you put up the correct code for image keypoint comparison, please?

[Emile]

I haven't tried this myself, however i would start with this tutorial, docs.opencv.org/doc/tutorials/features2d/feature_description‌​/… It has a slightly different implementation and initiates keypoints as std::vector<KeyPoint> keypoints_1, keypoints_2; It might be that for Android this is different. Afraid thats all i know.

[Karthik Balakrishnan]

My knowledge in C++ is quite poor. Could you port it to JAVA, please?

[Mr.Mountain]

Try to instantiate the MatOfKeyPoint with the Image-Matrix.

[Rui Marques]

Do you still need assistance, what is the problem?

[Karthik Balakrishnan]

Could you elucidate? I'd have to set a threshold for absDiff. So, that's flawed. Or perhaps you could tell me what the safest threshold value is?

[sschrass]

first, sorry you will have to use cv.sum since sumelems seems to be python only. Second, you need the threshhold for the return value from sum! You could wrap it in a method that returns true or false based on the return value of sum and your threshold.

[sschrass]

there you will have to try fo yourself. Since sum returns (for each channel ARGB I believe) a value that represents the deviance from 2 images (calculated by absDiff), 0 would be a perfect match.

[Karthik Balakrishnan]

Could please put in the exact syntax for it? Sorry to be a bother. It's hard finding the correct syntax when you're just new to OpenCV.

[sschrass]

opencv.willowgarage.com/documentation/cpp/… and docs.opencv.org/modules/core/doc/operations_on_arrays.html#s‌​um

[Karthik Balakrishnan]

Could you write the code of sumElems for the img1 and img2 in my code? I've written Scalar blah= Core.sumElems(img2); which is obviously incorrect. Could you create a function which returns the sum value, please?

[Karthik Balakrishnan]

Could you add links to tutorials, please?

[Mr.Mountain]

I added some additional material.