Java OpenCV - Rectangle Detection with Hough Transform

Question

I'm developing a program to detect rectangular shape and draw bounding box to the detected area.

For edge detection, I used Canny Edge Detection. Then, I use Hough Transform to extract lines.

This is the original image enter image description here

This is the result image enter image description here

My problem is that I can't draw a bounding box to the detected area. It seems that my program can only detect a single horizontal line. How can I detect rectangle shape and draw rectangle line to the detected shape?

I've read similar problems and it is required to find the 4 corner points of the rectangle, check if the point is 90 degree, and find the intersection. I'm really confused how to code it in Java opencv. Other methods to detect the shape and draw bounding box to the detected would be okay too.

Here's the code

import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.core.Point;
import org.opencv.core.Scalar;
import org.opencv.core.Size;
import org.opencv.imgcodecs.*;
import org.opencv.imgproc.Imgproc;

public class HoughTransformCV2 {

    public static void main(String[] args) {
        try {
            System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
            Mat source = Imgcodecs.imread("rectangle.jpg", Imgcodecs.CV_LOAD_IMAGE_ANYCOLOR);
            Mat destination = new Mat(source.rows(), source.cols(), source.type());

            Imgproc.cvtColor(source, destination, Imgproc.COLOR_RGB2GRAY);
            Imgproc.equalizeHist(destination, destination);
            Imgproc.GaussianBlur(destination, destination, new Size(5, 5), 0, 0, Core.BORDER_DEFAULT);

            Imgproc.Canny(destination, destination, 50, 100);
            //Imgproc.adaptiveThreshold(destination, destination, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 15, 40);
            Imgproc.threshold(destination, destination, 0, 255, Imgproc.THRESH_BINARY);

            if (destination != null) {
                Mat lines = new Mat();
                Imgproc.HoughLinesP(destination, lines, 1, Math.PI / 180, 50, 30, 10);
                Mat houghLines = new Mat();
                houghLines.create(destination.rows(), destination.cols(), CvType.CV_8UC1);

                //Drawing lines on the image
                for (int i = 0; i < lines.cols(); i++) {
                    double[] points = lines.get(0, i);
                    double x1, y1, x2, y2;
                    x1 = points[0];
                    y1 = points[1];
                    x2 = points[2];
                    y2 = points[3];

                    Point pt1 = new Point(x1, y1);
                    Point pt2 = new Point(x2, y2);

                    //Drawing lines on an image
                    Imgproc.line(source, pt1, pt2, new Scalar(0, 0, 255), 4);
                }

            }

            Imgcodecs.imwrite("rectangle_houghtransform.jpg", source);

        } catch (Exception e) {
            System.out.println("error: " + e.getMessage());
        }
    }
}

Any help in Java would be appreciated :) Thank you very much!

Solution

You can do this in these steps:

1. After you did the converting color to gray, perform a canny edge.

   int threshold = 100;

   Imgproc.Canny(grayImage, edges, threshold, threshold*3);

2. Now find the contours in edge image.

Imgproc.findContours(edges, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);

3. Then loop on all over the contours

.....

MatOfPoint2f matOfPoint2f = new MatOfPoint2f();
MatOfPoint2f approxCurve = new MatOfPoint2f();

for (int idx = 0; idx >= 0; idx = (int) hierarchy.get(0, idx)[0]) {
    MatOfPoint contour = contours.get(idx);
    Rect rect = Imgproc.boundingRect(contour);
    double contourArea = Imgproc.contourArea(contour);
    matOfPoint2f.fromList(contour.toList());
    Imgproc.approxPolyDP(matOfPoint2f, approxCurve, Imgproc.arcLength(matOfPoint2f, true) * 0.02, true);
    long total = approxCurve.total();
    if (total == 3) { // is triangle
        // do things for triangle
    }
    if (total >= 4 && total <= 6) {
        List<Double> cos = new ArrayList<>();
        Point[] points = approxCurve.toArray();
        for (int j = 2; j < total + 1; j++) {
            cos.add(angle(points[(int) (j % total)], points[j - 2], points[j - 1]));
        }
        Collections.sort(cos);
        Double minCos = cos.get(0);
        Double maxCos = cos.get(cos.size() - 1);
        boolean isRect = total == 4 && minCos >= -0.1 && maxCos <= 0.3;
        boolean isPolygon = (total == 5 && minCos >= -0.34 && maxCos <= -0.27) || (total == 6 && minCos >= -0.55 && maxCos <= -0.45);
        if (isRect) {
            double ratio = Math.abs(1 - (double) rect.width / rect.height);
            drawText(rect.tl(), ratio <= 0.02 ? "SQU" : "RECT");
        }
        if (isPolygon) {
            drawText(rect.tl(), "Polygon");
        }
    }
}

Helper methods:

private double angle(Point pt1, Point pt2, Point pt0) {
    double dx1 = pt1.x - pt0.x;
    double dy1 = pt1.y - pt0.y;
    double dx2 = pt2.x - pt0.x;
    double dy2 = pt2.y - pt0.y;
    return (dx1*dx2 + dy1*dy2)/Math.sqrt((dx1*dx1 + dy1*dy1)*(dx2*dx2 + dy2*dy2) + 1e-10);
}

private void drawText(Point ofs, String text) {
    Imgproc.putText(colorImage, text, ofs, Core.FONT_HERSHEY_SIMPLEX, 0.5, new Scalar(255,255,25));
}

Hope this can help you!!

Comments

[Jonathan Barbero]

@Bahrandum Adil would you clarify the methods "angle", "drawText" and the "triangle" in the if condition? I'm new with opencv and I don't realize if these are static methods in the library or functions you implemented by your own. What 'threshold' do you use for Imgproc.Canny ?

[Bahramdun Adil]

@JonathanBarbero Hi!. Most of the OpenCV functions are static. like the functions of Imgproc or Core, you can directly use these, just with Class name dot the functions name. And also the answer has been updated, you can have a look to the answer again. Good Luck!