Rotate VideoCapture in OpenCV on Android

Question

How to rotate the camera when using class VideoCapture on OpenCV? (Sample Face Detection on Android). I'm rotating the canvas with:

if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT)
{
    Matrix matrix = new Matrix();
    matrix.preTranslate(
    (canvas.getWidth() - bmp.getWidth()) / 2,
    (canvas.getHeight() - bmp.getHeight()) / 2);
    matrix.postRotate(270f, (canvas.getWidth()) / 2,
    (canvas.getHeight()) / 2);
    canvas.drawBitmap(bmp, matrix, null);
}

but image from Camera doesn't rotate: Face Detect dont work.

The camera receives the stream from the following:

protected Bitmap processFrame(VideoCapture capture) {

    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);

    capture.retrieve(mGray,
    Highgui.CV_CAP_ANDROID_GREY_FRAME);

UPDATE I did the following:

@Override
    protected Bitmap processFrame(VideoCapture capture) {

    if (getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT) {
        Core.flip(mRgba.t(), mRgba, 0);
    }

    else {
    }
    capture.retrieve(mRgba, Highgui.CV_CAP_ANDROID_COLOR_FRAME_RGBA);
    capture.retrieve(mDetect_thread.mGray,
            Highgui.CV_CAP_ANDROID_GREY_FRAME);

But is dont work. When I run the program in portret orientation(on android device)- program don't start When i run the rogram in landscape orientation - programm work, but when i rotation the device, program work, but image on display dont rotation

Solution 1

Your question is mostly a duplicate of this, except that you are looking for the Android version. It is quite similar but here it is, 90º rotation can be obtained by transposing and then flipping the image:

# rotate 90º counter-clockwise
Core.flip(mRgba.t(), mRgba, 0); //mRgba.t() is the transpose

# rotate 90º clockwise
Core.flip(mRgba.t(), mRgba, 1);

For other rotations you can use warpAffine:

Point center = new Point(x,y);
double angle = 90;
double scale = 1.0;

Mat mapMatrix = Imgproc.getRotationMatrix2D(center, angle, scale);
Imgproc.warpAffine(srcMat, dstMat, mapMatrix, Imgproc.INTER_LINEAR);

EDIT - more complete examples follow:

    /** 
     * Image is first resized-to-fit the dst Mat and then rotated. 
     * mRgba is the source image, mIntermediateMat should have the same type.
     */
    private void rotationTutorial(){
        double ratio =  mRgba.height() / (double) mRgba.width();

        int rotatedHeight = mRgba.height();     
        int rotatedWidth  = (int) Math.round(mRgba.height() * ratio);

        Imgproc.resize(mRgba, mIntermediateMat, new Size(rotatedHeight, rotatedWidth));

        Core.flip(mIntermediateMat.t(), mIntermediateMat, 0);

        Mat ROI = mRgba.submat(0, mIntermediateMat.rows(), 0, mIntermediateMat.cols());

        mIntermediateMat.copyTo(ROI);       
    }


    /** 
     * Image is rotated - cropped-to-fit dst Mat.
     * 
     */
    private void rotationAffineTutorial(){
        // assuming source image's with and height are a pair value:
        int centerX = Math.round(mRgba.width()/2);
        int centerY = Math.round(mRgba.height()/2);

        Point center = new Point(centerY,centerX);
        double angle = 90;
        double scale = 1.0;

        double ratio =  mRgba.height() / (double) mRgba.width();

        int rotatedHeight = (int) Math.round(mRgba.height());       
        int rotatedWidth  = (int) Math.round(mRgba.height() * ratio);

        Mat mapMatrix = Imgproc.getRotationMatrix2D(center, angle, scale);

        Size rotatedSize = new Size(rotatedWidth, rotatedHeight);
        mIntermediateMat = new Mat(rotatedSize, mRgba.type());

        Imgproc.warpAffine(mRgba, mIntermediateMat, mapMatrix, mIntermediateMat.size(), Imgproc.INTER_LINEAR);

        Mat ROI = mRgba.submat(0, mIntermediateMat.rows(), 0, mIntermediateMat.cols());

        mIntermediateMat.copyTo(ROI);
    }

Note:

• These examples might be orientation-specific, I made them for landscape orientation.
• You should not call the code from these examples for every video frame. Some of the code should only run once.

Solution 2

If you only need to do 90, 180, or 270 degrees rotation (Which seems to be your case) you better use Core.flip() which is faster. Here below a method that does it for you:

public static Mat rotate(Mat src, double angle)
{
    Mat dst = new Mat();
    if(angle == 180 || angle == -180) {
        Core.flip(src, dst, -1);
    } else if(angle == 90 || angle == -270) {
        Core.flip(src.t(), dst, 1);
    } else if(angle == 270 || angle == -90) {
        Core.flip(src.t(), dst, 0);
    }

    return dst;
}

Comments

[Rui Marques]

Off course it works. It is face recognition that does not work out of the box with rotated images.

[Rui Marques]

@gregm I have fixed a few typos with the original post and presented some more complete examples. If you didn't manage to get it to work before, you probably were unable to properly output the resulting image. These examples show how to output it (but can be improved efficiency-wise).

[Jenia Ivanov]

@RuiMarques Hi. The FeatureDetector is now 90 degree off though. How do I fix that?