Rotate cv::Mat using cv::warpAffine offsets destination image

Question

I'm trying to rotate a 1296x968 image by 90 degrees using the C++ API of OpenCV and I'm facing a few problems.

Input:

Rotated:

As you can see, the rotated image has a few problems. First, it has the same size of the original, even though I specifically create the destination Mat with the inverted size of the original. As a result, the destination image gets cropped.

I suspect this is happening because I'm calling warpAffine() and passing the size of the original Mat instead of the size of destination Mat. But I'm doing this because I followed this answer, but now I suspect that the answer may be wrong. So this is my first doubt/problem.

The second, is that warpAffine() is writing to the destination at a certain offset (probably to copy the rotated data to the middle of the image) and this operation leaves a horrible and large black border around the image.

How do I fix these issues?

I'm sharing the source code below:

#include <cv.h>
#include <highgui.h>
#include <iostream>

using namespace cv;
using namespace std;

void rotate(Mat& image, double angle)
{
    Point2f src_center(image.cols/2.0F, image.rows/2.0F);

    Mat rot_matrix = getRotationMatrix2D(src_center, angle, 1.0);

    Mat rotated_img(Size(image.size().height, image.size().width), image.type());

    warpAffine(image, rotated_img, rot_matrix, image.size());
    imwrite("rotated.jpg", rotated_img);
}

int main(int argc, char* argv[])
{
    Mat orig_image = imread(argv[1], 1);
    if (orig_image.empty())
    {
        cout << "!!! Couldn't load " << argv[1] << endl;
        return -1;
    }

    rotate(orig_image, 90);

    return 0;
}

Solution 1

I've found a solution that doesn't involve warpAffine().

But before that, I need to state (for future references) that my suspicion was right, you needed to pass the size of the destination when calling warpAffine():

warpAffine(image, rotated_img, rot_matrix, rotated_img.size());

As far as I can tell, the black border (caused by writing at an offset) drawed by this function seems to be it's standard behavior. I've noticed this with the C interface and also with the C++ interface of OpenCV running on Mac and Linux, using the versions 2.3.1a and 2.3.0.

The solution I ended up using is much simpler than all this warp thing. You can use cv::transpose() and cv::flip() to rotate an image by 90 degrees. Here it is:

Mat src = imread(argv[1], 1);

cv::Mat dst;
cv::transpose(src, dst);
cv::flip(dst, dst, 1);

imwrite("rotated90.jpg", dst);

----I>

Solution 2

A lot of people have had problems with rotating images or image chunks due to offsets etc. So, I'm posting a solution to allow you to rotate a region (or whole) of an image and stick it into another image or have the function compute an image where everything will just fit.

// ROTATE p by R
/**
 * Rotate p according to rotation matrix (from getRotationMatrix2D()) R
 * @param R     Rotation matrix from getRotationMatrix2D()
 * @param p     Point2f to rotate
 * @return      Returns rotated coordinates in a Point2f
 */
Point2f rotPoint(const Mat &R, const Point2f &p)
{
    Point2f rp;
    rp.x = (float)(R.at<double>(0,0)*p.x + R.at<double>(0,1)*p.y + R.at<double>(0,2));
    rp.y = (float)(R.at<double>(1,0)*p.x + R.at<double>(1,1)*p.y + R.at<double>(1,2));
    return rp;
}

//COMPUTE THE SIZE NEEDED TO LOSSLESSLY STORE A ROTATED IMAGE
/**
 * Return the size needed to contain bounding box bb when rotated by R
 * @param R     Rotation matrix from getRotationMatrix2D()
 * @param bb    bounding box rectangle to be rotated by R
 * @return      Size of image(width,height) that will compleley contain bb when rotated by R
 */
Size rotatedImageBB(const Mat &R, const Rect &bb)
{
    //Rotate the rectangle coordinates
    vector<Point2f> rp;
    rp.push_back(rotPoint(R,Point2f(bb.x,bb.y)));
    rp.push_back(rotPoint(R,Point2f(bb.x + bb.width,bb.y)));
    rp.push_back(rotPoint(R,Point2f(bb.x + bb.width,bb.y+bb.height)));
    rp.push_back(rotPoint(R,Point2f(bb.x,bb.y+bb.height)));
    //Find float bounding box r
    float x = rp[0].x;
    float y = rp[0].y;
    float left = x, right = x, up = y, down = y;
    for(int i = 1; i<4; ++i)
    {
        x = rp[i].x;
        y = rp[i].y;
        if(left > x) left = x;
        if(right < x) right = x;
        if(up > y) up = y;
        if(down < y) down = y;
    }
    int w = (int)(right - left + 0.5);
    int h = (int)(down - up + 0.5);
    return Size(w,h);
}

/**
 * Rotate region "fromroi" in image "fromI" a total of "angle" degrees and put it in "toI" if toI exists.
 * If toI doesn't exist, create it such that it will hold the entire rotated region. Return toI, rotated imge
 *   This will put the rotated fromroi piece of fromI into the toI image
 *
 * @param fromI     Input image to be rotated
 * @param toI       Output image if provided, (else if &toI = 0, it will create a Mat fill it with the rotated image roi, and return it).
 * @param fromroi   roi region in fromI to be rotated.
 * @param angle     Angle in degrees to rotate
 * @return          Rotated image (you can ignore if you passed in toI
 */
Mat rotateImage(const Mat &fromI, Mat *toI, const Rect &fromroi, double angle)
{
    //CHECK STUFF
    // you should protect against bad parameters here ... omitted ...

    //MAKE OR GET THE "toI" MATRIX
    Point2f cx((float)fromroi.x + (float)fromroi.width/2.0,fromroi.y +
               (float)fromroi.height/2.0);
    Mat R = getRotationMatrix2D(cx,angle,1);
    Mat rotI;
    if(toI)
        rotI = *toI;
    else
    {
        Size rs = rotatedImageBB(R, fromroi);
        rotI.create(rs,fromI.type());
    }

    //ADJUST FOR SHIFTS
    double wdiff = (double)((cx.x - rotI.cols/2.0));
    double hdiff = (double)((cx.y - rotI.rows/2.0));
    R.at<double>(0,2) -= wdiff; //Adjust the rotation point to the middle of the dst image
    R.at<double>(1,2) -= hdiff;

    //ROTATE
    warpAffine(fromI, rotI, R, rotI.size(), INTER_CUBIC, BORDER_CONSTANT, Scalar::all(0)); 

    //& OUT
    return(rotI);
}

Solution 3

I realize you've found other faster solutions (90 degree rotation should be very fast, and doesn't need all the machinery of warpAffine), but I want to address the black border problem for anyone else who runs across this.

What else could warpAffine do? The destination image was specified to be wider than it was tall, and the affine transform only specified rotation (around the center of the image), not scaling. That is exactly what it did. There is no information anywhere to tell warpAffine what should be drawn in those black borders, so it left them black.

Direct physical experiment: Put a sheet down on the table. Draw an outline around it (this is what you did when you specified that you wanted the result to be the same shape/size as the original). Now rotate that sheet 90 degrees around its center. Look at the region bound by the outline on the table. If it was a black table, it would look exactly like your result.

Solution 4

Maybe this can help someone.
variables are
img : original image
angle : degrees
scale
dst : destination image

int width = img.size().width, 
    height = img.size().height;
Mat rot = getRotationMatrix2D(Point2f(0,0), angle, scale)/scale; //scale later
double sinv = rot.at<double>(0,1),
       cosv = rot.at<double>(0,0);
rot.at<double>(1,2) = width*sinv;  //adjust row offset
Size dstSize(width*cosv + height*sinv, width*sinv + height*cosv);
Mat dst;
warpAffine(img, dst, rot, dstSize);
resize(dst, dst, Size(), scale, scale);  //scale now

Solution 5

One issue I found, is that you destination image size for warpAffine is is set to image.size() instead of the rotated_img.size(). However, after the warp, it is still translated too far to in x and y...I tried the exact same warp

[ 6.123031769111886e-17 1                     163.9999999999999;
 -1                     6.123031769111886e-17 1132;
  0                     0                     1]

from OpenCV's getRotationMatrix2D in Matlab, and it worked perfectly. I'm starting to smell a possible bug with warpAffine...

View more solutions

Comments

[karlphillip]

cvWarpAffine() behaves the same on OpenCV 2.3.1a

[mevatron]

I was using ~2 week old code from the SVN trunk, and the issue is there as well. I'll take a look at the warpAffine code tonight to see if there is something fishy going on.

[karlphillip]

From what I've seeing on other places this seems to be the standard behavior for this function, but I'm not 100%.

[karlphillip]

Furthermore, OpenCV 2.3.0 and 2.3.1a present the same behavior.

[karlphillip]

I up-voted your answer. Thanks for all the help! I ended up using an alternative to achieve this effect that requires less lines of code.

[mevatron]

Thanks, and no problem at all! I'm still not convinced that just because it's been that way for a while means that it is bug free :) I will dig into a bit more, and post back if I find anything interesting.

[Ela782]

You can specify the behaviour at the borders via various cv::BORDER_* flags.

[Qadir Hussain]

@karlphillip its for square matrix. what about the rectangular matrix. How to effectively rotate the the rectangular matrix?