Apply a 4x5 Matrix (instead of 4x4) to Transform in OpenCV (C++)

Question

On Flutter side I use a 4x5 Matrix to apply a Color Filter to an image (for preview). Via FFI, I then apply the filter with OpenCV and transform on the image and save the new image result.
However, the result is always different to the preview.

So I was wondering, how can I achieve the same result with the same Matrix (4x5) with OpenCV?

Flutter - ColorFilter Matrix (RGBA) for ColorFiltered Widget:

const ColorFilter _FILTER_2 = ColorFilter.matrix(<double>[
  0.9,  0.11, 0.11, 0.0,  0.0, //
  0.11, 0.7,  0.44, 0.0,  0.0, //
  0.22, 0.22, 0.9,  0.0,  0.0, //
  0.0,  0.0,  0.0,  1.0,  0.0
]);

C++ - transform function with matrix (BGRA):

__attribute__((visibility("default"))) __attribute__((used)) void filter_2(char *inputImagePath, char *outputImagePath)
{
    Mat input = imread(inputImagePath);
    Mat output = input.clone();

    Mat filter = (Mat_<float>(4, 4) <<
                  0.22, 0.22, 0.9,  0.0,
                  0.11, 0.7,  0.44, 0.0,
                  0.9,  0.11, 0.11, 0.0,
                  0.0,  0.0,  0.0,  1.0);

    transform(output, input, filter);
    imwrite(outputImagePath, input);
}

Results

Solution

The most common channel order of color images is RGB. OpenCV's native channel order is BGR (the reverse) for historical reasons.

// maps RGB to RGB
Mat filter = (Mat_<float>(4, 4) <<
    0.9,  0.11, 0.11, 0.0
    0.11, 0.7,  0.44, 0.0
    0.22, 0.22, 0.9,  0.0
    0.0,  0.0,  0.0,  1.0);

Your color matrix for mapping RGB to RGB (with some effect) has certain values in the top left 3x3 part, which combine the RGB values in new ways.

To account for the RGB/BGR difference in the output data, I see that you permuted rows of your mixing matrix. That matrix would correctly map RGB input to BGR output.

// maps RGB to BGR (rows permuted)
Mat filter = (Mat_<float>(4, 4) <<
    0.22, 0.22, 0.9,  0.0
    0.11, 0.7,  0.44, 0.0
    0.9,  0.11, 0.11, 0.0
    0.0,  0.0,  0.0,  1.0);

You could use that matrix, but use cvtColor on the input (with COLOR_BGR2RGB), to make it suitable for that matrix.

Or you could also permute the columns of the matrix, so its input is assumed to be BGR:

// maps BGR to BGR (columns and rows permuted)
Mat filter = (Mat_<float>(4, 4) <<
    0.9,  0.22, 0.22, 0.0,
    0.44, 0.7,  0.11, 0.0,
    0.11, 0.11, 0.9,  0.0,
    0.0,  0.0,  0.0,  1.0);

In the application of this matrix, an input pixel's color channels are taken as a column vector (and a 1 is appended to enable affine transformations). This vector is then multiplied as M v, so the coefficients in the first column are multiplied by the first color component (for each output component), and the first row mixes all components of the input pixel into the first component of the output pixel.

Comments

[Dan Mašek]

A necessary part of a minimal reproducible example, which is required for these sort of questions, is a pristine input image. Provide it.

[Christoph Rackwitz]

since your 4x5 matrix's last column is all 0, what's the point? it'll ignore the 5th value of the input. in fact, even the 4x4 matrix has nothing interesting in the 4th row and column (that would be the affine component), so you can just reduce this to a 3x3 matrix for transform. you should explain what those 4/5 values are in the input and output. opencv works with BGR data, so you should permute (reverse) your matrix's top left 3x3 values not just vertically but also horizontally. then you can have BGR input and BGR output.

[Christoph Rackwitz]

aha! so it's RGBA and an affine component: kindacode.com/article/flutter-colorfiltered yeah, your matrix has no affine values, and unless you use imread with IMREAD_UNCHANGED, you won't get an alpha channel either (and only if the file itself has an alpha channel!), so just reduce this to 3x3.

[goldensoju]

@ChristophRackwitz "Opencv works with BGR data, so you should permute (reverse) your matrix's top left 3x3 values not just vertically but also horizontally." I just realized that I did reverse the values only vertically. After correctly permuting the matrix (vertically and horizontally) I receive now the correct results. Thank you very much for your help and editing! You can post it as answer if you'd like.