How to draw on MTLTexture in bgra8Unorm pixel format

Question

My code works when I draw on MTLTexture with rgba32Float pixel format, I can take then CVPixelBuffer out of it.

But FlutterTexture requires bgra8Unorm format. I do not want to convert CVPixelBuffer due to performance overhead.

So I'm trying to render on MTLTexture with bgra8Unorm pixel format, but the following fragment shader code won't compile:

fragment vector_uchar4 fragmentShader2(Vertex interpolated [[stage_in]]) {
    return 0xFFFFFFFF;
}

With error: Invalid return type 'vector_uchar4' for fragment function I've tried to replace it with uint type, but it crashes with error:

Fatal error: 'try!' expression unexpectedly raised an error:
Error Domain=AGXMetalA11 Code=3 
"output of type uint is not compatible with a MTLPixelFormatBGRA8Unorm color attachement."
UserInfo={NSLocalizedDescription=output of type uint is not compatible with a MTLPixelFormatBGRA8Unorm color attachement.}

If I use vector_float4 or vector_half4 return type my texture and buffers are empty.

Which return type I have to use for bgra8Unorm pixel format and get non empty image? Is it possible with metal at all?

Solution 1

I've found answer on page 30 of Metal Shading Language specification

And finally this code draws image as expected:

fragment float4 fragmentShader2(Vertex interpolated [[stage_in]]) {
    // ...
    rgba8unorm<float4> rgba;
    rgba = float4(color.r, color.g, color.b, 1.0);
    return rgba;
}

If someone can explain what is happening under the hood, I would really like to not waste bounty.

Solution 2

It depends on many different factors. In most cases you should use float4 or half4.

All modern apple GPUs that support metal designed to perform calculation on ( 32-bit or 64-bit) floating point data. It's how GPUs works, this means that any read operation calculated by the shader on the Float, Snorm, Unorm formats will be performed on 32-bit or 64-bit floating point, regardless of the original input format. On any writing operation shader performs conversion from 32-bit or 64-bit floating point to target format.
For conversion rules please see Metal Shading Language specification page 217.

Any metal formats that use the Float, Snorm, Unorm suffix are floating-point formats, while Uint and Sint are unsigned and signed integer.

Float - A floating-point value in any of the representations defined by metal.
Unorm - A floating-point value in range [0.0, 1.0].
Snorm - A floating-point value in range [-1.0, 1.0].
Uint - A unsigned integer.
Sint - A signed integer.