FFmpeg: Applying effects to a video

Question

I'm trying to apply some effects to a video by adjusting colour, contrast, brightness, etc. Here are some of the effects I was trying. And following is the command I used to apply the "Kelvin" effect.

ffmpeg -i 3.mp4 -c:v libx264 -c:a libfaac -filter_complex "[0:v]eq=1.0:0:1.3:2.4:0.175686275:0.103529412:0.031372549:0.4[outv]" -map [outv] out.mp4

FFmpeg documentation helped me to figure out the boundaries for each value for eq filter. But still I'm not getting the expected output for "Kelvin" effect as in the link above. I calculated the rg, gg, bg values according to the values for sepia value used in css. (r=112, g=66, b=20 are the values I used for converting sepia to rgb) Following is the simple math I used to get the rg, gg, bg values.

rg = (112 / 255) * (gamma_r max - gamma_r min) * sepia value

Here gamma_r max is 10 and gamma_r min is 0.1 according to the documentation.

What am I doing wrong here? Is there any other alternative to get this done?

Solution

The eq filter adjusts the color channels relative to their current state, that is, increasing or decreasing their intensity (like a "volume" knob in sound). To achieve your target color temperature with eq you'd have to calculate the current color temperature of each region in the photo and then modify it - something for which you need a frame server such as AviSynth. But all this is not really needed, because you don't really want to fully control your white balance, you just want to apply a fixed filter that will result in a specific color tone - in this case sepia. For that you should use instead the colorchannelmixer filter which provides a way to manage the white balance of the pixels relative to each other. The filter documentation has a specific example for sepia:

colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131

How it works (very simplified explanation)

The colorchannelmixer filter describes the image as if it has 4 color channels - called Red, Green, Blue, and Alpha (the "mask" channel). By default, every channel represents the intensity of the color after which it is named as 1.0. So, the "Red" channel represents the portion of the image which is red times 1.0, the portion which is green times 0.0, the portion which is blue as 0.0., and the portion which is alpha times 0.0. Likewise, each other channel has a value of 1.0 for "its" color and 0.0 for all others. Now the filter enables you to "steal" a color from its channel and inject its "energy" to another channel. For example, you could increase the value of red in the "Green" channel to 1.0, and change the value of red in the "Red" channel to 0.0, so now the intensity of green will increase in every pixel by the original intensity of red, without leaving any red at all in the image, and without changing the relative intensity of blue and alpha. When applying this concept on all 16 color combinations you get a matrix that defines the resulting color intensity of each pixel as a product of all the original color intensities. The result is then normalized, and you get a simple way of expressing the color transformation curve.

The above is somewhat simplistic, and to understand how it applies to your specific question you can read more about Color Temperature theory. Note though that with the colorchannelmixer filter you can do also other stuff such as generate color negatives, reduce color depths to achieve comics-like effects, and much more.

Hope this works well for you!

Comments

[Chamath]

can you please provide little bit more information about colorchannelmixer filter please. Documentation it self seems not contain much details. What are those values separated by colons?

[Chamath]

That is informative. It will be great if you can explain how colorchannelmixer is associated with Hue, Saturation, Contrast and Brightness ?

[avnr]

The colorchannelmixer filter operates on an RGBa color space, but doesn't enforce this color space on the stream's pixel format so from the practical standpoint it doesn't limit you in terms of your ability to chain additional filters that operate on other color spaces. The theoretical side of color spaces is rather wide, if you want to dig into it I'd recommend to start by reading this Wikipedia article: en.wikipedia.org/wiki/Color_space