Overlay animated images with transparency over a static background image using ffmpeg?

Question

I'm looking to create a video using a set of png images that have transparency merged with a static background.

After doing a lot of digging I seems like it's definitely possible by using the filters library.

My initial video making without including the background is:

ffmpeg -y -qscale 1 -r 1 -b 9600 -loop -i bg.png -i frame_%d.png -s hd720 testvid.mp4

Using -vf I can apply the background as overlay:

ffmpeg -y -qscale 1 -r 1 -b 9600 -i frame_%d.png -vf "movie=bg.png [wm];[in][wm] overlay=0:0 [out]" -s hd720 testvid.mp4

However the problem is it's overlaying the background over the input. According libacfilter I can split the input and play with it's content. I'm wondering if I can somehow change the overlay order?

Any help greatly appreciated!

UPDATE 1:
I'm trying to make the following filter work but I'm getting the movie without the background:

ffmpeg -y -qscale 1 -r 1 -b 9600 -i frame_%d.png -vf "movie=bg.png [bg]; [in] split [T1], fifo, [bg] overlay=0:0, [T2] overlay=0:0 [out]; [T1] fifo [T2]" -s hd720 testvid.mp4

UPDATE 2:
Got video making using -vf option. Just piped the input slit it applied image over it and overlayed the two split feeds! Probably not the most efficient way... but it worked!

ffmpeg -y -r 1 -b 9600 -i frame_%d.png -vf "movie=bg.png, scale=1280:720:0:0 [bg]; [in] format=rgb32, split [T1], fifo, [bg] overlay=0:0, [T2] overlay=0:0 [out]; [T1] fifo [T2]" -s hd720 testvid.mp4

Solution 1

The overlay order is controlled by the order of the inputs, from the ffmpeg docs

[...] takes two inputs and one output, the first input is the "main" video on which the second input is overlayed.

You second command thus becomes:

ffmpeg -y -loop 1 -qscale 1 -r 1 -b 9600 -i frame_%d.png -vf "movie=bg.png [wm];[wm][in] overlay=0:0" -s hd720 testvid.mp4

With the latest versions of ffmpeg the new -filter_complex command makes the same process even simpler:

ffmpeg -loop 1 -i bg.png -i frame_%d.png -filter_complex overlay -shortest testvid.mp4

---

A complete working example:

The source of our transparent input images (apologies for dancing):

Exploded to frames with ImageMagick:

convert dancingbanana.gif -define png:color-type=6 over.png

(Setting png:color-type=6 (RGB-Matte) is crucial because ffmpeg doesn't handle indexed transparency correctly.) Inputs are named over-0.png, over-1.png, over-2.png, etc.

Our background image (scaled to banana):

Using ffmpeg version N-40511-g66337bf (a git build from yesterday), we do:

ffmpeg -loop 1 -i bg.png -r 5 -i over-%d.png -filter_complex overlay -shortest out.avi

-loop loops the background image input so that we don't just have one frame, crucial!
-r slows down the dancing banana a bit, optional.
-filter_complex is a very recently added ffmpeg feature making handling of multiple inputs easier.
-shortest ends encoding when the shortest input ends, which is necessary as looping the background means that that input will never end.

Using a slightly less cutting-edge build, ffmpeg version 0.10.2.git-d3d5e84:

ffmpeg -loop 1 -r 5 -i back.png -vf 'movie=over-%d.png [over], [in][over] overlay' -frames:v 8 out.avi

movie doesn't allow rate setting, so we slow down the background instead which gives the same effect. Because the overlaid movie isn't a proper input, we can't use -shortest and instead explicitly set the number of frames to output to how many overlaid input frames we have.

The final result (output as a gif for embedding):

Solution 2

for references in the future as of 17/02/2015, the command-line is :

ffmpeg -loop 1 -i images/background.png -i images/video_overlay%04d.png -filter_complex overlay=shortest=1 testvid.mp4

thanks for llogan who took the time to reply here : https://trac.ffmpeg.org/ticket/4315#comment:1