Convert thousands of .pngs to animated .gif, `convert` uses too much memory

Question

Many of the questions asking how to create an animated gif from a set of png images suggest to use a variant of ImageMagick's convert command:

convert -delay 2 -loop 0 *.png animated.gif

However, I have a few thousand images and thus convert uses up all my memory, swap, and then crashes. What alternative software exists, which is more memory-conscious? I could use another open format if .gif is not supported, and I do prefer a CLI tool.

Solution 1

It sounds like you're trying to make a video. If that's the case, then I'd use a proper video format.

In this case, I'd use ffmpeg to convert the individual PNG files to a H.264 video. Since ffmpeg is made to work with videos that can be hours long, it should have no problem with your thousands of images. Using H.264 instead of animated gif will result in a vast improvement in image quality.

Something like this should work for you:

 ffmpeg -framerate 1/2 -i img%04d.png -c:v libx264 -r 30 out.mp4

• -framerate 1/2: This sets the framerate to one-half FPS, or 2 seconds per frame.
• -i img%04d.png: This tells ffmpeg to read the files img0000.png though img9999.png.
• -c:v libx264: Use video codec libx264.
  • You can specify video compression parameters here, if you like:
  • -crf <number>: Quality setting. 0 to 51. 23 is the default. 0 is true lossless encoding, which will be quite high bandwidth. 18 is nearly visually lossless.
• -r 30: Set the output framerate to 30 FPS. Each of the input images will be duplicated to make the output what you specify here. You can leave this parameter off, and the output file will be at the input framerate, but the resulting movie didn't display properly when I tried it just now.
• out.mp4: Output filename.

References:

• Create a video slideshow from images
• FFmpeg and H.264 Encoding Guide

Solution 2

Personally, I would just launch it on limited numbers of files instead of all at once. For example, something like this:

#!/usr/bin/env bash

## Collect all png files in the files array
files=( *png )
## How many should be done at once
batch=50

## Read the array in batches of $batch
for (( i=0; $i<${#files[@]}; i+=$batch ))
do
    ## Convert this batch
    convert -delay 2 -loop 0 "${files[@]:$i:$batch}" animated.$i.gif
done

## Now, merge them into a single file
convert  animated.*.gif all.gif

Solution 3

Use -limit memory 1GiB to limit the amount of memory convert uses.

1000s of images would create a huge GIF that most computers will struggle to display. I keep my animated GIFs below 200 images when possible. The fewer the better. If you number your images, this command will delete the odd numbered images rm *[13579].png.

So here is my typical workflow for creating an animated GIF from a movie scene:

avconv -ss 00:26:00 -i someMovie.mpg %5d.png
rm  *[13579].png
convert -limit memory 1GiB -loop 0 -layers optimize -resize 400 *.png output.gif

Solution 4

I could use another open format if .gif is not supported

Perhaps APNG is of use to you. It's supported by some browsers, including Firefox but at the moment excluding Chrome and IE. Since it's just a PNG extension, it's very simple to convert PNGs to APNG. The apngasm tool can do that. But the format is so simple that I recently wrote an APNG assembler myself for Sage. Adapting that code would be an alternative.

Solution 5

If you have thousands of png-s, the anigif format is weird. I would do it in this way, using avconv:

 avconv -i "%d.png" -r 25 -c:v libx264 -crf 20 -pix_fmt yuv420p animated.mov

View more solutions

Comments

[Pharap]

I'd like to also point out that APNG is non-standard and despite having been around for 7 years is still very obscure.

[MvG]

For a scientific demonstration, where you might be able to choose the browser used for it, this might still be a simple and robust solution.

[Pharap]

It depends who has to view the demonstration. It wouldn't be very good to hand copies out to the viewers and then two thirds have to go off and download Firefox just to view it. Not the kind of impression I'd like to make if I were making an important scientific demonstration. If it's only intended to be viewed as part of the demonstration then fair enough, but it may not be suitable for other situations.

[codehead]

Let me put it this way: gifsicle already has a smaller footprint than ImageMagick - if you find that it still uses too much memory for the task at hand, then you can use --conserve-memory

[curiousdannii]

Ohh, I think I misread what you were saying, that the switch would use more memory, which doesn't make much sense. never mind.

[dotancohen]

Thank you Sergey. In fact this is not for display on a webpage. +1 for the idea, though.

[Elder Geek]

This approach has the added benefit that you could later mux in voice-over or music audio which is impossible with the Graphics Interchange Format.

[Elder Geek]

Nice expansion of Frantique answer from the day before.

[Ciro Santilli OurBigBook.com]

Here is a simple ffmpeg vs ImageMagick benchmark with concrete test data: askubuntu.com/questions/648244/…

[Louis Gagnon]

Thank you, that solution works well. It may also be necessary to increase the allowed memory limits in /etc/ImageMagick-6/policy.xml see github.com/ImageMagick/ImageMagick/issues/… for more info