Meaningful thumbnails for a Video using FFmpeg
Solution 1
How about looking for, ideally, the first >40%-change frame within each of 5 time spans, where the time spans are the 1st, 2nd, 3rd, 4th, and 5th 20% of the video.
You could also split it into 6 time spans and disregard the 1st one to avoid credits.
In practice, this would mean setting the fps to a low number while applying your scene change check and your argument to throw out the first bit of the video.
...something like:
ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.4)" -frames:v 5 -vsync vfr -vf fps=fps=1/600 out%02d.jpg
Solution 2
Defining meaningful is hard but if you want to make N thumbnails efficiently spanning whole video file this is what I use to generate thumbnails on production with user uploaded content.
Pseudo-code
for X in 1..N
T = integer( (X - 0.5) * D / N )
run `ffmpeg -ss <T> -i <movie>
-vf select="eq(pict_type\,I)" -vframes 1 image<X>.jpg`
Where:
- D - video duration read from
ffmpeg -i <movie>
alone orffprobe
which has nice JSON output writer btw - N - total number of thumbnails you want
- X - thumbnail number, from 1 to N
- T - time point for tumbnail
Simply the above writes down center key-frame of each partition of the movie. E.g. if movie is 300s long and you want 3 thumbnails then it takes one key frame after 50s, 150s and 250s. For 5 thumbnails it would be 30s, 90s, 150s, 210s, 270s. You can adjust N depending on movie duration D, that e.g. 5 minute movie will have 3 thumbnails but over 1 hour will have 20 thumbnails.
Performance
Each invocation of above ffmpeg
command takes a fraction of second (!) for ~1GB H.264. That is because it instantly jumps to <time>
position (mind -ss
before -i
) and takes first key frame which is practically complete JPEG. There is no time wasted for rendering the movie to match exact time position.
Post-processing
You can mix above with scale
or any other resize method. You can also remove solid color frames or try to mix it with other filters like thumbnail
.
Solution 3
Try this
ffmpeg -i input.mp4 -vf fps= no_of_thumbs_req/total_video_time out%d.png
Using this command I am able to generate the required number of thumbnails which are representative of the entire video.
Solution 4
I once did something similar, but I exported all frames of the video (in 1 fps) and compared them with a perl utility I found which computes the difference between images. I compared each frame to previous thumbnails, and if it was different from all thumbnails, I added it to the thumbnails collection. The advantage here is that if your video moves from scene A to B and them returns to A, ffmpeg will export 2 frames of A.
Solution 5
I can't add comments yet, so in reference to @jaygooby solution causing a syntax error Meaningful thumbnails for a Video using FFmpeg, the problem is that $N (number of screenshots) wasn't declared. The fix is to add it at the beginning of the command, like this:
N=30; D=180; for X in $(seq 1 $N); do echo $X; T=$(bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i in.mp4 -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done
Furthermore, @jaygooby's solution works well and is very fast (because of putting -ss before -i in ffmpeg's command). I combined both of his suggestions (the one with mediainfo) plus my fix:
N=60; INPUT="path/to/filename"; D=$(bc <<< $(mediainfo --Inform="Video;%Duration%" "$INPUT")/1000) ; echo "Duration: $D seconds"; for X in $(seq 1 $N); do echo $X; T=$(bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i $INPUT -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done
where:
N = number of desired screenshots
INPUT = path to the filename
Change those two variables and you're good to go.
Related videos on Youtube
GlienKim
Updated on September 18, 2022Comments
-
GlienKim over 1 year
FFmpeg can capture images from videos that can be used as thumbnails to represent the video. Most common ways of doing that are captured in the FFmpeg Wiki.
But, I don't want to pick random frames at some intervals. I found some options using filters on FFmpeg to capture scene changes:
The filter
thumbnail
tries to find the most representative frames in the video:ffmpeg -i input.mp4 -vf "thumbnail,scale=640:360" -frames:v 1 thumb.png
and the following command selects only frames that have more than 40% of changes compared to previous (and so probably are scene changes) and generates a sequence of 5 PNGs.
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.4),scale=640:360" -frames:v 5 thumb%03d.png
Info credit for the above commands to Fabio Sonnati. The second one seemed better as I could get n images and pick the best. I tried it and it generated the same image 5 times.
Some more investigation led me to:
ffmpeg -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.png
-vsync vfr
ensures that you get different images. This still always picks the first frame of the video, in most cases the first frame is credits/logo and not meaningful, so I added a-ss
3 to discard first 3 seconds of the video.My final command looks like this:
ffmpeg -ss 3 -i input.mp4 -vf "select=gt(scene\,0.5)" -frames:v 5 -vsync vfr out%02d.jpg
This was the best I could do. I have noticed that since I pick only 5 videos , all of them are mostly from beginning of the video and may miss out on important scenes that occur later in the video
I would like to pick your brains for any other better options.
-
slhck over 11 yearsNice command examples. FWIW, I didn't run into any issues with FFmpeg-generated JPEG pictures on OS X (10.8, FFmpeg 1.1 and below). Your second to last command works fine for me—so does the last—and none of these results in blank JPG files. I did compile with
libopenjpeg
.. not sure if that makes a difference. -
GlienKim over 11 yearsThanks slhck. Edited the question with ffmpeg config/version details. I have not upgraded to 1.1 on this machine. I will do that and see if it changes any results.
-
slhck over 11 yearsSo you're on Ubuntu? Can you try the latest Git Master version from a static build or compiling yourself and running again? Or the latest stable. I just checked, it uses the
mjpeg
encoder for me as well, and I also checkedjpegoptim
andexiv2
, both of which work fine for me with all the JPG results from your example commands. -
GlienKim over 11 yearsI updated, and it works now! I guess the previous version had some bugs.
-
Lizz about 11 yearsCan you go ahead and post the solution- new version, preferably with link to changelog showing the bug you encountered and subsequently fixed with new version?
-
GlienKim about 11 yearsHi @Lizz ,The same commands above were returning blank images in the older version of FFmpeg. No, I am on the latest versions and all the commands work!
-
Sam Watkins almost 9 yearsMy approach is to generate perhaps a dozen jpeg thumbnails at different times, and choose the one with the largest file size. The largest file tends to be more "complex" and interesting, it would seldom be a blurry image, and it will never be a boring image such as a plain black frame. This approach might not be ideal for you, but perhaps you can use it together with other methods.
-
chovy over 8 yearsThese are great commands, but is there anyway to reduce the quality further? I like the dimensions, but its still 400KB to download. Maybe something 100KB in size would be nice.
-
Tina J almost 6 yearsI'm looking for an opposite solution: select more frames from periods where camera is more stable and not moving? (where the difference between successive frames are less, not higher). Is there a way to do that?
-
dǝɥɔS ʇoıןןƎ almost 3 yearsUltimate solution is to write an image processing python script
-
-
Eran Ben-Natan about 11 yearsUnfortunately I don't remember, it was quite a long ago. You need to decide first which compare method to use, and then do some tests to find correct factor. This shouldn't take long.
-
GlienKim almost 11 yearsYes, this does help in picking thumbnails from different sections of the Video. thanks!
-
GlienKim almost 11 yearsCompletely black or white images get picked up.How do I avoid those?
-
A.M. almost 11 yearsThere are filters to detect black frames and sequences of them (ffmpeg.org/ffmpeg-filters.html#blackframe or ffmpeg.org/ffmpeg-filters.html#blackdetect). You make not be able to work either of them into your growing one-liner, but you should definitely be able to strip out the black frames (separate step) and extract thumbnails from the resulting video.
-
A.M. almost 11 yearsAs for white frames, well now it's getting complicated, but it still looks like there is a way: 1. strip out black frames 2. negate (white turns to black) 3. strip out white frames 4. negate again 5. extract thumbnails (If you do manage to strip out the black or white frames, especially on one line, can you post it back here? I can add it to the answer for future readers. ...or actually you could create a new question and answer it. I would definitely upvote it.)
-
Liam over 10 yearsWould it be possible to also capture a few seconds worth of frames after the beginning of each selected scene?
-
Samuel over 10 yearsI'm just thinking out loud here but isn't this worse option? When you compare 2 exported images from video you've already lost some information from it. I mean, it is easier to calculate similarity from codec information than it is from 2 pictures, isn't it? Recently I've been looking into some algorithms / libraries to compare images and they don't work as well as one may require.
-
Eran Ben-Natan over 10 yearsWell, with ffmpeg you can extract frames without quality loss using the same_quality flag. If you use codec information you need to check that you don't just get the Iframes. Anyway that perl utility worked just fine for me, and is very configurable. Look here: search.cpan.org/~avif/Image-Compare-0.3/Compare.pm
-
Sam Watkins almost 9 yearsTo avoid boring images such as plain black and white: generate more thumbnails than you need, compress them with jpeg, and choose the images with larger file size. This works surprisingly well by itself to get decent thumbnails.
-
Steve Tarver over 8 years@SamWatkins that's a great idea!
-
apinstein about 8 yearswow moving
-ss N
to before-i
is an amazing tip. thank you! -
John Wiseman about 8 yearsThis command didn't work, I got a the error
Unable to find a suitable output format for 'fps=fps=1/600'
. The solution is to add-vf
before that argument (see stackoverflow.com/questions/28519403/ffmpeg-command-issue) -
flolilo over 6 yearsWouldn't the correct formula for
fps
be(no_of_frames_req * fps_of_vid) / total_video_frames
? -
A.M. about 6 yearsI see someone edited my answer to add that tidbit to my original stab. It should work as written in the answer now.
-
Suhail Gupta almost 6 yearsWould you know a way where I could also have the seconds in the output filename, that represents the number of seconds elapsed in a video? I have also asked it on SO: stackoverflow.com/questions/51007328/…
-
Tina J almost 6 yearsI'm looking for an opposite solution: select more frames from periods where camera is more stable and not moving? (where the difference between successive frames are less, not higher). Is there a way to do that?
-
Tina J almost 6 yearsI'm looking for an opposite solution: select more frames from periods where camera is more stable and not moving? (where the difference between successive frames are less, not higher). Is there a way to do that?
-
Phani Rithvij about 4 yearsFastest answer possible . Great answer.
-
jaygooby almost 4 yearsIn bash and using
bc
for the math, with a 3 minute (180 second) duration video:D=180; for X in $(seq 1 $N); do echo $X; T=$(bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i in.mp4 -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done
-
jaygooby almost 4 yearsAnd you can use mediainfo to get the duration too, but it's in milliseconds, so needs converting to seconds:
D=$(bc <<< $(mediainfo --Inform="Video;%Duration%" "in.mp4")/1000)
-
Someone almost 4 years@jaygooby your bash code gives this error: (standard_in) 2: syntax error on Debian 10.4
-
Someone almost 4 yearsGetting this: "Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)" No images is produced. There was this at the beginning: "Multiple -filter, -af or -vf options specified for stream 0, only the last optn '-filter:v fps=fps=1/600' will be used."
-
jaygooby almost 4 yearsYou might need to override what
bc
uses for your locale decimal separator. You can do this withLC_ALL=C.UTF-8 bc
so the script would be:D=180; for X in $(seq 1 $N); do echo $X; T=$(LC_ALL=C.UTF-8 bc <<< "($X-0.5)*$D/$N"); ffmpeg -y -hide_banner -loglevel panic -ss $T -i in.mp4 -vf select="eq(pict_type\,I)" -vframes 1 $X.jpg; done
and themediainfo
version would be:D=$(LC_ALL=C.UTF-8 bc <<< $(mediainfo --Inform="Video;%Duration%" "in.mp4")/1000)