How to extract time-accurate video segments with ffmpeg?

Question

This is not a particularly new question area around here, but I've tried what's been suggested there without much luck. So, my story:

I've got a hunk of 15 seconds of straight-from-the-camera.mov video out of which I want to extract a specific chunk, which I can identify by start time and stop time, in seconds. I started by trying to do what I'll call a "copy extraction": to get seconds 9 to 12,

ffmpeg -i test.mov -vcodec copy -acodec copy -ss 9 -to 12 test-copy.mov

This was a not-bad start, but there are some black frames at the beginning and end of the clip, which I can't have -- it has to be a clean edit from the original. So, I tried recoding the original into a new, trimmed clip:

ffmpeg -i test.mov -ss 00:00:09 -t 00:00:03 test-out.mov

This is better, but not quite: There are no longer any black frames at the beginning of the clip, but they're still there at the end.

After some more browsing and reading, I then suspected that the problem is that ffmpeg is having trouble finding the proper points because of a lack of keyframes in the original video. So I recoded the original video to (presumably) add keyframes, in a couple of different ways. Since I want to be able to pick video at boundaries of a second ("from 9 seconds to 12 seconds"), I tried, copying various suggestions around the web,

ffmpeg -i test.mov -force_key_frames "expr:gte(t, n_forced)" test-forced.mp4

and

ffmpeg -i test.mov -g 1 test-g-inserted.mp4

(I built these as mp4's based on some comments about an mp4 container being needed to support the keyframe search, but I'm honestly just hacking here.) I then tried the extraction as before, but on these new videos that presumably now have keyframes in them. No luck -- both seem to be about the same; the start is OK but there are still black frames at the end. (FWIW, both test-forced.mp4 and test-g-inserted.mp4 also have trailing black frames.)

So: I'm still stuck, and would like to not be. Any insights out there as to what I'm doing wrong? I feel like I'm close, but I really need to get rid of those trailing black frames....

Solution 1

Ok, first of all assuming you know the start and stop duration; we will add key-frames at that duration.

ffmpeg -i a.mp4 -force_key_frames 00:00:09,00:00:12 out.mp4

Most of the time you can directly cut the video with perfection but in your case it does not help you; so we have taken care of it by above command. Here be cautious to not add too many key frames as it can be a problem while encoding as per Ffmpeg Docs.

Now you can again try to cut the video from specific time.

ffmpeg -ss 00:00:09 -i out.mp4 -t 00:00:03 -vcodec copy -acodec copy -y final.mp4

This will solve the problem as we have manually added the keyframes at start and end points of the cut . It worked for me.

Cheers.:)

Solution 2

I think the problem the question and other answers are having is that they're using -ss as an option on the output file, not on the input file. Most ffmpeg options aren't global, but instead only apply to the file they precede. It's often not obvious where an option needs to go, so sometimes trial and error is required.

Used correctly, before the input file they're meant to apply to, -ss and -t worked fine for me. When including audio in the output, I had to use -shortest as an option for the output file, or I'd get 2 mins of audio with 2 secs of video.

ffmpeg version N-67413-g2a88c74 (basically git source from Dec 14 2014)

Here's a command line I used to make a clip recently. (actually tweaked to be a better example, since I left in audio for this, and didn't slo-mo it.)

ffmpeg -ss 120.2 -t 0.75 -i ../mcdeint.60p.lossless264.slow.mkv -c:a libopus -shortest -aspect 16:9 -preset veryslow -x264-params nr=250:ref=6 -crf 22 -movflags +faststart clip2.mkv

With -c:a copy (source has AC3 audio), playback starts up wonky with mplayer. It probably grabs audio from the beginning of the audio frame containing the start, and then has to use a a/v offset in the container. On startup, it takes fraction of a second for the audio to get ahead of the video by that offset, and until then the video is playing at very low FPS. So I xcoded the audio. Neither opus nor pcm_s16le can go in mp4, so I used an mkv container for this example.

The source is a lossless x264 encode (-qp 0) from the output of a very slow yadif=3:1,mcdeint=3:1:10 (on some BFF interlaced video from an NTSC DVD, probably from a DV camera). It's NOT all I frames, it's P frames with a normal keyframe interval.

Adjusting -ss by 0.2 secs did exactly what I hoped, so ffmpeg must handle decoding up to the required point. Wasn't just a coincidence of an I frame where I wanted one. Maybe -accurate_seek is the default? I also get the same result (byte-for-byte identical gif output) as when I use a ffvhuff lossless source as input. (but it runs faster, since it doesn't have to decode up to the requested point.)

Another possibly relevant option is -seek2any, but it says "seek to non-keyframe on demuxer level", which sounds like it would let you seek in ways that would produce garbled output. (i.e. start decoding without actually generating the references that the current frame requires, just use all-gray?)

I haven't tried using -c:v copy, since I'm cutting out a really short clip to loop, so I know there won't be I frames where I need them.

This is the command line I actually used, to make a short slo-mo clip with no sound.

ffmpeg -ss 120.2 -t 0.75 -i ../vid.yadif3.1,mcdeint3.1.10.ffvhuff.mkv -an -shortest -aspect 16:9 -c:v libx264 -preset veryslow -x264-params nr=250:ref=6 -crf 22 -filter:v "setpts=3.0*PTS" -movflags +faststart -r 20 clip.mp4

Note that the -r 20 was important, because unlike mkv, ffmpeg's MP4 output is constant-frame-rate-only (edit: because -vsync vfr isn't the default with the mp4 muxer). Without telling it different, it would set the output FPS = input FPS, and duplicate frames when needed to make that happen. x264 and animated gif (with transparency) can both encode duplicate frames very efficiently, but it's still silly.

Before cooking this up as an example, I had done it in 2 steps, one outputting to mkv, then ffmpeg -i clip.mkv -c:v copy -movflags +faststart -r 20 clip.mp4 to remux. BTW, it's possible to change the fps of a video when remuxing, without xcoding, just not with ffmpeg. https://superuser.com/questions/740196/reducing-video-size-with-avconv-why-does-the-size-increase. But anyway, ffmpeg only sent 45 frames to libx264 when making the mkv, even though it thought it was making a 2.2 second video at 60fps. Don't use ffmpeg with mp4 to work with variable FPS stuff.

edit: turns out ffmpeg defaults to -vsync vfr for mkv output, but not for mp4. With -vsync vfr, ffmpeg can write VFR into mp4 output just fine.

And again for gif output, in case I decide not to put it up with HTML5 video (<video controls autoplay loop> <source src="clip.mp4" type="video/mp4"> </video>)

ffmpeg -ss 120.2 -t 0.75 -i ../vid.yadif3.1,mcdeint3.1.10.ffvhuff.mkv -an -shortest -aspect 16:9 -filter:v "setpts=3.0*PTS,scale=854x480" -r 20 clip.gif

I had to use scale= because the gif container doesn't store an aspect ratio, so it can't get auto-scaled on playback. (my 720x480 pixel 16:9 video gets scaled to 854x480 on playback. Actually should be 853.333, but that gets rounded, and then ffmpeg stores 853x480 in the mkv container, hence using -aspect 16:9 all the time, so my mp4 will store the proper aspect ratio [SAR 32:27 DAR 16:9], instead of [SAR 186:157 DAR 279:157])

Solution 3

Poking a sleeping bear in the woods 🧟‍♂️. I didn't see any mention of the -to flag mentioned in this thread, and I found it more useful too use than the -t flag. An example command that works well for me,

ffmpeg -y -i [INPUT.file] -ss 00:42:42 -to 00:84:84 -codec copy [OUTPUT.file]

Solution 4

[UPDATE: this answer may not be correct - see the comments]

There is no need to add keyframes; as Peter says, it's simply a question of getting the options in the right order. However see https://trac.ffmpeg.org/wiki/Seeking for the definitive, official guide on how to do it right.

View more solutions

Comments

[M Abdul Sami]

have you tried setting duration of output video file? I was having the same issue and resolved it using output video duration: ./ffmpeg -ss 0:1:30.00 -i Abduls.mp4 -t 0:0:30.00 -filter_complex "[0:v]trim=duration=30[b]; [b]scale=720:trunc(ow/a/2)*2[a];" -map [a] -map 0:a -c:a copy -c:v libx264 -t 30 short3.mp4

[Jim Miller]

Thanks for the pointers! This seemed to work well in terms of timing, although it does require a recode that I'd like to avoid, and I'm still getting black frames at the end of the trimmed video. Some more experimentation showed that my first recoding-less "copy" attempt is only slightly off in timing from this one, and, although it leaves me with black frames at the start, it turns out that the things I do with the trimmed video (long story omitted) ignore those frames -- maybe they're an artifact of the player? So, thanks again, even if I end up staying with the copy approach.

[Jim Miller]

Followup: I'll approve this as an answer, but if anyone else has any ideas about how to avoid those black frames, it'd be appreciated...

[BlueSword]

@JimMiller would you please give a link to your video..I would check it out with it.

[BlueSword]

@JimMiller I don't know what wrong you are doing or may be update your Ffmpeg because I worked on your given video and the result is perfect. Here check the frames I extracted at 20 fps from your given video, after I ran my command. There is not a single black frame. Result to see

[Jim Miller]

Hmm. I'm running ffmpeg version git-2013-10-30-4fab08c (new as of Nov 1), but I suppose something could have changed since then. Could you post the .mov or .mp4 that you got from yours?

[BlueSword]

@JimMiller its already included. Do have a look at the link again.

[Jim Miller]

Oops -- missed it. But, interesting: I'm seeing black frames in your file, too, in the sense that, when I open the file in QuickTime Player, the video continues to play for a short time after the last visible frame, and I can scrub the player bar a little with all black on the screen. This is increasing my suspicion that all of this is an artifact of the player -- maybe that there's a difference between the real length of the video and what the player thinks is there? In any case, I guess I'm getting a valid file and that I'm ok; thanks VERY much for the help!

[Peter Cordes]

edit: -vsync vfr isn't the default for mp4, but it is for mkv. (The default is selected by the muxer, but ffmpeg's mp4 muxer DOES support VFR.)

[Konstantin]

The only way to make it fast is use c:v copy. Transcoding always takes a lot of time.

[Adam Spiers]

To whomever downvoted this: please be courteous and explain why in a comment, so I can address your grievance.

[Javi]

I can guess is because the answer is not self-contained. Just saying: "other answer is right, check this other thing".

[foba]

Does this work for splitting video into 1 second chunks? Wouldn't that be a lot fo key frames?

[Alex Volkov]

Yes! Among all comands shown here this one worked for me, (ffmpeg 4.2.2) and was way faster than anything else. If you need to cut sub-second segments use the following format 00:00:00.000

[jbielick]

The question title says "time-accurate" and the OP is using codec copy. If you read the section "Seeking while doing a codec copy" of the linked guide on Seeking, you will see that it clearly states that -ss as an input option with codec copy "might not be accurate since ffmpeg is forced to only use/split on i-frames". The documentation is basically saying it won't be accurate, which is my experience as well and invalidates this answer.

[Adam Spiers]

Thanks @jbielick, that sounds like a fair point. I wrote that answer 5.5 years ago so I can't really remember, but I suspect that I based this answer on simple experimentation and a hurried reading of the linked article - in which case maybe I got lucky with it being accurate in my case but not necessarily in general.