FFmpeg concat video and audio out of sync

Question

Joining multiple files using ffmpeg concat seems to result in a mismatch of the timestamps or offsets for the audio. I've tried with several videos and noticed the same problem for h.264 / MP4.

Using concat and encoding the video seems to work fine. The audio stays in sync as ffmpeg does the full conversion calculations and seems to get everything right.

However, simply concatenating the videos without any transformation or encoding results in a slowly increasing sync issue. Obviously, encoding the videos rather than simply joining them will result in a loss of information/quality so I would rather find a way around this problem.

I've tried several flags to sort out this problem that appears to be based on the timestamps. None of these seem to correct the problem though.

ffmpeg -f concat -fflags +genpts -async 1 -i segments.txt test.mov
ffmpeg -auto_convert 1 -f concat -fflags +genpts -async 1 -i segments.txt -c copy test2.mov
ffmpeg -f concat -i segments.txt -c copy -fflags +genpts test3.mp4
ffmpeg -f concat -fflags +genpts -async 1 -i segments.txt -copyts test4.mov
ffmpeg -f concat -i segments.txt -copyts test5.mov
ffmpeg -f concat -i segments.txt -copyts -c copy test6.mov
ffmpeg -f concat -fflags +genpts -i segments.txt -copyts -c copy test7.mov

Note: all other questions that I could find on SO seem to "fix" the problem by simply encoding the videos over again. Not a good solution.

Update

I realized the concat wasn't the problem. The original set of clips had mis-matched timestamps. Somehow concat + encoding fixed the issue, but I don't want to re-encode the videos and loose quality each time.

ffmpeg -y -ss 00:00:02.750 -i input.MOV -c copy -t 00:00:05.880 output.MOV

Which resulted in the following data

ffprobe -v quiet -show_entries stream=start_time,duration output.MOV

start_time=-0.247500
duration=6.131125
start_time=-0.257333
duration=6.155333

Since then I've tried to use -tom and -t in different places along with -af apad -c:v copy and I've still failed to get the duration to be the same.

Here is the full ffprobe output

Here is the original (red) vs the segment (green)

Detailed Sample Files

I recorded a sample video, added the commands to chop it up, then concat it. http://davidpennington.me/share/audio_sync_test_video.zip

Solution 1

This two step process should work

Step 1 Pad out the audio in each segment

ffmpeg -i segment1.mov -af apad -c:v copy <audio encoding params> -shortest -avoid_negative_ts make_zero -fflags +genpts padded1.mov

Or

Generate segments with synced streams

ffmpeg -y -ss 00:00:02.750 -i input.MOV -c copy -t 00:00:05.880 -avoid_negative_ts make_zero -fflags +genpts segment.MOV

Step 2 Concat

ffmpeg -f concat -i segments.txt -c copy test.mov

where segments.txt consists of the names of the padded files.

Solution 2

I encountered a similar problem and found a solution that worked, at least for me. In my case, I was also concatenating files, and found audio/video sync problems with iOs, but not with Windows (e.g., VLC media player showed no synchronization problems using the same mp4 file). The symptom for iOs playing this concatenated mp4 was initially good synchronization followed by an increasing loss of synchronization as the movie played, with audio going faster than video. Interestingly, the sync could be restored temporarily by advancing the movie progress slider to any point in the movie, but then the sync would be lost again as the movie continued to play in iOs. By playing the same movie simultaneously in both iOs and Windows VLC, and initially synchronized with each other as well as I could, by observing the evolution of the "echo" between them, I concluded that the iOs audio was going too fast (assuming the Windows player is correct).

For me, the solution was to add the audio filter option -af aresample=async=1000 to the ffmpeg command, which I found as an example in the ffmpeg online documentation and used verbatim. I don't know if this setting is optimal, but the result was a mp4 with audio and video remaining synchronized when played by both iOs and VLC. This ffmpeg option yielded proper iOs synchronization both during concatenation and afterwards when re-encoding the already concatenated file.

Solution 3

I have been struggling with this one for quite some time as well. Particularly when working with Panasonic AVCHD-generated MTS files. My current solution is to concatenate them on the OS level not ffmpeg. I do this on windows and it looks something like this:

COPY /b input_1.mts + input_2.mts + input_3.mts output.mts

On linux it should be something like:

$ cat input_1.mts input_2.mts input_3.mts > output.mts

You can look up documentation for the windows and linux binary concatenation.

This method of concatenation as apposed to transcoding is the way to go if the original format will work for you. This method practically uses no CPU processing and preserves the original quality. A win-win when dealing with bulk media of high quality.

Solution 4

you can use filter_complex to concat different options in one go

ffmpeg -i input1.mp4 -i input2.webm \
-filter_complex "[0:v:0] [0:a:0] [1:v:0] [1:a:0] concat=n=2:v=1:a=1 [v] [a]" \
-map "[v]" -map "[a]" <encoding options> output.mkv

Solution 5

If the input videos have the same video format, audio format, dimensions, etc., you can use mkvmerge from mkvtoolnix to concatenate the videos without re-encoding:

mkvmerge -o output.mkv file1.mkv + file2.mkv + file3.mkv

mkvmerge also accepts input files with an MP4 container, but the output file will have an MKV container even if you try to specify the filename extension of the output file as .mp4. You can change the container with ffmpeg:

mkvmerge -o output.mkv file1.mp4 + file2.mp4 + file3.mp4
ffmpeg -i output.mkv -c copy output.mp4

I needed to concatenate videos from different sources that had been encoded with different settings, so I first used a command like this to resize and re-encode the input videos:

for f in *.mp4;do w=1280;h=720;ffmpeg -i $f -filter:v "scale=iw*min($w/iw\,$h/ih):ih*min($w/iw\,$h/ih),pad=$w:$h:($w-iw*min($w/iw\,$h/ih))/2:($h-ih*min($w/iw\,$h/ih))/2" -c:v libx264 -crf 22 -preset slow -pix_fmt yuv420p -c:a aac -q:a 1 -ac 2 -ar 44100 ${f%mp4}mkv;done

Some of my input videos didn't have an audio channel, so I used a command like this to add a silent audio channel to the videos:

for f in *.mkv;do ffprobe $f|&grep -q 1:\ Audio||{ ffmpeg -i $f -f lavfi -i anullsrc -c:a aac -shortest -c:v copy temp-$f;mv temp-$f $f;};done

I then concatenated the videos using mkvmerge:

mkvmerge -o output.mkv `printf %s\\n *.mkv|sed '1!s/^/+ /'`

View more solutions