ffmpeg segment by video-time instead of wall-time

Question

I have a system that records video to disk, and sometimes the system has an error. I'm trying to recreate the recording system in the lab, without the actual camera (it's expensive and there aren't any spares), to attempt to replicate the usage pattern that causes the error.

I'm having trouble taming ffmpeg.

Background:

The video is captured in one thread continuously from an rtsp:// URL provided by the attached camera with this command:

ffmpeg -i rtsp://192.168.0.10/h264 -c copy -map 0 -f segment -segment_time 900 -segment_format mp4 -segment_atclocktime 1 -strftime 1 /tmp/capture-%s.mp4

This works and produces video files of 15 minutes duration as expected.

Problem:

In the lab, I don't have the camera nor the rtsp:// stream. I've instead copied one of the capture MP4s from the real system and am using that as an input to ffmpeg instead. Something like:

ffmpeg -stream_loop -1 -i capture-1469547000.mp4 -c copy -map 0 -f segment **-{NEED PARAM}** -segment_format mp4 -strftime 1 /tmp/captest-%s.mp4

The -stream_loop -1 parameter does what's expected: it reads from the input file (which is 15 minutes in duration) and produces an infinite output stream. This is a reasonable approximation of reading from the rtsp:// stream.

What I can't figure out is what parameters to use to cause this lab system to segment the video into chunks of 15 minute duration just as the real system does. What I expect is a sequence of output files each roughly the same size as the input file.

Attempt #1

Using -segment_time 900 appears to either a) want to use the wall clock value of 15 minutes or b) be ignored. Since copying from an existing MP4 is much much faster than copying from the rtsp:// stream, the resulting capture file is many many copies of the original.

Attempt #2

I used this command

ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 capture-1469547000.mp4

to determine that the input file has 13494 frames.

Using -segment_frames 13494 appears to be ignored, and the output is not segmented at all.

Attempt #3

I've read a bunch of the ffmpeg documentation and am either missing what I need or it doesn't exist.

Using -ss *position* -t *duration* doesn't do continuous recording & segmentation of output.

Request for help

What parameters should I use to get the segmentation to a) work and b) be 15 minutes of video duration?

Possible complication

The timestamps (DTS?) in the example MP4 are not 'good' in that this error is continuously produced:

[segment @ 0x2abc7c0] Non-monotonous DTS in output stream 0:0; previous: 80990160, current: -74730276972; changing to 80990161. This may result in incorrect timestamps in the output file.  
DTS 191648787, next:-830336344130 st:0 invalid dropping  
PTS 191648787, next:-830336344130 invalid dropping st:0

However, I don't need the resulting video files to actually play or be intact, so I am ignoring this, unless it's affecting the segmentation I need to recreate.

Further info

ffmpeg version:

\# /usr/share/local/bin/ffmpeg -version  
ffmpeg version N-79587-g9f9c833 Copyright (c) 2000-2016 the FFmpeg developers  
built with gcc 4.8 (Ubuntu/Linaro 4.8.2-16ubuntu4)  
configuration: --prefix=/home/t/dev/j/third-party/ffmpeg/../build --cross-prefix=/usr/bin/arm-linux-gnueabihf- --cpu=armv7-a --disable-shared --enable-static --enable-gpl --enable-pthreads --enable-nonfree --enable-libx264 --enable-filters --extra-libs=-static --extra-cflags=--static --enable-cross-compile --target-os=linux --disable-inline-asm --arch=armv7 --disable-debug --disable-altivec --disable-sse --disable-armv6 --disable-armv6t2 --disable-mmx --disable-neon --disable-amd3dnow --disable-thumb --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/../build/lib --extra-cflags=-I/home/t/dev/j/third-party/ffmpeg/../build/include --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavcodec --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavdevice --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavfilter --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavformat --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavresample --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libavutil --extra-ldflags=-L/home/t/dev/j/third-party/ffmpeg/libswscale --extra-ldflags=-lx264 --extra-ldflags=-lm --extra-ldflags=-ldl --extra-cflags='-fpic -mthumb'  
libavutil      55. 22.101 / 55. 22.101  
libavcodec     57. 38.100 / 57. 38.100  
libavformat    57. 34.103 / 57. 34.103  
libavdevice    57.  0.101 / 57.  0.101  
libavfilter     6. 44.100 /  6. 44.100  
libswscale      4.  1.100 /  4.  1.100  
libswresample   2.  0.101 /  2.  0.101  
libpostproc    54.  0.100 / 54.  0.100

Solution 1

I have a working solution.

The root cause

The sample capture file I saved turned out to not be 'good' for the purposes I want (an infinite input stream), although it's a good video file otherwise (it plays fine, although it's not seekable). The issue seems to be related to the timestamps, and also possibly one or more bugs with the segment muxer.

The solution

I ran

ffmpeg -i capture-1469547000.mp4 -c copy captemp.mp4

Using captemp.mp4 instead I get a good stream with either stream_loop or the concat muxer.

I'm not sure what the difference is between capture-1469547000.mp4 and captemp.mp4; AtomicParsley shows captemp.mp4 is 12 bytes shorter in the 'elst' atom.

Taking another look at my original setup, and adding segment_list was revealing: the segments were being generated correctly but very very quickly. They were just being appended to the existing segment file instead of making a new one. That's partly the fault of...

strftime potential bug

I was using strftime with a %s format. strftime turned out to be using the host machine's clock time, rather than the time from within the video segment. This is true even in the 'working' case; I've switched to using the segment muxer's %d formating instead.

This is probably a bug, and why in the non-working case the segments were appending to each other.

I'm fairly certain that using the -re flag would workaround this problem by slowing down the processing, but I actually want accelerated processing. So I haven't tried this.

Solution 2

segment_time should work. It does refer to segment duration, not wall clock.

The keyframe placement may be getting in the way, so try

ffmpeg -fflags +genpts -i capture-1469547000.mp4 -c copy -map 0 -f segment -segment_time 900 -segment_format mp4 -break_non_keyframes 1 -strftime 1 /tmp/capture-%s.mp4

I've removed the stream_loop option since there's currently a timestamp generation bug related to its use. That may be interfering here as well.

If you need a really long stream to work with, use the concat demuxer.

Create a text file

file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'
file 'capture-1469547000.mp4'

And then use

ffmpeg -f concat -i list.txt -c copy ...

Comments

[studog]

Thanks for the help, much appreciated. The concat demuxer works. Using -break_non_keyframes 1 doesn't change the behaviour. :-( All combinations of { stream_loop, concat } x { <null>, break_non_keyframes } the behaviour is the same: there is a first segment created at about 3x the size of the input file (I wasn't waiting long enough yesterday), and the second segment appears to grow without bound, to at least more than 3x the size of the first segment. The video-time of the input isn't being respected. I'm going to look at those 'non-monotonous DTS' errors more closely.