Capturing multiple RTSP streams simultaneously in sync

Question

I would like to capture h264 rtsp streams from up to six IP cameras simultaneously and save them to separate files. The cameras are connected to a Gigabit switch which is connected to my pc. The output videos must be synchronized for further processing: the videos need to have the same duration, start time, stop time and framerate. I am attemping to use ffmpeg for this task.

Apparently, my IP cameras give output with a variable frame rate. To record multiple streams in full hd simultanously I plan to copy the stream first and re-encode to a constant frame rate later. This requires that the captured stream carries the time stamp of each frame.

My first attempt to record streams for, say, 10 seconds is:

ffmpeg -loglevel verbose -i rtsp://192.168.1.101 -i rtsp://192.168.1.102 \
-map 0:v -vcodec copy -an -t 10 -y out1.mp4 \
-map 1:v -vcodec copy -an -t 10 -y out2.mp4  

This should save seconds of two streams to out1.mp4 and out2.mp4, copying video, discarding audio, overwriting if the files exist. The result, however, is two videos that have a different length, between 5 and 7 seconds, and they are not synchronized. Also, the videos freeze every three seconds or so. The (verbose) log output reads:

ffmpeg version N-50911-g9efcfbe Copyright (c) 2000-2013 the FFmpeg developers
  built on Mar 13 2013 21:26:48 with gcc 4.7.2 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libg
sm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libo
pencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-li
bschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-lib
twolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enabl
e-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 19.100 / 52. 19.100
  libavcodec     55.  0.100 / 55.  0.100
  libavformat    55.  0.100 / 55.  0.100
  libavdevice    54.  4.100 / 54.  4.100
  libavfilter     3. 45.103 /  3. 45.103
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[rtsp @ 027698e0] SDP:
v=0
o=StreamingServer 3331435948 1116907222000 IN IP4 192.168.1.101
s=h264.mp4
c=IN IP4 0.0.0.0
t=0 0
a=control:*
m=video 0 RTP/AVP 96
a=control:trackID=0
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z0KAKNoB4AiTAWoEBARgQAAF9eE
AAC+vCve+Eg==,aM48gA==
m=application 0 RTP/AVP 107
a=control:trackID=2
a=rtpmap:107 vnd.onvif.metadata/90000


[h264 @ 0276c520] Current profile doesn't provide more RBSP data in PPS, skippin
g
[rtsp @ 027698e0] Estimating duration from bitrate, this may be inaccurate
Input #0, rtsp, from 'rtsp://192.168.1.101':
  Metadata:
    title           : h264.mp4
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: h264 (Baseline), yuv420p, 1920x1088 [SAR 1:1 DAR 30:17],
 30.17 tbr, 90k tbn, 180k tbc
    Stream #0:1: Data: none
[rtsp @ 027682a0] SDP:
v=0
o=StreamingServer 3331435948 1116907222000 IN IP4 192.168.1.102
s=h264.mp4
c=IN IP4 0.0.0.0
t=0 0
a=control:*
m=video 0 RTP/AVP 96
a=control:trackID=0
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1; sprop-parameter-sets=Z0KAKNoB4AiTAWoEBARgQAB9AAA
DARlAve+EgA==,aM48gA==
m=application 0 RTP/AVP 107
a=control:trackID=2
a=rtpmap:107 vnd.onvif.metadata/90000


[h264 @ 0495b420] Current profile doesn't provide more RBSP data in PPS, skippin
g
[rtsp @ 027682a0] Estimating duration from bitrate, this may be inaccurate
Input #1, rtsp, from 'rtsp://192.168.1.102':
  Metadata:
    title           : h264.mp4
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #1:0: Video: h264 (Baseline), yuv420p, 1920x1088 [SAR 1:1 DAR 30:17],
 30.17 tbr, 90k tbn, 180k tbc
    Stream #1:1: Data: none
Output #0, mp4, to 'out1.mp4':
  Metadata:
    title           : h264.mp4
    encoder         : Lavf55.0.100
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1920x1088 [SAR 1
:1 DAR 30:17], q=2-31, 90k tbn, 90k tbc
Output #1, mp4, to 'out2.mp4':
  Metadata:
    title           : h264.mp4
    encoder         : Lavf55.0.100
    Stream #1:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1920x1088 [SAR 1
:1 DAR 30:17], q=2-31, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #1:0 -> #1:0 (copy)
Press [q] to stop, [?] for help
[NULL @ 0495b420] Current profile doesn't provide more RBSP data in PPS, skippin
g
[NULL @ 0276c520] RTP: missed 362 packets
[NULL @ 0276c520] Current profile doesn't provide more RBSP data in PPS, skippin
g
    Last message repeated 1 times
frame=   44 fps=0.0 q=-1.0 q=-1.0 size=     560kB time=00:00:03.24 bitrate=1412.
frame=   60 fps= 59 q=-1.0 q=-1.0 size=     622kB time=00:00:03.77 bitrate=1350.
Current profile doesn't provide more RBSP data in PPS, skipping
[NULL @ 0276c520] Current profile doesn't provide more RBSP data in PPS, skippin
g
frame=   75 fps= 49 q=-1.0 q=-1.0 size=     813kB time=00:00:04.27 bitrate=1559.
frame=   90 fps= 45 q=-1.0 q=-1.0 size=     877kB time=00:00:04.76 bitrate=1507.
Current profile doesn't provide more RBSP data in PPS, skipping
[NULL @ 0276c520] Current profile doesn't provide more RBSP data in PPS, skippin
g
frame=  106 fps= 42 q=-1.0 q=-1.0 size=    1080kB time=00:00:05.29 bitrate=1669.
frame=  121 fps= 40 q=-1.0 q=-1.0 size=    1142kB time=00:00:05.79 bitrate=1613.
Current profile doesn't provide more RBSP data in PPS, skipping
[NULL @ 0276c520] Current profile doesn't provide more RBSP data in PPS, skippin
g
frame=  136 fps= 38 q=-1.0 q=-1.0 size=    1327kB time=00:00:06.29 bitrate=1727.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  151 fps= 37 q=-1.0 q=-1.0 size=    1382kB time=00:00:06.78 bitrate=1667.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  166 fps= 36 q=-1.0 q=-1.0 size=    1574kB time=00:00:07.28 bitrate=1770.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  182 fps= 36 q=-1.0 q=-1.0 size=    1643kB time=00:00:07.81 bitrate=1722.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  183 fps= 33 q=-1.0 q=-1.0 size=    1647kB time=00:00:06.49 bitrate=2078.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  183 fps= 30 q=-1.0 q=-1.0 size=    1647kB time=00:00:06.98 bitrate=1931.
frame=  183 fps= 28 q=-1.0 q=-1.0 size=    1647kB time=00:00:07.48 bitrate=1803.
Current profile doesn't provide more RBSP data in PPS, skipping
frame=  183 fps= 26 q=-1.0 q=-1.0 size=    1647kB time=00:00:07.98 bitrate=1690.
No more output streams to write to, finishing.
frame=  183 fps= 26 q=-1.0 Lq=-1.0 size=    1649kB time=00:00:08.01 bitrate=1685
.8kbits/s
video:3803kB audio:0kB subtitle:0 global headers:0kB muxing overhead -56.630223%

How can I get these streams recorded in sync?

Solution

I'm also confronted to this problem.

The asynchronism comes from the different connection time to the cameras.

The solution I'm working on is to use ffserver to catch the streams continously and broadcas them on localhost then you can use ffmpeg to access to the streams without connexion delay.

Comments

[MartinKS]

If you get this working, I'd love to see an example. Unfortunately, ffserver is not available on Windows. VLC's VLM broadcast may provide a similar solution though.