Streaming video from an image using FFMPEG on Windows

Question

I wrote a program that simulates a camera and converts the output into a video stream. The program is required to be able to run on Windows. There are two components in the system:

1. Camera Simulator. A C++ program that simulates the camera. It copies a pre-generated frame (i.e. PNG file) every 0.1 seconds, using the windows copy command, to a destination path ./target/target_image.png
2. Video Stream. Using FFmpeg, it creates a video stream out of the copied images. FFmpeg is ran with the following command: ffmpeg -loop 1 -i ./target/target_image.png -r 10 -vcodec mpeg4 -f mpegts udp://127.0.0.1:1234

When running the whole thing together, it works fine for a few seconds until the ffmpeg halts. Here is a log while running in debug mode:

ffmpeg version N-52458-gaa96439 Copyright (c) 2000-2013 the FFmpeg developers
  built on Apr 24 2013 22:19:32 with gcc 4.8.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libcaca --enable-libfreetype --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      52. 27.101 / 52. 27.101
  libavcodec     55.  6.100 / 55.  6.100
  libavformat    55.  3.100 / 55.  3.100
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 60.101 /  3. 60.101
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
Splitting the commandline.
Reading option '-loop' ... matched as AVOption 'loop' with argument '1'.
Reading option '-i' ... matched as input file with argument './target/target_image.png'.
Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '10'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'mpeg4'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mpegts'.
Reading option 'udp://127.0.0.1:1234' ... matched as output file.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input file ./target/target_image.png.
Successfully parsed a group of options.
Opening an input file: ./target/target_image.png.
[AVIOContext @ 02678840] Statistics: 234307 bytes read, 0 seeks
[AVIOContext @ 02678840] Statistics: 221345 bytes read, 0 seeks
    Last message repeated 1 times
[AVIOContext @ 02678840] Statistics: 226329 bytes read, 0 seeks
    Last message repeated 2 times
[AVIOContext @ 02678840] Statistics: 228676 bytes read, 0 seeks
    Last message repeated 2 times
[AVIOContext @ 02678840] Statistics: 230685 bytes read, 0 seeks
    Last message repeated 2 times
[AVIOContext @ 02678840] Statistics: 232697 bytes read, 0 seeks
    Last message repeated 5 times
[AVIOContext @ 02678840] Statistics: 234900 bytes read, 0 seeks
    Last message repeated 2 times
[AVIOContext @ 02678840] Statistics: 236847 bytes read, 0 seeks
[image2 @ 02677ac0] Probe buffer size limit of 5000000 bytes reached
Input #0, image2, from './target/target_image.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0, 22, 1/25: Video: png, rgb24, 1274x772 [SAR 1:1 DAR 637:386], 1/25, 25 fps, 25 tbr, 25 tbn, 25 tbc
Successfully opened the file.
Parsing a group of options: output file udp://127.0.0.1:1234.
Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 10.
Applying option vcodec (force video codec ('copy' to copy stream)) with argument mpeg4.
Applying option f (force format) with argument mpegts.
Successfully parsed a group of options.
Opening an output file: udp://127.0.0.1:1234.
Successfully opened the file.
[graph 0 input from stream 0:0 @ 02769280] Setting 'video_size' to value '1274x772'
[graph 0 input from stream 0:0 @ 02769280] Setting 'pix_fmt' to value '2'
[graph 0 input from stream 0:0 @ 02769280] Setting 'time_base' to value '1/25'
[graph 0 input from stream 0:0 @ 02769280] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 02769280] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 02769280] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 02769280] w:1274 h:772 pixfmt:rgb24 tb:1/25 fr:25/1 sar:1/1 sws_param:flags=2
[format @ 02768ba0] compat: called with args=[yuv420p]
[format @ 02768ba0] Setting 'pix_fmts' to value 'yuv420p'
[auto-inserted scaler 0 @ 02768740] Setting 'w' to value '0'
[auto-inserted scaler 0 @ 02768740] Setting 'h' to value '0'
[auto-inserted scaler 0 @ 02768740] Setting 'flags' to value '0x4'
[auto-inserted scaler 0 @ 02768740] w:0 h:0 flags:'0x4' interl:0
[format @ 02768ba0] auto-inserting filter 'auto-inserted scaler 0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 026772c0] query_formats: 4 queried, 3 merged, 1 already done, 0 delayed
[auto-inserted scaler 0 @ 02768740] w:1274 h:772 fmt:rgb24 sar:1/1 -> w:1274 h:772 fmt:yuv420p sar:1/1 flags:0x4
[mpeg4 @ 02785020] detected 4 logical cores
[mpeg4 @ 02785020] intra_quant_bias = 0 inter_quant_bias = -64
[mpegts @ 0277da40] muxrate VBR, pcr every 1 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'udp://127.0.0.1:1234':
  Metadata:
    encoder         : Lavf55.3.100
    Stream #0:0, 0, 1/90000: Video: mpeg4, yuv420p, 1274x772 [SAR 1:1 DAR 637:386], 1/10, q=2-31, 200 kb/s, 90k tbn, 10 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (png -> mpeg4)
Press [q] to stop, [?] for help
*** drop!
    Last message repeated 10 times
frame=   11 fps=0.0 q=4.0 size=     118kB time=00:00:01.10 bitrate= 875.1kbits/s dup=0 drop=11    
Statistics: 242771 bytes read, 0 seeks
[AVIOContext @ 02674a60] Statistics: 246525 bytes read, 0 seeks
*** drop!
[AVIOContext @ 02674a60] Statistics: 230678 bytes read, 0 seeks
[AVIOContext @ 02674a60] Statistics: 244023 bytes read, 0 seeks
*** drop!
[AVIOContext @ 02674a60] Statistics: 246389 bytes read, 0 seeks

*** drop!
[AVIOContext @ 02674a60] Statistics: 224478 bytes read, 0 seeks
[AVIOContext @ 02674a60] Statistics: 228013 bytes read, 0 seeks
*** drop!
[image2 @ 02677ac0] Could not open file : ./target/target_image.png
./target/target_image.png: Input/output error
[output stream 0:0 @ 02768c20] EOF on sink link output stream 0:0:default.
No more output streams to write to, finishing.
frame=  164 fps= 17 q=31.0 Lsize=     959kB time=00:00:16.40 bitrate= 478.9kbits/s dup=0 drop=240    

video:869kB audio:0kB subtitle:0 global headers:0kB muxing overhead 10.285235%
404 frames successfully decoded, 0 decoding errors
[AVIOContext @ 026779c0] Statistics: 0 seeks, 746 writeouts

It seems to me there's some kind of collision between the reading and writing to/from the same file. What's also interesting is that on Linux (while replacing the copy with cp) the program works just fine.

Can someone suggest a way to solve this issue? Alternatives solutions are also acceptable as long as the logical workflow remains the same.

Solution 1

Following Nick van Tilborg's comment, I ended up using FFmpeg's image2pipe. This feature allows streamlining the image data into FFmpeg, instead of two processes accessing the same file simultaneously.

Below is the C++ code I wrote. It might not be fully optimized, but it does the job. It was compiled and tested with Visual Studio 2012 on Windows 7.

#include "stdafx.h"
#include "windows.h"
#include "iostream"

#include "stdio.h"

using namespace std;

#pragma warning (disable : 4996)
int _tmain(int argc, _TCHAR* argv[])
{
    int count;
    int times = 2200;
    FILE *pPipe;
    FILE * pFile;
    long lSize;
    char * buffer;
    size_t result;

    // open a pipe to FFmpeg
    if( (pPipe = _popen( "ffmpeg -re -f image2pipe -vcodec mjpeg -i - -vcodec h264 -r 10 -f mpegts udp://127.0.0.1:1234", "wb")) == NULL ) {exit( 1 );}

    for ( count = 1; count <= times; count++) {
        char filename[40];
        sprintf(&filename[0], ".\\images\\image-%07d.jpg", count);

        pFile = fopen ( filename , "rb" );
        if (pFile==NULL) {fputs ("File error",stderr); exit (2);}

        // obtain file size:
        fseek (pFile , 0 , SEEK_END);
        lSize = ftell (pFile);
        rewind (pFile);

        // allocate memory to contain the whole file:
        buffer = (char*) malloc (sizeof(char)*lSize);
        if (buffer == NULL) {fputs ("Memory error",stderr); exit (3);}

        // copy the file into the buffer:
        result = fread (buffer, 1, lSize, pFile);
        if (result != lSize) {fputs ("Reading error",stderr); exit (4);}

        // write to pipe
        fwrite(buffer, 1, lSize, pPipe); 
        fflush(pPipe);

        // clean
        fclose (pFile);
        free (buffer);

        //
        Sleep(100);
    }

    //
    return 0;
}

Solution 2

Using your current command line, FFmpeg is running at the highest possible speed. This means FFmpeg is reading the frames not at 10 fps, but as fast as possible. It is also not transmitting the transport stream at 10 fps. Because your program is fixed writing frames at 10 fps, this is the reason you probably can't read your file sometimes, since the file is writed at these particular times.

To solve this, try using the -re flag in your FFmpeg command line to force FFmpeg to read the input at the native framerate.

ffmpeg -re -loop 1 -i ./target/target_image.png -r 10 -vcodec mpeg4 -f mpegts udp://127.0.0.1:1234

Comments

[Daniel Zohar]

Thanks! There's seems to be a slight improvement, as ffmpeg fails after a longer running time, but the problem still persists. I still think it is a synchronisation issue. Even if the images are copied at 10fps and the ffmpeg reads the image at 10fps, there is a chance that both processes will access the file at the same time, isn't it?

[Omega]

Yes, that is true. Maybe you can look in some sort of piping direction like using the -f image2pipe. I never used it myself so I am not sure if this is right.