Receiving RTSP stream using FFMPEG library

Question

I have an IPCamera on my LAN streaming video using RTSP. I have been able to capture and display it successfully using ffplay command:

ffplay rtsp://admin:[email protected]:7070 

(with authentication)

So I would like to achieve the same using programming in C/C++ using ffmpeg library. I guess this must be possible.

So let me phrase two simple questions :

1. How do I receive the stream in a C/C++ program using FFMPEG library? (just provide some URL/tutorial, as google was not helpful)

2. How do I display the received video? (same here, some good URL to direct me).

Solution 1

For rtsp streams the following is working for me (after receiving frames i save the result to a ppm file):

    #include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <fstream>
#include <sstream>

extern "C"
{
    #include <avcodec.h>
    #include <avformat.h>
    #include <avio.h>
    #include <swscale.h>
}

void log_callback(void *ptr, int level, const char *fmt, va_list vargs)
{
   static char message[8192];
   const char *module = NULL;

    if (ptr)
    {
        AVClass *avc = *(AVClass**) ptr;
        module = avc->item_name(ptr);
    }
   vsnprintf_s(message, sizeof(message), fmt, vargs);

   std::cout << "LOG: " << message << std::endl;
}


int main(int argc, char** argv) {

    SwsContext *img_convert_ctx;
    AVFormatContext* context = avformat_alloc_context();
    AVCodecContext* ccontext = avcodec_alloc_context();
    int video_stream_index;

    av_register_all();
    avformat_network_init();
    //av_log_set_callback(&log_callback);

    //open rtsp
    if(avformat_open_input(&context, "rtsp://134.169.178.187:8554/h264.3gp",NULL,NULL) != 0){
        return EXIT_FAILURE;
    }

    if(avformat_find_stream_info(context,NULL) < 0){
        return EXIT_FAILURE;
    }

    //search video stream
    for(int i =0;i<context->nb_streams;i++){
        if(context->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
            video_stream_index = i;
    }

    AVPacket packet;
    av_init_packet(&packet);

    //open output file
    //AVOutputFormat* fmt = av_guess_format(NULL,"test2.mp4",NULL);
    AVFormatContext* oc = avformat_alloc_context();
    //oc->oformat = fmt;
    //avio_open2(&oc->pb, "test.mp4", AVIO_FLAG_WRITE,NULL,NULL);

    AVStream* stream=NULL;
    int cnt = 0;
    //start reading packets from stream and write them to file
    av_read_play(context);//play RTSP

    AVCodec *codec = NULL;
    codec = avcodec_find_decoder(CODEC_ID_H264);
    if (!codec) exit(1);

    avcodec_get_context_defaults3(ccontext, codec);
    avcodec_copy_context(ccontext,context->streams[video_stream_index]->codec);
    std::ofstream myfile;

    if (avcodec_open(ccontext, codec) < 0) exit(1);

    img_convert_ctx = sws_getContext(ccontext->width, ccontext->height, ccontext->pix_fmt, ccontext->width, ccontext->height,
                            PIX_FMT_RGB24, SWS_BICUBIC, NULL, NULL, NULL);

    int size = avpicture_get_size(PIX_FMT_YUV420P, ccontext->width, ccontext->height);
    uint8_t* picture_buf = (uint8_t*)(av_malloc(size));
    AVFrame* pic = avcodec_alloc_frame();
    AVFrame* picrgb = avcodec_alloc_frame();
    int size2 = avpicture_get_size(PIX_FMT_RGB24, ccontext->width, ccontext->height);
    uint8_t* picture_buf2 = (uint8_t*)(av_malloc(size2));
    avpicture_fill((AVPicture *) pic, picture_buf, PIX_FMT_YUV420P, ccontext->width, ccontext->height);
    avpicture_fill((AVPicture *) picrgb, picture_buf2, PIX_FMT_RGB24, ccontext->width, ccontext->height);

    while(av_read_frame(context,&packet)>=0 && cnt <1000)
    {//read 100 frames

        std::cout << "1 Frame: " << cnt << std::endl;
        if(packet.stream_index == video_stream_index){//packet is video
            std::cout << "2 Is Video" << std::endl;
            if(stream == NULL)
            {//create stream in file
                std::cout << "3 create stream" << std::endl;
                stream = avformat_new_stream(oc,context->streams[video_stream_index]->codec->codec);
                avcodec_copy_context(stream->codec,context->streams[video_stream_index]->codec);
                stream->sample_aspect_ratio = context->streams[video_stream_index]->codec->sample_aspect_ratio;
            }
            int check = 0;
            packet.stream_index = stream->id;
            std::cout << "4 decoding" << std::endl;
            int result = avcodec_decode_video2(ccontext, pic, &check, &packet);
            std::cout << "Bytes decoded " << result << " check " << check << std::endl;
            if(cnt > 100)//cnt < 0)
            {
                sws_scale(img_convert_ctx, pic->data, pic->linesize, 0, ccontext->height, picrgb->data, picrgb->linesize);
                std::stringstream name;
                name << "test" << cnt << ".ppm";
                myfile.open(name.str());
                myfile << "P3 " << ccontext->width << " " << ccontext->height << " 255\n";
                for(int y = 0; y < ccontext->height; y++)
                {
                    for(int x = 0; x < ccontext->width * 3; x++)
                        myfile << (int)(picrgb->data[0] + y * picrgb->linesize[0])[x] << " ";
                }
                myfile.close();
            }
            cnt++;
        }
        av_free_packet(&packet);
        av_init_packet(&packet);
    }
    av_free(pic);
    av_free(picrgb);
    av_free(picture_buf);
    av_free(picture_buf2);

    av_read_pause(context);
    avio_close(oc->pb);
    avformat_free_context(oc);

    return (EXIT_SUCCESS);
}

Solution 2

FWIW, I've updated the code provided by @technique to work with the libraries I have from FFMPEG 3.2.2. Hopefully this helps someone Googling this. There are some small changes that could be confusing for people stumbling upon this answer.

#include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <fstream>
#include <sstream>

extern "C" {
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavformat/avio.h>
#include <libswscale/swscale.h>
}

int main(int argc, char** argv) {

    // Open the initial context variables that are needed
    SwsContext *img_convert_ctx;
    AVFormatContext* format_ctx = avformat_alloc_context();
    AVCodecContext* codec_ctx = NULL;
    int video_stream_index;

    // Register everything
    av_register_all();
    avformat_network_init();

    //open RTSP
    if (avformat_open_input(&format_ctx, "rtsp://134.169.178.187:8554/h264.3gp",
            NULL, NULL) != 0) {
        return EXIT_FAILURE;
    }

    if (avformat_find_stream_info(format_ctx, NULL) < 0) {
        return EXIT_FAILURE;
    }

    //search video stream
    for (int i = 0; i < format_ctx->nb_streams; i++) {
        if (format_ctx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
            video_stream_index = i;
    }

    AVPacket packet;
    av_init_packet(&packet);

    //open output file
    AVFormatContext* output_ctx = avformat_alloc_context();

    AVStream* stream = NULL;
    int cnt = 0;

    //start reading packets from stream and write them to file
    av_read_play(format_ctx);    //play RTSP

    // Get the codec
    AVCodec *codec = NULL;
    codec = avcodec_find_decoder(AV_CODEC_ID_H264);
    if (!codec) {
        exit(1);
    }

    // Add this to allocate the context by codec
    codec_ctx = avcodec_alloc_context3(codec);

    avcodec_get_context_defaults3(codec_ctx, codec);
    avcodec_copy_context(codec_ctx, format_ctx->streams[video_stream_index]->codec);
    std::ofstream output_file;

    if (avcodec_open2(codec_ctx, codec, NULL) < 0)
        exit(1);

    img_convert_ctx = sws_getContext(codec_ctx->width, codec_ctx->height,
            codec_ctx->pix_fmt, codec_ctx->width, codec_ctx->height, AV_PIX_FMT_RGB24,
            SWS_BICUBIC, NULL, NULL, NULL);

    int size = avpicture_get_size(AV_PIX_FMT_YUV420P, codec_ctx->width,
            codec_ctx->height);
    uint8_t* picture_buffer = (uint8_t*) (av_malloc(size));
    AVFrame* picture = av_frame_alloc();
    AVFrame* picture_rgb = av_frame_alloc();
    int size2 = avpicture_get_size(AV_PIX_FMT_RGB24, codec_ctx->width,
            codec_ctx->height);
    uint8_t* picture_buffer_2 = (uint8_t*) (av_malloc(size2));
    avpicture_fill((AVPicture *) picture, picture_buffer, AV_PIX_FMT_YUV420P,
            codec_ctx->width, codec_ctx->height);
    avpicture_fill((AVPicture *) picture_rgb, picture_buffer_2, AV_PIX_FMT_RGB24,
            codec_ctx->width, codec_ctx->height);

    while (av_read_frame(format_ctx, &packet) >= 0 && cnt < 1000) { //read ~ 1000 frames

        std::cout << "1 Frame: " << cnt << std::endl;
        if (packet.stream_index == video_stream_index) {    //packet is video
            std::cout << "2 Is Video" << std::endl;
            if (stream == NULL) {    //create stream in file
                std::cout << "3 create stream" << std::endl;
                stream = avformat_new_stream(output_ctx,
                        format_ctx->streams[video_stream_index]->codec->codec);
                avcodec_copy_context(stream->codec,
                        format_ctx->streams[video_stream_index]->codec);
                stream->sample_aspect_ratio =
                        format_ctx->streams[video_stream_index]->codec->sample_aspect_ratio;
            }
            int check = 0;
            packet.stream_index = stream->id;
            std::cout << "4 decoding" << std::endl;
            int result = avcodec_decode_video2(codec_ctx, picture, &check, &packet);
            std::cout << "Bytes decoded " << result << " check " << check
                    << std::endl;
            if (cnt > 100)    //cnt < 0)
                    {
                sws_scale(img_convert_ctx, picture->data, picture->linesize, 0,
                        codec_ctx->height, picture_rgb->data, picture_rgb->linesize);
                std::stringstream file_name;
                file_name << "test" << cnt << ".ppm";
                output_file.open(file_name.str().c_str());
                output_file << "P3 " << codec_ctx->width << " " << codec_ctx->height
                        << " 255\n";
                for (int y = 0; y < codec_ctx->height; y++) {
                    for (int x = 0; x < codec_ctx->width * 3; x++)
                        output_file
                                << (int) (picture_rgb->data[0]
                                        + y * picture_rgb->linesize[0])[x] << " ";
                }
                output_file.close();
            }
            cnt++;
        }
        av_free_packet(&packet);
        av_init_packet(&packet);
    }
    av_free(picture);
    av_free(picture_rgb);
    av_free(picture_buffer);
    av_free(picture_buffer_2);

    av_read_pause(format_ctx);
    avio_close(output_ctx->pb);
    avformat_free_context(output_ctx);

    return (EXIT_SUCCESS);
}

It can be compiled by something to the effect of:

g++ -w my_streamer.cpp -o my_streamer $(pkg-config --cflags --libs  libavcodec libavformat libswscale libavutil)

I include the -w since there are a lot of deprecation warnings for these functions in the library. You will need to have pkg-config as well as the libavcodec libavformat libswscale and libavutil libraries installed. (h/t @BobCheng for missing libraries).

Solution 3

FFmpeg can open rtsp streams directly just like opening local video files. Here is a tutorial code that updates with the most recent version of FFmpeg.

Solution 4

check your ./configure file. May be its compile for armlinux not for the x86. thats why you are getting undefined reference to 'avcodec_register_all'

Here is the solution:-

$ cd ffmpeg 

$export LD_RUN_PATH=/usr/local/lib:/usr/lib:/lib 

$ ./configure

$ make && make install

And after that compile your application.

View more solutions

Comments

[Bhanu Kiran]

thanks for the reply. I have installed ffmpeg and x264 following these instructions. The installation is successful and the libraries are installed in /usr/lib of my ubuntu system. And when I'm trying to compile ur code (from /home/bhanu/main.cpp) using But I'm facing some problems with appropriate flags I keep getting linker errors. I will post them, please guide me what/where could the mistake be ?

[Bhanu Kiran]

The command i am using is g++ main.cpp -lavcodec -lavdevice -lavfilter -lavformat -lavutil -logg -lrtmp -lswscale -lx264 -lpthread -lvorbis -L/usr/lib, i am getting the following output main.cpp: In function 'int main(int, char**)': /home/bhanu/work_environment/softwares/ffmpeg/libavformat/al‌​lformats.c:49: undefined reference to 'avcodec_register_all' /usr/lib/libavformat.a(oggparsevorbis.o): In function 'vorbis_packet': collect2: ld returned 1 exit status and few similar errors. Please refer to link here for complete compiler output.

[Pimgd]

cnt <1000) {//read 100 frames it looks like the code reads 1000 frames instead...

[Tariq]

I created a gist for the error logs that is generated during decoding RTSP stream with FFMPEG. gist.github.com/anonymous/64b3df25b7f103c0da54 Any idea how to resolve the above error logs. The picture displayed is blurred.

[syntheticgio]

@technique This was helpful for me and I've added an updated version as a new answer which works with libraries with ffmpeg 3.2.2 in case anyone is finding this via Google. They have changed some of the functions and enums which threw me for a loop at first.

[Folkert van Heusden]

since your update a few things have changed. Please consider adding a new version. Thanks :-)

[hagor]

is it memory leak safe?

[syntheticgio]

@FolkertvanHeusden I only work with 3.2.2 (and I haven't had to do even that in a while), so I'm sadly not up to date on any new features or changes :(

[syntheticgio]

@hagor I haven't seen any memory issues with similar code.

[Bob Cheng]

Minor edit: you'll need libavcodec & libavutil to compile successfully: g++ -w my_streamer.cpp -o my_streamer $(pkg-config --cflags --libs libavcodec libavformat libswscale libavutil)

[syntheticgio]

Appreciate it @BobCheng - not sure if my setup was allowing me not to include those for some reason but still work. I've added it to the answer for anyone copy/pasting