How to read any frame while having frame number using ffmpeg av_seek_frame()

Question

int64_t timeBase;
timeBase = (int64_t(pavStrm-> time_base.num) * AV_TIME_BASE) / int64_t(pavStrm->time_base.den);
int64_t seekTarget = int64_t(iFrameNumber) * timeBase;
av_seek_frame(fmt_ctx, -1, seekTarget, AVSEEK_FLAG_FRAME);

here I want to read next 5 frame after iFrameNumebr

for(int iCnt = 0; iCnt <= 4; iCnt++)
{
    iRet = av_read_frame(fmt_ctx, &pkt);
        do 
        {
            ret = decode_packet(&got_frame, 0);
            if (ret < 0)
                break;
            pkt.data += ret;
            pkt.size -= ret;

        }while (pkt.size > 0);
    av_free_packet(&pkt);
}

static int decode_packet(int *got_frame, int cached)
{
int ret = 0;
int decoded = pkt.size;
*got_frame = 0;

if (pkt.stream_index == video_stream_idx)
{
    /* decode video frame */
    ret = avcodec_decode_video2(video_dec_ctx, frame, got_frame, &pkt);
}

when i am using AVSEEK_FLAG_BACKWARD its return 5 packet and 5 frame first two is blank but correct.

when i am using AVSEEK_FLAG_FRAME its return 5 packet and 3 frame which are not first 3 frame its return specific frame from video.

for any iFrameNumber

so please help me how to get frame while having frame number and what is exact value of seektarget 3rd param of av_seek_frame()

also I have problem while converting frame to rgb24 format

Solution 1

int64_t FrameToPts(AVStream* pavStream, int frame) const
{
return (int64_t(frame) * pavStream->r_frame_rate.den *  pavStream-
>time_base.den) / 
(int64_t(pavStream->r_frame_rate.num) * 
pavStream->time_base.num);
}

iSeekTarget = FrameToPts(m_pAVVideoStream, max(0, lFrame));
iSuccess = av_seek_frame(m_pAVFmtCtx, m_iVideo_Stream_idx, 
iSeekTarget, iSeekFlag);

AVPacket avPacket;
iRet = av_read_frame(m_pAVFmtCtx, &avPacket);

Solution 2

I think av_seek_frame() is one of the most common but difficult to understand function, also not well commented enough.

If the flag AVSEEK_FLAG_FRAME is set, the third parameter should be a frame number you want to seek, which you're doing fine.

Let's see a example to have a better understand of av_seek_frame():

Say I have a video of 10 frames, with fps=10. The first and fifth frame is key frame (I Frame or intra frame). Others are P frames or even B frames in some format.

0 1 2 3 4 5 6 7 8 9 (frame number)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 (timebase)

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME);
av_seek_frame(fmt_ctx, -1, 0.15, 0);
// These will seek to the fifth frame. Cause `AVSEEK_FLAG_ANY` is not given. Seeking to the next key frame after third parameter.

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME | AVSEEK_FLAG_ANY);
// This will seek to exactly the third parameter specified. But probably only a frame with no actual meaning. (We can't get a meaningful image if no related I/P/B frames given.)

av_seek_frame(fmt_ctx, -1, 0.15, AVSEEK_FLAG_ANY);
// Seek to 0.2. Nothing interesting as above.

av_seek_frame(fmt_ctx, -1, 0.15, AVSEEK_FLAG_ANY | AVSEEK_FLAG_BACKWARD);
// Seek to 0.1. Also nothing interesting.

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME | AVSEEK_FLAG_BACKWARD);
// Got the first frame. Seeking to the nearest key frame before the third parameter.

So if I'd like to get arbitrary frame, usually seeking with AVSEEK_FLAG_BACKWARD first, decoding as usual. Then check the first several packets pts and duration, see if we need to drop them.

Solution 3

timeBase = (int64_t(video_stream-> time_base.num) * AV_TIME_BASE) / int64_t(video_stream->time_base.den);
int64_t seekTarget = int64_t(iFrameNumber) * timeBase * (video_stream->time_base.den / video_stream->avg_frame_rate.num);


int iiiret = av_seek_frame(fmt_ctx, -1, seekTarget, AVSEEK_FLAG_FRAME);

Comments

[P Akhtar]

Its still enable to reach desired frame and normally when i was building it in play mode not returning continuous frame bit upset by the way thank for comment hope i will start after getting continuous frame.

[ZeroDefect]

@halfelf What is the expected behaviour when seek to frame number '4' (5th frame) with flags (ANY | BACKWARD). Should that seek to '0' or '4' ?

[halfelf]

@ZeroDefect According to document of avformat_seek_file(), which will be called by av_seek_frame(), the fifth parameter of avformat_seek_file, max_ts, means largest acceptable timestamp, which is equivalent to the third parameter of av_seek_frame, if AV_SEEK_FLAG_BACKWARD is given, so that seeking to '0.4' is expected. btw, there may be another uncommon pitfall , that AVSEEK_FLAG_ANY is not supported by some demuxers.

[Bill Yan]

so after seeking, how can I check which frame is the current frame? you mentioned pts, is that how you do it?

[Bill Yan]

this one works! AVSEEK_FLAG_FRAME, suggested by other answers, didn't work for me.

[Bill Yan]

I doubt this works? After checking libav's code, I think av_seek_frame doesn't accept AVSEEK_FLAG_FRAME at all. And I tried it myself too, with or without AVSEEK_FLAG_FRAME, av_seek_frame always treats the input as a timestamp. AVSEEK_FLAG_FRAME appears to be a flag of avformat_seek_file() only

[halfelf]

@BillYan actually AVSEEK_FLAG_FRAME is used by read_seek (and some of its cousins), which is a function pointer of AVInputFormat struct, and implemented in some format decoders. Since read_seek is an old api and replaced by read_seek2, I believe it is not widely used anymore.

[Bill Yan]

doing a grep "AVSEEK_FLAG_FRAME" in libavformat, you will see, it is not being used anywhere.

[Bill Yan]

And I also experimented, passing a frame number, with or without AVSEEK_FLAG_FRAME, the frame number will be treated as timestamp

[Christoph Rackwitz]

this assumes constant frame rate. if you have variable frame rate (variable duration frames), this simple calculation can't be used.

[isudfv]

with or with not AVSEEK_FLAG_FRAME didn't make any difference, at least for 4.4.1#12