How can I seek to frame No. X with ffmpeg?

Question

I'm writing a video editor, and I need to seek to exact frame, knowing the frame number.

Other posts on stackoverflow told me that ffmpeg may give me a few broken frames after seeking, which is not a problem for playback but a big problem for video editors.

And I need to seek by frame number, not by time, which will become inaccurate when converted to frame number.

I've read dranger's tuts (which is outdated now), and end up with:

av_seek_frame(fmt_ctx, video_stream_id, frame, AVSEEK_FLAG_ANY);

It always seek to frame No. 0, and always return 0 which means success. Then I tried to read Blender's source code and found it really complex(maybe I should implement an image buffer?).

So, is there any simple way to seek to a frame with just a simple call like seek(context, frame_number)(while getting a full frame, not a broken one)? Or, is there any lightweight library that simplifies this?

EDIT: Thanks to praks411，I found the solution:

void AV_seek(AV * av, size_t frame)
{
    int frame_delta = frame - av->frame_id;
    if (frame_delta < 0 || frame_delta > 5)
        av_seek_frame(av->fmt_ctx, av->video_stream_id,
                frame, AVSEEK_FLAG_BACKWARD);
    while (av->frame_id != frame)
        AV_read_frame(av);
}
void AV_read_frame(AV * av)
{
    AVPacket packet;
    int frame_done;
    while (av_read_frame(av->fmt_ctx, &packet) >= 0) {
        if (packet.stream_index == av->video_stream_id) {
            avcodec_decode_video2(av->codec_ctx, av->frame, &frame_done, &packet);
            if (frame_done) {
                ...
                av->frame_id = packet.dts;
                av_free_packet(&packet);
                return;
            }
        }
        av_free_packet(&packet);
    }
}

EDIT2: Turns out there is a library for this: FFMS2. It is "an FFmpeg based source library [...] for easy frame accurate access", and is portable (at least across Windows and Linux).

Solution

av_seek_frame will only seek based on timestamp to the key-frame. Since it seeks to the keyframe, you may not get what you want. Hence it is recommended to seek to nearest keyframe and then read frame by frame util you reach the desired frame.

However, if you are dealing with fixed FPS value, then you can easily map timestamp to frame index.

Before seeking you will need to convert your time to AVStream.time_base units if you have specified stream. Read ffmpeg documentation of av_seek_frame in avformat.h.

For example, if you want to seek to 1.23 seconds of clip:

 double m_out_start_time = 1.23;
 int flgs = AVSEEK_FLAG_ANY;
 int seek_ts = (m_out_start_time*(m_in_vid_strm->time_base.den))/(m_in_vid_strm->time_base.num);
 if(av_seek_frame(m_informat, m_in_vid_strm_idx,seek_ts, flgs) < 0)
 {
     PRINT_MSG("Failed to seek Video ")
 }

Comments

[Giumo]

Oh, and I finally know the documentation is in the header files, no wonder I can't find any online.

[Giumo]

Then, how can I know where the nearest key-frame is?

[Giumo]

Isn't time_base.den/time_base.num the fps? Isn't seek_ts=time*fps the frame number?

[praks411]

You are talking about codec context time base, I've taken stream time which is different. For seeking to nearest time key frame. I think you should keep some tolerance in seeking that is try to seek let say 3 frame before your desire frame so you will not miss your frame if it is not key frame. Then from there read frame by frame.

[Giumo]

After some trying, I found that I have to set flags to AVSEEK_FLAG_BACKWARD to seek to the keyframe. And seek_ts is the frame number. I also found that the video file I tried before has only 4 keyframes (a WebM, 360 frames in total, 30fps, don't know why), which really interfered my experiments on seeking.

[Sam]

Shouldn't you also add the m_in_vid_strm->start_time to seek_ts?