errors when decode H.264 frames using ffmpeg

Question

I am getting the following errors when decoding H.264 frames received from the remote end of a H.264 based SIP video call. Appreciate any help in understanding there errors.

non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!

non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!

Solution 1

That just means that ffmpeg has not seen a keyframe yet, which carries SPS and PPS information. SPS and PPS are crucial in decoding an incoming frame/slice. Keyframes are sent periodically(i.e. every 5-10 seconds or more); so if turns out that you joined a stream before the keyframe arrived; you will see this warning for every frame until a keyframe shows up.

As soon as the keyframe shows up from the wire, ffmpeg will have enough information to decode that frame(and any subsequent frames until the next keyframe), so those warnings will go away.

Solution 2

you need to add frames sps and pps information. ffmpeg needs these information to make decoding. You can find these values in SDP file.

In SDP file, you should look NAL units, you can see something like that z0IAHukCwS1xIADbugAzf5GdyGQl, aM4xUg

these values based64 encoded you should convert it to hex format. I am using wireshark and wireshark converts itself these values for you. After that you have sps and pps values.

Now you have to add these Nal information before data frame.

00 00 00 01 sps 00 00 00 01 pps 00 00 00 01 data

for h264 these format i have been using to decode.

Solution 3

To decode a frame or a slice, sliceHeader is decoded, which refers to a PPS or "Picture Parameter Set". It has information regarding the specifics of the frame like width, height etc.

I guess your data is coming through a streaming input channel, in which case SPS and PPS would have been sent earlier in the stream.

You may have to concatenate the same to your stream.

Comments

[John Qualis]

That seems to be the problem. The remote end is not sending a keyframe even when my client requests for it using a SIP INFO. Any ideas how this can be solved? I am using an old version of Bria at the remote end.

[Aki]

The remote side is not sending any keyframes? That is very odd. Not sure if Bria has an option to configure the keyframe interval, but it would be worth looking into. So in general, you are not seeing any video from the remote side at all, or it takes a while to see it?

[Aki]

Your are most likely having general decoding issues. The errors that you are seeing, do they ever stop? Can we see some of your ffmpeg decoding code?

[Aki]

Or you could be having rendering issues of the decoded frames(i.e. mismatched colorspaces in between the decoder and renderer). Anyhow, there is a lot of moving parts here. If you could specify the platform, the way you are rendering video; that would also give me a better idea on what's happening.

[victor1234]

Try to set key frame interval to few seconds on remote coder. If it set in automatic mode and camera is static you can receive frames through very long periods of time

[nmxprime]

@Aki See this. If missing of keyframe is the issue, how can i solve it?

[Jon Watte]

A Raspberry Pi hardware encoded stream with keyframes set to every second will still generate this error for every frame, and never sync up to a keyframe. What else can generate this error?