FFMPEG and got_picture_ptr

This is a tad on the extreme nerd side, so apologies to all of the readers expecting something different. I am merely posting it here because after scouring the internet, I was unable to find a solution to this problem.

A Tough Nut to Crack

Whilst attempting to write a high quality post production decoder, I came across a rather difficult issue with FFMPEG.

There is a basic loop in FFMPEG to decode a video stream into the raw whole-frame packets. After correct setup, it is loosely: The Presentation Time Stamp (PTS) and the Decode Time Stamp (DTS) are always in PTS order out of avcodec_decode_video2. This presents a bit of a problem for a video file of n length, as the loop will continue calling av_read_frame until it returns zero.

The problem here is that at various points in the frame stream, if a frame is a B (Bi-Predictive) frame before the needed whole frame I (Intra-Coded Picture) frame or P (Predictive) frame, avcodec_decode_video2 will return the got_picture_ptr with zero. If avcodec_decode_video2 returned a non-zero value, your frames would be out of order as the B frame requires a reference frame after it, and the decoder would buffer the before and after frames prior to decoding the needed B frame. (Phew.)

In this instance, the out of order frame is buffered internally.

So What is the Problem?

The main problem is that if our stream contains n number of whole frames, and we iterate using the av_read_frame loop, av_read_frame will return finished at precisely the final frame count of n. This means that if there are buffered P frames where got_picture_ptr returned zero and as a result we could not perform an action on the frame, they will not show up in our total frame counts in the inner loop.

What is the Solution?

The solution is as easy as either:

1. Finish the main loop and call avcodec_decode_video2 until got_picture_ptr returns zero outside of it.
2. Store a total count of skipped frames and perform a for loop after the main loop concludes.

If you perform either of the two options, you should be able to fetch all of the frames to match your overall frame count.

Hopefully someone will find this useful out there. It took me far too long to sort the problem out...