Is it possible to iterate a video stream over presentation (not decoding) time?

Created by: h-mayorquin

Overview

First, my understanding is that when I iterate over a video in the way that the documentation suggests for frame in container.decode(video=0) the elements are returned in decoding order (which makes a lot of sense). Is this correct? I am basing my assumption over a specific example and I show this in the investigation below.

My question is, it possible to iterate the frames of a video in the order of pts (presentation timestamps) instead of dts (decoding timestamps)? And if so, how would I do this?

Any pointers about a specific solution that does not involve loading and re-ordering the whole video will be greatly appreciated.

Expected behavior

Does not apply

Actual behavior

Does not apply

Investigation

If I used the following code:

import av

container = av.open(str(video_file_path))

for index, frame in enumerate(container.decode(video=0)):
    print(f"dts = {frame.dts}, pts={frame.pts}, frame={frame.index}, time={frame.time:2.2f}")

I get the following output:

dts = 20, pts=19, frame=0, time=0.32
dts = 21, pts=22, frame=1, time=0.37
dts = 22, pts=21, frame=2, time=0.35
dts = 23, pts=23, frame=3, time=0.38
dts = 24, pts=20, frame=4, time=0.33
dts = 25, pts=26, frame=5, time=0.43
dts = 26, pts=25, frame=6, time=0.42

Research

I have done the following:

Checked the PyAV documentation
Searched on Google
Searched on Stack Overflow
Looked through old GitHub issues
Asked on PyAV Gitter
... and waited 72 hours for a response.

Additional context

We are working in an application for research purposes (academia). Here our concerns is that we need the precise timestamps of each frame. Therefore, it is very important for us to understand properly how to synchronize timestamps with the correct frame. A possible solution is to extract all the frames and then re-order them by presentation time but as the the videos we deal with are rather large, this is unfeasible due to memory concerns.