Skip to content
GitLab
Open
Issue created by Hotte@horsto3 of 6 checklist items completed3/6 checklist items

Inaccessible frames when using threading

I am using av in combination with PIMS. I am using pims.PyAVReaderIndexed (https://github.com/soft-matter/pims/blob/85b1adcab4e1d2881a9915ac29da6b1556bb904f/pims/pyav_reader.py#L274) to build a toc of the video I am reading, to index frames precisely in my file. The decoding process takes a very long time (almost 30 min for a 1.5 hour video file) and I was looking into threading as described here https://pyav.org/docs/stable/cookbook/basics.html#threading.

I have subclassed pims.PyAVReaderIndexed and added the flag stream.thread_type = "AUTO", which speeds up the table of content calculation about 2.5x - not too bad. However, I realized when looking at the toc returned by pims.PyAVReaderIndexed that the threaded version misses 15 frames at the end compared to the non -threaded version. The cumsum of all frames is reporting the same number of frames in the toc, but I cannot access the last 15 frames of the file loaded in pims. I am not sure where the root of this problem is - can anybody help?

Versions

  • OS: Windows 11 Pro
  • PyAV runtime:
PyAV v10.0.0
library configuration: --prefix=/d/bld/ffmpeg_1666310407352/_h_env/Library --cc=clang.exe --cxx=clang++.exe --nm=llvm-nm --ar=llvm-ar --disable-doc --disable-openssl --enable-demuxer=dash --enable-hardcoded-tables --enable-libfreetype --enable-libfontconfig --enable-libopenh264 --ld=lld-link --target-os=win64 --enable-cross-compile --toolchain=msvc --host-cc=clang.exe --extra-libs=ucrt.lib --extra-libs=vcruntime.lib --extra-libs=oldnames.lib --strip=llvm-strip --disable-stripping --host-extralibs= --enable-gpl --enable-libx264 --enable-libx265 --enable-libaom --enable-libsvtav1 --enable-libxml2 --enable-pic --enable-shared --disable-static --enable-version3 --enable-zlib --pkg-config=/d/bld/ffmpeg_1666310407352/_build_env/Library/bin/pkg-config
library license: GPL version 3 or later
libavcodec     59. 37.100
libavdevice    59.  7.100
libavfilter     8. 44.100
libavformat    59. 27.100
libavutil      57. 28.100
libswresample   4.  7.100
libswscale      6.  7.100
  • FFmpeg:
ffmpeg version 5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
built with clang version 15.0.2
configuration: --prefix=/d/bld/ffmpeg_1666310407352/_h_env/Library --cc=clang.exe --cxx=clang++.exe --nm=llvm-nm --ar=llvm-ar --disable-doc --disable-openssl --enable-demuxer=dash --enable-hardcoded-tables --enable-libfreetype --enable-libfontconfig --enable-libopenh264 --ld=lld-link --target-os=win64 --enable-cross-compile --toolchain=msvc --host-cc=clang.exe --extra-libs=ucrt.lib --extra-libs=vcruntime.lib --extra-libs=oldnames.lib --strip=llvm-strip --disable-stripping --host-extralibs= --enable-gpl --enable-libx264 --enable-libx265 --enable-libaom --enable-libsvtav1 --enable-libxml2 --enable-pic --enable-shared --disable-static --enable-version3 --enable-zlib --pkg-config=/d/bld/ffmpeg_1666310407352/_build_env/Library/bin/pkg-config
libavutil      57. 28.100 / 57. 28.100
libavcodec     59. 37.100 / 59. 37.100
libavformat    59. 27.100 / 59. 27.100
libavdevice    59.  7.100 / 59.  7.100
libavfilter     8. 44.100 /  8. 44.100
libswscale      6.  7.100 /  6.  7.100
libswresample   4.  7.100 /  4.  7.100
libpostproc    56.  6.100 / 56.  6.100

Research

I have done the following: