Skip to content
GitLab
Open
Issue created by Administrator@rootContributor6 of 6 checklist items completed6/6 checklist items

Segfault when trying to use h264_v4l2m2m encoder on Raspberry Pi 4 with 64-bit OS

Created by: samiamlabs

Overview

Hi!

I'm trying to stream video with decent resolution and framerate over WebRTC (aiortc) with my Raspberry Pi 4. This pull request adding support for h264_omx seemed really promising at first: https://github.com/aiortc/aiortc/pull/488 Unfortunately, it seems that h264_omx is considered deprecated and won't be supported on arm64. (At the very least, I can't get it to work...)

I tried to just swap the codec name in the code related to the pull request to aiortc I mentioned above from "h264_omx" to "h264_v4l2m2m". The encoder context seems to be created without any exceptions (some warnings get printed though) but segfaults when trying to run self.codec.encode(frame).

Should it be possible to use the "h264_v4l2m2m" codec or is it simply not supported at this point?

Expected behavior

Well... I was hoping that h264_v4l2m2m would work as a drop-in replacement for h264_omx and allow me to stream video to remote control a robot using a Raspberry Pi 4.

Actual behavior

Hard to debug segfault which seems to happen outside the python code

Traceback:

gdb -ex r --args python3 webcam.py 
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...
(No debugging symbols found in python3)
Starting program: /usr/bin/python3 webcam.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[Detaching after fork from child process 1707]
======== Running on http://0.0.0.0:8080 ========
(Press CTRL+C to quit)
INFO:aiohttp.access:127.0.0.1 [06/Jul/2021:21:19:35 +0000] "GET / HTTP/1.1" 200 979 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
INFO:aiohttp.access:127.0.0.1 [06/Jul/2021:21:19:35 +0000] "GET /client.js HTTP/1.1" 200 2426 "http://localhost:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
[New Thread 0xffffea2901e0 (LWP 1708)]
INFO:aioice.ice:Connection(0) Check CandidatePair(('172.19.0.2', 47684) -> ('192.168.1.60', 45032)) State.FROZEN -> State.WAITING
INFO:aiohttp.access:127.0.0.1 [06/Jul/2021:21:19:36 +0000] "POST /offer HTTP/1.1" 200 2557 "http://localhost:8080/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
Connection state is connecting
INFO:aioice.ice:Connection(0) Check CandidatePair(('172.19.0.2', 47684) -> ('192.168.1.60', 45032)) State.WAITING -> State.IN_PROGRESS
INFO:aioice.ice:Connection(0) Check CandidatePair(('172.19.0.2', 47684) -> ('192.168.1.60', 45032)) State.IN_PROGRESS -> State.SUCCEEDED
INFO:aioice.ice:Connection(0) ICE completed
Connection state is connected
[New Thread 0xffffe9a161e0 (LWP 1709)]
--------- Creating h264 encoder ----------
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-isp' on card 'bcm2835-isp'
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-isp' on card 'bcm2835-isp'
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-isp' on card 'bcm2835-isp'
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-isp' on card 'bcm2835-isp'
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-codec' on card 'bcm2835-codec-isp'
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-codec' on card 'bcm2835-codec-encode'
[h264_v4l2m2m @ 0xffffe4002550] Using device /dev/video11
[h264_v4l2m2m @ 0xffffe4002550] driver 'bcm2835-codec' on card 'bcm2835-codec-encode'
[h264_v4l2m2m @ 0xffffe4002550] Failed to set number of B-frames
[h264_v4l2m2m @ 0xffffe4002550] Failed to set number of B-frames
[h264_v4l2m2m @ 0xffffe4002550] Failed to set gop size
[h264_v4l2m2m @ 0xffffe4002550] h264 profile not found
[h264_v4l2m2m @ 0xffffe4002550] Encoder adjusted: qmin (0), qmax (51)
[h264_v4l2m2m @ 0xffffe4002550] Failed to set minimum video quantizer scale
[h264_v4l2m2m @ 0xffffe4002550] Failed to set maximum video quantizer scale
--Type <RET> for more, q to quit, c to continue without paging--

Thread 2 "python3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffea2901e0 (LWP 1708)]
0x0000fffff5a34440 in ?? () from /lib/aarch64-linux-gnu/libavcodec.so.58
(gdb) bt
#0  0x0000fffff5a34440 in  () at /lib/aarch64-linux-gnu/libavcodec.so.58
#1  0x0000fffff5a35f54 in  () at /lib/aarch64-linux-gnu/libavcodec.so.58
#2  0x0000fffff562cba0 in avcodec_send_frame () at /lib/aarch64-linux-gnu/libavcodec.so.58
#3  0x0000ffffec058f34 in __pyx_f_2av_5codec_7context_12CodecContext__send_frame_and_recv (__pyx_v_self=0xffffe9a1c430, __pyx_v_frame=0xffffe9a24340)
    at src/av/codec/context.c:5334
#4  0x0000ffffec06247c in __pyx_f_2av_5codec_7context_12CodecContext_encode
    (__pyx_v_self=0xffffe9a1c430, __pyx_skip_dispatch=<optimized out>, __pyx_optional_args=<optimized out>) at src/av/codec/context.c:6484
#5  0x0000ffffec0583f8 in __pyx_pf_2av_5codec_7context_12CodecContext_22encode (__pyx_v_frame=<optimized out>, __pyx_v_self=0xffffe9a1c430)
    at src/av/codec/context.c:6685
#6  __pyx_pw_2av_5codec_7context_12CodecContext_23encode (__pyx_v_self=0xffffe9a1c430, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>)
    at src/av/codec/context.c:6662
#7  0x000000000048b7c4 in  ()
#8  0x00000000004fccac in _PyEval_EvalFrameDefault ()
#9  0x000000000047a648 in  ()
#10 0x000000000055dd8c in  ()
#11 0x00000000004fcb00 in _PyEval_EvalFrameDefault ()
#12 0x0000000000493328 in  ()
#13 0x00000000005011d4 in _PyEval_EvalFrameDefault ()
#14 0x00000000004f9dd4 in _PyEval_EvalCodeWithName ()
#15 0x00000000005952dc in _PyFunction_Vectorcall ()
#16 0x0000000000493408 in  ()
#17 0x00000000005916e4 in PyObject_Call ()
#18 0x00000000004fe054 in _PyEval_EvalFrameDefault ()
#19 0x00000000005950e8 in _PyFunction_Vectorcall ()
#20 0x00000000004fccac in _PyEval_EvalFrameDefault ()
#21 0x00000000005950e8 in _PyFunction_Vectorcall ()
#22 0x00000000005916e4 in PyObject_Call ()
#23 0x00000000004fe054 in _PyEval_EvalFrameDefault ()
--Type <RET> for more, q to quit, c to continue without paging--
#24 0x00000000005950e8 in _PyFunction_Vectorcall ()
#25 0x00000000004fccac in _PyEval_EvalFrameDefault ()
#26 0x00000000005950e8 in _PyFunction_Vectorcall ()
#27 0x00000000004fccac in _PyEval_EvalFrameDefault ()
#28 0x00000000005950e8 in _PyFunction_Vectorcall ()
#29 0x0000000000493504 in  ()
#30 0x00000000005916e4 in PyObject_Call ()
#31 0x00000000006100dc in  ()
#32 0x00000000006403ec in  ()
#33 0x0000fffff7e234fc in start_thread (arg=0xffffffffcbff) at pthread_create.c:477
#34 0x0000fffff7f1f67c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

Investigation

Python debugging in VSCode and GDB

Reproduction

Run the webcam.py example in aiortc (modified to select h264 using the approach from this issue: https://github.com/aiortc/aiortc/issues/502) on Ubuntu 20.04.2 LTS for arm64 on a Raspberry Pi 4. Also, change the codec name from "h264_omx" to "h264_v4l2m2m" in https://github.com/aiortc/aiortc/blob/main/src/aiortc/codecs/h264.py Got an error related to the "profile" option after that so changed it to "profile:v" based on https://video.stackexchange.com/questions/2634/how-do-i-specify-baseline-h264-profile-using-ffmbc

I used a source build of FFmpeg 4.4 to get rid of the green overlay issue in 4.2.4 that is distributed by apt on focal (explained more here: https://www.willusher.io/general/2020/11/15/hw-accel-encoding-rpi4) but I don't know if that is relevant for this issue.

I can provide a Dockerfile for building FFmpeg etc if someone wants it.

Versions

  • OS: Ubuntu 20.04.2 LTS for arm64 inside Docker
  • PyAV runtime:
PyAV v8.0.3
git origin: git@github.com:PyAV-Org/PyAV
git commit: unknown-commit
library configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
library license: GPL version 2 or later
libavcodec     58. 54.100
libavdevice    58.  8.100
libavfilter     7. 57.100
libavformat    58. 29.100
libavutil      56. 31.100
libswresample   3.  5.100
libswscale      5.  5.100
  • PyAV build:
Binary installation from pip3
  • FFmpeg:
ffmpeg version 031c0cb Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
configuration: --prefix=/home/dyno/ffmpeg_build --bindir=/home/dyno/bin --enable-gpl --enable-nonfree --arch=aarch64 --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libwebp --enable-libdrm
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100

Research

I have done the following: