Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Intel QuickSync for ffmpeg in a headless server

Graphical Environments, Managers, Multimedia & Desktop questions.
Post Reply
Message
Author
sulla
Posts: 1
Joined: 2022-05-23 17:12

Intel QuickSync for ffmpeg in a headless server

#1 Post by sulla »

Dear all!

I would like to transcode video files on a headless server using intel QuickSync hardware acceleration in ffmpeg, but I cannot get it work. Could you point me to into the right direction?

I am running:

Code: Select all

$ cat /proc/version
Linux version 5.10.0-14-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.113-1 (2022-04-29)
My CPU is an Intel G3258 (haswell, pentium gold) processor and according to the intel specs it should support QuickSync hardware acceleration for MPEG-2 (h.262) und AVC (h.264) video streams.

ffmpeg shipping with debian bullseye should support VAAPI acceleration

Code: Select all

$ ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
vdpau
vaapi
qsv
drm
opencl
According to https://trac.ffmpeg.org/wiki/Hardware/QuickSync VAAPI-acceleration should work with the i965 driver.

I seem to have the i915 driver in use currently

Code: Select all

$ lspci -k | grep -EA3 'VGA|3D|Display'
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
        Subsystem: Gigabyte Technology Co., Ltd Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller
        Kernel driver in use: i915
        Kernel modules: i915
but the i965 driver is a mesa driver, and I'm running a headless server, only connecting to it via ssh.

I installed the i965 driver, however, via a

Code: Select all

sudo apt install i965-va-driver-shaders
which is what https://wiki.debian.org/HardwareVideoAcceleration told me to do.

just to test the decoding part (which should also work with the non-free drivers), I did one of the tests mentioned in https://trac.ffmpeg.org/wiki/Hardware/VAAPI:

Code: Select all

ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -init_hw_device -hwaccel vaapi -hwaccel_device intel -i ~/test.ts -f null -
where test.ts is a transport-stream in h.264 format (from a CCTV-camera, in this case)
this command fails, with the following messages:

Code: Select all

ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[AVHWDeviceContext @ 0x55bcb2cb2280] No VA display found for device /dev/dri/renderD128.
Device creation failed: -22.
Failed to set value 'vaapi=intel:/dev/dri/renderD128' for option 'init_hw_device': Invalid argument
Error parsing global options: Invalid argument
turns out, vaapi seems not to be offered by ffmpeg:

Code: Select all

$ ffmpeg -decoders| grep vaapi
ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
however, h264_qsv seems a supported codec in ffmpeg (for encoding as well as for decoding), and that should be fine:

Code: Select all

$ ffmpeg -decoders| grep h264_qsv
ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
[...]
 V....D h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
So I tried it hte h264_qsv-way, but it doesn't work either:

Code: Select all

ffmpeg -init_hw_device qsv:hw -hwaccel qsv -hwaccel_device qsv -c:v h264_qsv -i ~/test.ts -f null -
giving me the following errors:

Code: Select all

ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Device creation failed: -542398533.
Failed to set value 'qsv:hw' for option 'init_hw_device': Generic error in an external library
Error parsing global options: Generic error in an external library
doing a software-decoding test on that same file gives:

Code: Select all

$ ffmpeg -i /srv/data/data/CCTV/full-videos/Schildis-aussen/2022-05-22_21.27.ts -f null -
gives:

Code: Select all

ffmpeg version 4.3.4-0+deb11u1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr --extra-version=0+deb11u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '~/test.ts':
  Duration: 00:10:00.04, start: 1.400000, bitrate: 2670 kb/s
  Program 1
    Metadata:
      service_name    : IP Camera Video
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuvj420p(pc, progressive), 2304x1536, 20 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Data: bin_data ([6][0][0][0] / 0x0006)
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: wrapped_avframe, yuvj420p, 2304x1536, q=2-31, 200 kb/s, 20 fps, 20 tbn, 20 tbc
    Metadata:
      encoder         : Lavc58.91.100 wrapped_avframe
[null @ 0x55b5b6adf800] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 16 >= 16
[...]
[null @ 0x55b5b6adf800] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 11985 >= 11985
frame=11886 fps=310 q=-0.0 Lsize=N/A time=00:09:59.50 bitrate=N/A speed=15.6x
video:6222kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
so, transcoding in software works but is veeery slow and uses 70% CPU-load.

what do I do wrong, how can I make use of QuickSync to accelerate ffmpeg?
regards,
sulla

Segfault
Posts: 993
Joined: 2005-09-24 12:24
Has thanked: 5 times
Been thanked: 17 times

Re: Intel QuickSync for ffmpeg in a headless server

#2 Post by Segfault »

You are confusing encoding and decoding. Hardware encoding with Intel, in particular with older Intel chipsets is no use IMHO. I tried with my i7-7700K. Decoding works wonderfully. For encoding I need to load additional CPU microcode. OK, that done I had encoding available, but it was still slow and the quality of picture was literally not usable. The only area of usage would have been encoding some security camera output just to have the stream watchable live over the internet. In this case quality is not paramount, important is to have real time encoding. Granted, it was some time ago and there has been some Intel microcode update after my tests. So maybe it looks better now. I resorted to nVidia TU116 when it became affordable and it beats any Intel hands down. Quality-wise and speed-wise. For comparison, using pure software HEVC encoding I got 2-3 FPS with certain HD video. With my nVidia I get around 200 FPS. Almost 100 times faster. Nothing more is to be said.

Post Reply