Toying with ffmpeg and hardware acceleration is intriguing. Why? because of experimenting, now we know that we can maximize the hardware we had. Why Hardware Acceleration? you might ask again. Using Hardware Acceleration can help accelerate encoding time as it offloads the video decoding process from the CPU to the GPU, which is generally more efficient.

Before we start with the result, let me give you the hardware ingredients (read: specifications):

CPU: Intel Core i5-3570
GPU 0: Intel HD Graphics 2500
GPU 1: AMD Radeon R5 340X
RAM: 16GB DDR3 1600MHz
OS: Windows Server 2022 Datacenter
FFMPEG version: 4 and 7

Why the 2 GPU's you might ask? I can think two reasons for this. The first one is... since the idea of this note entry is to maximize our old hardware, might as well activate that unused Integrated Graphics, right? And the other reason is because we need that Integrated Graphics to use Intel QSV. Intel QSV is Intel Quick Sync Video. It's a feature of Intel's integrated graphics lineup ever since the first gen HD Graphics back in 2010. Click here to learn more. Intel QSV had a Decode and Encode capability, which we will use in this note entry.

Since our hardware is a 3rd generation Intel Core i5, it placed us as "Ivy Bridge" generation. "Ivy Bridge" and "Haswell" generation had similar feature sets, since at the time, Intel was using a "tick-tock" strategy for their CPUs, which meant, a lot of their line up (at the time) didn't really have a significant upgrade for 2 generations.

According to the Wiki, our Ivy Bridge QSV hardware supports Encoding and Decoding on MPEG-2, AVC (or H264). and only Decode support for VC-1 (usually used for Blu-ray). This means, our integrated graphics is able to help us decode and encode videos with that codec.

Why not just use your much newer "AMD Radeon R5 340X" discrete GPU for this (hardware acceleration thingy)?

The answer is: this GPU does NOT support video encoding. However, since it does support video decoding, that's why we will use it capability in tandem with our integrated graphics.

For these experiments, we will try to convert a mkv file with H264 codec, 2.8Mbit/s, Stereo AC3 audio (track 1), file to a mp4 file with H264 codec, 2Mbit/s file, and Stereo AAC Audio.

As for the experiment, this is the results:

FFMPEG 4 with D3D11VA hw decode and qsv igpu encode:

ffmpeg.exe -hwaccel d3d11va -i words.mkv -c:v h264_qsv -b:v 2M -c:a aac words.mp4
ffmpeg version 4.4.1-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'words.mkv':
  Metadata:
    creation_time   : 2020-10-15T12:23:55.000000Z
    ENCODER         : Lavf58.29.100
  Duration: 00:02:52.02, start: 0.000000, bitrate: 2839 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], 29.97 fps, 29.97 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:02:52.015000000
  Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
  Stream #0:2: Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:02:52.000000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'words.mp4':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), nv12(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
    Metadata:
      DURATION        : 00:02:52.015000000
      encoder         : Lavc58.134.100 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
      encoder         : Lavc58.134.100 aac
frame= 5155 fps=196 q=18.0 Lsize=   43849kB time=00:02:52.01 bitrate=2088.3kbits/s speed=6.56x
video:40977kB audio:2700kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.392108%
[aac @ 000002d80e614cc0] Qavg: 184.948

FFMPEG 7 with D3D11VA hw decode and libx264 cpu encode:

ffmpeg -hwaccel d3d11va -i words.mkv -c:v libx264 -b:v 2M -c:a aac words.mp4
ffmpeg version 4.4.1-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'words.mkv':
  Metadata:
    creation_time   : 2020-10-15T12:23:55.000000Z
    ENCODER         : Lavf58.29.100
  Duration: 00:02:52.02, start: 0.000000, bitrate: 2839 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], 29.97 fps, 29.97 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:02:52.015000000
  Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
  Stream #0:2: Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:02:52.000000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000022f50c0ccc0] using SAR=1/1
[libx264 @ 0000022f50c0ccc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0000022f50c0ccc0] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0000022f50c0ccc0] 264 - core 164 r3075 66a5bc1 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'words.mp4':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), nv12(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
    Metadata:
      DURATION        : 00:02:52.015000000
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
      encoder         : Lavc58.134.100 aac
frame= 5155 fps= 56 q=-1.0 Lsize=   45094kB time=00:02:52.01 bitrate=2147.6kbits/s speed=1.85x
video:42207kB audio:2700kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.413861%
[libx264 @ 0000022f50c0ccc0] frame I:35    Avg QP: 8.17  size: 48905
[libx264 @ 0000022f50c0ccc0] frame P:1554  Avg QP:10.75  size: 17056
[libx264 @ 0000022f50c0ccc0] frame B:3566  Avg QP:13.23  size:  4207
[libx264 @ 0000022f50c0ccc0] consecutive B-frames:  1.2% 18.5%  3.3% 77.0%
[libx264 @ 0000022f50c0ccc0] mb I  I16..4: 47.7% 22.2% 30.1%
[libx264 @ 0000022f50c0ccc0] mb P  I16..4: 16.9%  8.5%  2.5%  P16..4: 33.8% 13.7%  7.5%  0.0%  0.0%    skip:17.1%
[libx264 @ 0000022f50c0ccc0] mb B  I16..4:  1.1%  0.6%  0.1%  B16..8: 31.1%  4.9%  0.6%  direct: 8.8%  skip:52.8%  L0:46.2% L1:46.4% BI: 7.5%
[libx264 @ 0000022f50c0ccc0] final ratefactor: 14.56
[libx264 @ 0000022f50c0ccc0] 8x8 transform intra:30.3% inter:44.5%
[libx264 @ 0000022f50c0ccc0] coded y,uvDC,uvAC intra: 26.0% 63.3% 40.4% inter: 11.0% 28.4% 3.4%
[libx264 @ 0000022f50c0ccc0] i16 v,h,dc,p: 61% 22% 11%  6%
[libx264 @ 0000022f50c0ccc0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 14% 50%  2%  2%  3%  2%  2%  2%
[libx264 @ 0000022f50c0ccc0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 20% 21%  4%  6%  5%  5%  4%  3%
[libx264 @ 0000022f50c0ccc0] i8c dc,h,v,p: 52% 19% 21%  7%
[libx264 @ 0000022f50c0ccc0] Weighted P-Frames: Y:17.0% UV:13.0%
[libx264 @ 0000022f50c0ccc0] ref P L0: 62.7%  7.5% 20.2%  8.7%  0.9%
[libx264 @ 0000022f50c0ccc0] ref B L0: 83.6% 12.9%  3.5%
[libx264 @ 0000022f50c0ccc0] ref B L1: 96.7%  3.3%
[libx264 @ 0000022f50c0ccc0] kb/s:2010.16
[aac @ 0000022f50c0ea40] Qavg: 184.948

FFMPEG 7 with D3D11VA hw decode and qsv igpu encode

ffmpeg -hwaccel d3d11va -i words.mkv -c:v h264_qsv -b:v 2M -c:a aac words.mp4
ffmpeg version 4.4.1-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11.2.0 (Rev1, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'words.mkv':
  Metadata:
    creation_time   : 2020-10-15T12:23:55.000000Z
    ENCODER         : Lavf58.29.100
  Duration: 00:02:52.02, start: 0.000000, bitrate: 2839 kb/s
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], 29.97 fps, 29.97 tbr, 1k tbn, 180k tbc (default)
    Metadata:
      DURATION        : 00:02:52.015000000
  Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 224 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
  Stream #0:2: Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s
    Metadata:
      title           : Surround
      DURATION        : 00:02:52.000000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
  Stream #0:1 -> #0:1 (ac3 (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'words.mp4':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), nv12(tv, bt709, progressive), 928x698 [SAR 1:1 DAR 464:349], q=2-31, 2000 kb/s, 29.97 fps, 30k tbn (default)
    Metadata:
      DURATION        : 00:02:52.015000000
      encoder         : Lavc58.134.100 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:02:52.021000000
      encoder         : Lavc58.134.100 aac
frame= 5155 fps=196 q=18.0 Lsize=   43844kB time=00:02:52.01 bitrate=2088.1kbits/s speed=6.53x
video:40973kB audio:2700kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.392149%
[aac @ 0000020a36ef3c80] Qavg: 184.948

What we can observe:

Just 2.4 watt power draw from iGPU while we use it to encode the video!

Our AMD graphics also got some load, since we use Windows's D3D11 Video Acceleration to decode the video

Key points:

Between FFMPEG 4 and 7, we did not see any performance difference when using "hwaccel" flag (for decode) and "qsv" (qsv encode) or "libx264" (cpu encode). Results stay consistent between the two ffmpeg versions.
QSV were able to help accelerate our video conversion process approximately 3.53 times faster than the libx264 encode speed. That's quite fast. Also it's using the hardware we already have laying around!
While QSV using iGPU to offload the video process from CPU to our iGPU, it also does it efficiently. We're talking only 2 watts of added power under load. That much of energy efficiency is impressive if you ask me.

That's it! Utilizing the hardware we already have. Turns out, ancient can still rocks!

Thanks for coming to my TED talk (reading my experiment and notes entry)

Hardware Accelerated Video Encoding on Ancient iGPU: Is It Worth It?