News

FFmpeg 9.0 "Lei" Is Here: This Is What's New

ConvertMyVideo

— 3 min read
Share

FFmpeg, the well-known opensource multimedia framework powering everything from streaming platforms to video editors has just released version 9.0, codenamed "Lei". The project notes that a whopping 2200+ commits have been merged compared to outgoing version 8.1. The biggest theme of this release is GPU acceleration, as Vulkan support got a big expansion, although AMD, NVIDIA and Apples' so-called "vendor-specific code" also got major improvements. The new version is now available for download on the official FFmpeg.org website.

Should developers and pipeline maintainers upgrade?

Since not only FFmpeg, but also libavcodec and libavformat got new major version numbers, you might want to hold back a bit. This is because of possible breaking changes, but only if you use FFmpeg as a library, as the command line interface still works as per usual.

The rule of thumb here is to check for two things:

  • Does my code still pass the usual unit/regression tests in my own codebase?
  • Does my code touch changed parts in FFmpeg, in this case NVENC, CELT/Ogg, or Dolby Vision?

What vendor-specific changes have been introduced?

FFmpeg's Vulkan implementation now supports hardware accelerated encoding and decoding for Samsung's APV codec, which is a purpose-built codec for professional-level high-quality video recording and post production, similar to Apple ProRes. The FFV1 Vulkan encoder also gained a 32-bit float RGB path, which matters for lossless GPU-speed compression of camera sensor data used in digital preservation workflows. The panoramic video projection filter now also runs as a Vulkan compute shader, moving a historically CPU-heavy filter onto the GPU which is part of a push to use Vulkan for more use-cases other than just accelerated decoding. Better filter chaining without GPU to CPU round-trips also contribute to speed boosts seen in situations like HDR tone mapping and scaling pipelines.

The Advanced Media Framework by AMD also introduced a list of additions. A frame-rate converter filter has been added to replace CPU-based frame interpolation with hardware acceleration. A video quality enhancement filter has also been added. Hardware memory mapping support has been added as well to reduce CPU to GPU copies in filter chains.

If you are running on Apple hardware and work with ProRes RAW files: you can now use hardware accelerated decoding when working with these files because FFmpeg now can do this through the VideoToolbox framework. This is Apple's framework to directly access hardware encoders and decoders on Apple hardware.

NVIDIA/NVENC users beware! Support for NVIDIA Video Codec SDKs older than version 11.1 have been dropped. Does this mean old GPUs can no longer use NVENC? Not at all. Just make sure you are on driver version 470 (2021) or later. This means you can still use that Kepler, Maxwell or Pascal GPU for hardware accelerated video encoding. Also, if you use libnpp-based filters (like scale_npp or transpose_npp for example) please migrate to their CUDA-native equivalents. They are no longer deprecated, in this release all npp filters are removed.

What new formats are supported in FFmpeg 9.0?

Quite a few:

  • Animated WebP gets a native decoder and a demuxer.
  • HE-AAC with 960-sample frames is now decodable, which is a variant used by DAB+ digital radio as opposed to the more common 1024-sample AAC configuration.
  • LCEVC tracks can now be muxed directly into MP4 files. LCEVC layers an enhancement stream on top of a base video stream, so compatible decoders get higher quality while older decoders can still play the base layer.
  • SMPTE 2094-50 dynamic HDR metadata, which shares functional similarities to HDR10+ is now parsed, written and passed through.
  • A Playdate encoder and muxer was also found newly added. It outputs 1-bit, 400x240 video using delta coding and zlib compression, which is what the small handheld game console is based on.

And also very noteworthy are the changes in Dolby Vision for this release. Profile 7 - used in UHD Blu-ray discs - now has proper handling which means that a new dovi_split bitstream filter can handle the extraction or isolation of the base layer for this profile, which was not possible before.

Are there any improvements in speed?

We are happy to announce that the swscale architecture has been completely rewritten. This means that upscaling, downscaling, colorspace conversion, HDR tone-mapping and more operations like these examples to be much master, which speedups averaging at between 2 to 2.6 times! Everytime you use the scale filter, you will notice the improvements!

Because FFmpeg comes with CPU architecture-bound assembly code, it was worth checking what's new in that area as well. The reported swscale improvements also actively reflect on this, since every CPU architecture got it's own code, with most work done for x86 and ARM, and minor work for RISC-V.

How ConvertMyVideo is going to use FFmpeg 9.0

All of our pipelines are currently built on FFmpeg 8.1.2 as of the creation of this report. We are happy to announce that once we migrate to FFmpeg 9.0, we are able to build a GIF to animated WebP converter. Nothing major unfortunately, but that's not stopping our creativity to make new useful tools for the modern web!

Read more