VP9 encoding limited to 4 threads?

Question

I am considering to use VP9 to encode my BluRays in the future, since its an open source codec. But I cannot get Handbrake or ffmpeg use more than 50% (4) of my (8) cores. The encoding time is therefore much worse than x264/5 which uses all cores.

In Handbrake I just set encoder to VP9 and CQ19. There is no difference if I add threads 8, threads 16 or threads 64 in the parameters field.

Testing ffmpeg in the command line (-c:v libvpx-vp9 -crf 19 -threads 16 -tile-columns 6 -frame-parallel 1 -speed 0) also does not use any more cpu threads.

Is the current encoder not capable of encoding on more than 4 threads or am I doing something wrong?

• Linux Mint 18
• handbrake 0.10.2+ds1-2build1
• ffmpeg 2.8.10-0ubuntu0.16.04.1
• libvpx3 1.5.0-2ubuntu1

Solution 1

Libvpx uses tile threading, which means you can at most have as many threads as the number of tiles. The -tile-columns option is in log2 format (so -tile-columns 6 means 64 tiles), but is also limited by the framesize. The exact details are here, it basically means that max_tiles = max(1, exp2(floor(log2(sb_cols)) - 2)), where sb_cols = ceil(width / 64.0). You can write a small script to calculate the number of tiles for a given horizontal resolution:

Width: 320 (sb_cols: 5), min tiles: 1, max tiles: 1
Width: 640 (sb_cols: 10), min tiles: 1, max tiles: 2
Width: 1280 (sb_cols: 20), min tiles: 1, max tiles: 4
Width: 1920 (sb_cols: 30), min tiles: 1, max tiles: 4
Width: 3840 (sb_cols: 60), min tiles: 1, max tiles: 8

So even for 1080p (1920 horizontal pixels), you only get 4 tiles max, so 4 threads max, i.e. a bitstream limitation. To get 8 tiles, you need at least a width of 1985 pixels (2048-64+1, which gives sb_cols=32). To get more threads than the max. number of tiles at a given resolution, you need frame-level multithreading, which libvpx doesn't implement. Other encoders, like x265/x264, do implement this.

EDIT

As some people in comments and below have already commented, more recent versions of libvpx support -row-mt 1 to enable tile row multi-threading. This can increase the number of tiles by up to 4x in VP9 (since the max number of tile rows is 4, regardless of video height). To enable this, use -tile-rows N where N is the number of tile rows in log2 units (so -tile-rows 1 means 2 tile rows and -tile-rows 2 means 4 tile rows). The total number of active threads will then be equal to $tile_rows * $tile_columns.

Solution 2

According to webmproject.org libvpx VP9 encoder supports multi-threading within a single column tile since 1.7.0 tag.

All you have to do is to set -row-mt 1

i.e ffmpeg -i input.mp4 -c:v libvpx-vp9 -b:v 1000K -threads 8 -speed 4 -row-mt 1 -f webm /tmp/test

Comments

[kellerkindt]

Thanks for your explanation, do you know any other method to get VP9 encoded with a reasonable performance?

[Ronald S. Bultje]

Your best bet is to use higher values for the -speed parameter. If that's not sufficient or quality suffers too much for your intended use case, I'm afraid you're out of luck...

[Programster]

Apparently since ffmpeg 3.3/libvpx 1.6.2, there is now support -row-mt=1 option which provides better encoding parallelization and more than 4 threads for 1080p content. Threadripper guys go nuts! - groups.google.com/a/webmproject.org/forum/#!topic/codec-deve‌​l/…

[MrPotatoHead]

@RonaldS.Bultje thanks for taking the time to post this great info! FBO of anyone who hasn't reviewed the link in his answer, Ronald is one of the VP9 code authors (ffmpeg code)! Nice to get this type of info so concisely AND from the source! :)

[MrPotatoHead]

Google's documentation on VP9 libvpx codec is poor (and that's putting it mildly). There is only one reference I've found in it regarding the -row-mt parameter, and no mention of -tile-rows. developers.google.com/media/vp9/live-encoding

[Vitaly Zdanevich]

-speed 4 is necessary here?