FFMPEG: m4a to mp3 conversion using multiple cores

Question

I use the following command to convert m4a format to mp3

ffmpeg -i audio.m4a -acodec libmp3lame audio.mp3

I've 32 x86 cores, however libmp3lame processes in a single thread. I know libmp3lame does not support multithreading, thus I'm open to other alternates that can be executed in ubuntu CLI.

If audio.m4a is 2hours long video, that usually takes > 3minutes and speed appears to be 45x to 50x.

My primary goal is to convert multiple youtube videos in mp3 format within seconds.

Update 1:

Since I'm using 32 cores CPU, when there's only one video conversion it utilizes just one core. So in that cases how to use multiple cores, to get output faster. I want to achieve the maximum from the CPU. Also if FFmpeg is not the answer, is there any other way.

Solution 1

GNU Parallel sounds exactly like what you are looking for.

Here is an example of how you could use GNU Parallel (assuming you already have a directory with all your *.m4a files named videos) using your original commands:

find /path/to/videos -name "*.m4a" | parallel 'ffmpeg -i {} -acodec libmp3lame {.}.mp3'

Two things to explain:

1: The output of the find command (i.e. the list of .m4a files to convert) will be piped into the parallel command, which will in turn (by default) execute as many jobs as there are cores. In essence it is splitting the output from the find command, and distributing it across the different jobs to work on.

2: Per the GNU Parallel Tutorial, {} is where your arguments will go, {.} will substitute in your arguments and remove the terminal extension (in this case the .m4a), and your output will be the original file name but with .mp3 extension.

NOTE: This example will write all your converted files to where your original files are. If this is an issue, then merely pass a directory to store the converted files in the argument string containing the ffmpeg command:

find /path/to/videos -name "*.m4a" | parallel 'ffmpeg -i {} -acodec libmp3lame /path/to/converted/vids/{.}.mp3'

Solution 2

Since the encoder is single-threaded only, in order to make the encoding the most efficient on a multi-core system, all you can do is launch several processes in parallel.

For example, if you're in a shell:

ffmpeg -i input1.mp4 -c:a libmp3lame output1.mp3 &
ffmpeg -i input2.mp4 -c:a libmp3lame output2.mp3 &
…

The & puts the command into the background. The commands will be run in parallel and the load should be distributed among your CPU cores.

Solution 3

There are multi-threaded implementations of lame mp3 encoder on the internet. I could track one here

https://github.com/dheller1/lame_pthreads

If you have large storage space, you could use ffmpeg to convert all m4a files into wav files and then point lame_pthreads to that folder.

Comments

[Johny Pie]

Yes dear, I know. What I want is make the single process happen in multithread.

[Johny Pie]

I've updated my question, please check. Thank you

[slhck]

You can't magically make a single-threaded process execute in multiple threads. There is a standalone multithread version of LAME available, but I cannot guarantee that it works well: github.com/dheller1/lame_pthreads

[Johny Pie]

have a look at this website onlinevideoconverter.com/es it does conversions almost instantly.

[Scott - Слава Україні]

(1) So, if (as stated) it takes the OP three minutes to process a file, with your technique he will be able to process 32 files in three minutes.  That may be the best he can get, but that's not what he's looking for — he's looking for a way to process one file in under ten seconds.  (2) Please don't use ls to generate filenames for another process to use (or advise people to do so).