how to reduce cpu usage of ffmpeg?

Question

hi i am restreaming an hls stream as a hls stream

SETLOCAL
:loop

ffmpeg -thread_queue_size 32768 -i "http://xx/636.m3u8" -f hls -hls_time 5 -hls_list_size 5 -hls_allow_cache 0 -hls_flags delete_segments -segment_list_flags +live -hls_base_url "../ts/" -hls_segment_filename "C:\nginx\html\ts\lig-%random%%random%-%%3d.svgz" -y "C:\nginx\html\hls\lig1.m3u8" > log.txt 

goto loop

but it uses %15-20 of cpu i have to make 16 streams like that in same server but i can't.

cant i make a configuration like just downloading .ts files cloning the m3u8?

Solution 1

By default, FFmpeg will try to reencode the input, even if you don't use any encoding related parameters.

If you want to just copy the stream as is, you should try the streamcopy function:

-c:v copy -c:a copy

By copying the incoming streams, you can skip the encoding process entirely.

So your command would look like this:

ffmpeg -thread_queue_size 32768 -i "http://xx/636.m3u8" -f hls -c:v copy -c:a copy -hls_time 5 -hls_list_size 5 -hls_allow_cache 0 -hls_flags delete_segments -segment_list_flags +live -hls_base_url "../ts/" -hls_segment_filename "C:\nginx\html\ts\lig-%random%%random%-%%3d.svgz" -y "C:\nginx\html\hls\lig1.m3u8" > log.txt

(Not sure if it'll work, though.)

---

When you need to re-encode the incoming stream, you should consider to add some encoding parameters to the command.

By default, ffmpeg tries to match the source parameters and quality, which not always the most optimal in live applications.

The "veryfast, superfast and ultrafast" h264 presets is a good start to get some performance boost.

-c:v h264 -preset:v ultrafast

You can also fiddle with CRF (Constant Rate Factor) encoding, higher bitrates, etc.

More about H264 enoding: https://trac.ffmpeg.org/wiki/Encode/H.264

Solution 2

Although this question is already considered answered, I believe it lacks an answer that doesn't involve tuning the command parameters and yet can be used with other tools or commands and not just FFmpeg.

The CPUlimit tool is designed exactly for this purpose and will effectively work not just for FFmpeg but also for any other CPU intensive process you want to reduce this resource utilization.

The most important argument of CPUlimit is -l, which allows you to specify the CPU percentage the process will be allowed to use (as its top limit).

Please, also observe that this limit depends on the number of CPUs/Cores/Threads the machine has, for example, if the machine has 8 CPUs, CPULimit -l valid values will range from 0 to 800 (0 being useless, of course, and 800 meaning not limiting the process at all as 800% means all the machine processing capacity).

Example for using half the capacity for an 8 CPUs machine:

cpulimit -l 400 ffmpeg ...

Hope this helps for this specific question as well as similar needs in the future.

Update: 2020/04/19

The original CPUlimit project seems to have moved to github and is now being maintained by user denji here. I'm leaving the original link because the new project must be compiled and does not offer a binary download.

Also, regarding the original question, if you're trying to limit a bash script (or anything that forks a child process) make sure to add the option -i or --include-children and specify the location of the script if it's not in PATH.

Example update:

cpulimit -l 400 -i ./script.sh

Solution 3

-re (input)

Read input at native frame rate. Mainly used to simulate a grab device, or live input stream (e.g. when reading from a file). Should not be used with actual grab devices or live input streams (where it can cause packet loss).

By default FFmpeg attempts to read the input(s) as fast as possible. This option will slow down the reading of the input(s) to the native frame rate of the input(s). It is useful for real-time output (e.g. live streaming).

Comments

[Arya Same]

when i type -preset:v ultrafast to command line ffmpeg stucks freezes

[Gergely Lukacsy]

If you're reencoding, you have to define the codec too e.g.: -c:v h264!!! Also, cut down the > log part, it prohibits ffmpeg to tell what's wrong.

[4pie0]

Works for me, -c:v copy -c:a copy and -segment_list_flags +live reduced 99% CPU usage to 1%.

[Peter]

This is the perfect answer. It keeps my laptop from shutting down due to critical core temperature. Thanks a lot!

[Janis Peisenieks]

cpulimit was also the exact thing I was looking for. Thanks!

[Giacomo1968]

Late to learning about this, but when the answer example refers to 8 CPUs does that mean a single 8 core CPU or 8 physical CPUs? I am assuming 8 cores, but just asking for clarification.

[Marcelo]

@JakeGould, actually it would make no difference for the purpose of the cpulimit tool. What matters is how much CPUs the Linux kernel "sees". As a matter of fact, even multiple threads counts here, a quad-core HT will be seen as 8 CPUs by the linux kernel.

[Giacomo1968]

@Marcelo Thanks! Great answer and appreciate the follow up.