Number of threads used by Intel TBB

Question

How does Intel TBB choose the number of threads to used for a parallel section?

Is there some kind of specification available?

Solution 1

As of TBB Version 2.2 the task scheduler will be automatically initialized and on runtime take care of the numbers of threads to use, if you manually want to change that number, you can use one of the following methods:

When you create the scheduler, you can specify the number of threads as

tbb::task_scheduler_init init(nthread);

else you can use

tbb::task_scheduler_init init(tbb::task_scheduler_init::automatic);

In this case, tbb scheduler creates as many threads as your CPU cores

Solution 2

Letting TBB decide the number of threads in the pool is the recommended option - it will usually create as many worker threads as there are logical CPUs on the machine - see Class reference for tbb::task_scheduler_init.

It's not easy to find out how many worker threads exist or are executing tasks at any given time - this is a deliberate design choice. From Intel's TBB Parallel Programming Course:

How do I know how many threads are available?

Do not ask!

• Not even the scheduler knows how many threads really are available
• There may be other processes running on the machine
• Routine may be nested inside other parallel routines

Comments

[Petr Hudeček]

You must keep the tbb::task_scheduler_init object alive after this, or it will revert back to automatic.

[Wenzel Jakob]

Also, the task scheduler object just affects TBB jobs launched from the current thread. When creating further jobs from another thread, you need another tbb::task_scheduler_init.

[ogrisel]

Is the task scheduler able to detect if it running in a cgroups limited environment (e.g. a docker container?). For instance openmp runtimes are typically not and will tend to cause oversubscription when running in docker containers (e.g. limited to a 2000 mCPU quota on a machine with 32 cores).

[ogrisel]

If several processes are running on the same host (for instance when using multiprocessing in Python to spawn workers), can the TBB scheduler detect the concurrent version of the code running in other processes and automatically tune the number of active threads to avoid oversubscription problems? What if the processes are not started by the same unix user?

[ogrisel]

Replying to myself: TBB can use interprocess communication to coordinate the schedulers and avoid oversubscription. This can be done by setting the ENABLE_IPC environment variables.

[ogrisel]

I have tried to run some numpy / MKL / TBB benchmarks in a script launched with docker run --cpus 2 on a host with 28 cores and I observed quite sever oversubscription related performance degradation.

[ogrisel]

For the later issue with docker, I have filed a report at: github.com/intel/tbb/issues/190

[Dmitry Mikushin]

This is actually not a solution, because given any nthread value, TBB still creates # of threads equal to hardware CPU threads. And, even worse, if you specify nthreads=2, TBB creates a second thread and runs TBB work on it, and not on the master thread. Please see this question for details: stackoverflow.com/questions/59736661/…