OpenMP set_num_threads() is not working

Question

I am writing a parallel program using OpenMP in C++.

I want to control the number of threads in the program using omp_set_num_threads(), but it does not work.

#include <iostream>
#include <omp.h>
#include "mpi.h"

using namespace std;

int myrank;
int groupsize;
double sum;
double t1,t2;
int n = 10000000;

int main(int argc, char *argv[])
{
    MPI_Init( &argc, &argv);
    MPI_Comm_rank( MPI_COMM_WORLD, &myrank );
    MPI_Comm_size(MPI_COMM_WORLD,&groupsize);

    omp_set_num_threads(4);

    sum = 0;
    #pragma omp for  reduction(+:sum)
    for (int i = 0; i < n; i++)
        sum+= i/(n/10);

    cout<<"sum="<<sum<<endl;
    cout<<"threads="<<omp_get_num_threads()<<endl;

    MPI_Finalize();
    return 0;
}

The program outputs:

sum = 4.5e+007
threads=1

How to control the number of threads?

Solution 1

Besides calling omp_get_num_threads() outside of the parallel region in your case, calling omp_set_num_threads() still doesn't guarantee that the OpenMP runtime will use exactly the specified number of threads. omp_set_num_threads() is used to override the value of the environment variable OMP_NUM_THREADS and they both control the upper limit of the size of the thread team that OpenMP would spawn for all parallel regions (in the case of OMP_NUM_THREADS) or for any consequent parallel region (after a call to omp_set_num_threads()). There is something called dynamic teams that could still pick smaller number of threads if the run-time system deems it more appropriate. You can disable dynamic teams by calling omp_set_dynamic(0) or by setting the environment variable OMP_DYNAMIC to false.

To enforce a given number of threads you should disable dynamic teams and specify the desired number of threads with either omp_set_num_threads():

omp_set_dynamic(0);     // Explicitly disable dynamic teams
omp_set_num_threads(4); // Use 4 threads for all consecutive parallel regions
#pragma omp parallel ...
{
    ... 4 threads used here ...
}

or with the num_threads OpenMP clause:

omp_set_dynamic(0);     // Explicitly disable dynamic teams
// Spawn 4 threads for this parallel region only
#pragma omp parallel ... num_threads(4)
{
    ... 4 threads used here ...
}

Solution 2

The omp_get_num_threads() function returns the number of threads that are currently in the team executing the parallel region from which it is called. You are calling it outside of the parallel region, which is why it returns 1.

Solution 3

According to the GCC manual for omp_get_num_threads:

In a sequential section of the program omp_get_num_threads returns 1

So this:

cout<<"sum="<<sum<<endl;
cout<<"threads="<<omp_get_num_threads()<<endl;

Should be changed to something like:

#pragma omp parallel
{
    cout<<"sum="<<sum<<endl;
    cout<<"threads="<<omp_get_num_threads()<<endl;
}

The code I use follows Hristo's advice of disabling dynamic teams, too.

Solution 4

I was facing the same problem . Solution is given below

Right click on Source Program > Properties > Configuration Properties > C/C++ > Language > Now change Open MP support flag to Yes....

You will get the desired result.

View more solutions

Comments

[Nurlan]

But I couldn't get any acceleration by changing the number of threads and also by setting the omp_set_dynamic(0)??

[Hristo Iliev]

First, the serial version takes 50 ms on my CPU. You cannot expect speedup for such fast codes because of the OpenMP overhead. Put 100x more interations in the loop. Second, you are missing parallel region in your original code. You have written #pragma omp for ... while it should be #pragma omp parallel for .... Third, you are mixing MPI and OpenMP. Are you sure you know exactly what you are doing?

[manatttta]

@HristoIliev may a user dynamically control the number of threads? i.e. instead of using a hardcoded 4. use a variable?

[Hristo Iliev]

@manatttta, yes. The value gets passed to the OpenMP runtime library each time the parallel region is encountered during the execution of the program, therefore using a variable works as expected.

[Omar Haque]

With the num_threads OpenMP clause option, is it possible to use a variable rather than hard code a number? E.g. int n = 4; #pragma omp parallel ... num_threads(n) ?

[Hristo Iliev]

@OmarHaque, yes, the num_threads clause accepts variables too, so it is possible to control the number of threads in run time.

[Guibao Wang]

I can confirm this. GNU implementation of omp documents this as well. gcc.gnu.org/onlinedocs/libgomp/…