How to use MPI scatter and gather with array

Question

I am new to MPI and has written the following program using C language. Instead of using pointers, I would like to set up my array as shown below. My first array element reads correctly, after that, it won't read array elements. Can you please tell me if this is not the correct way of using scatter and gather
Following is the Result I get:

$ mpicc test.c -o test
$ mpirun -np 4 test
1. Processor 0 has data 0 1 2 3
2. Processor 0 has data 0
3. Processor 0 doubling the data, now has 5
2. Processor 1 has data 32767
3. Processor 1 doubling the data, now has 5
2. Processor 2 has data -437713961
3. Processor 2 doubling the data, now has 5
2. Processor 3 has data 60
3. Processor 3 doubling the data, now has 5
4. Processor 0 has data: 5 1 2 3

Correct Result should be:

$ mpicc test.c -o test
$ mpirun -np 4 test
1. Processor 0 has data 0 1 2 3
2. Processor 0 has data 0
3. Processor 0 doubling the data, now has 5
2. Processor 1 has data 1
3. Processor 1 doubling the data, now has 5
2. Processor 2 has data 2
3. Processor 2 doubling the data, now has 5
2. Processor 3 has data 3
3. Processor 3 doubling the data, now has 5
4. Processor 0 has data: 5 5 5 5

Any help would be greatly appreciated. Following code is run using 4 processors:

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    int size, rank;

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    int globaldata[4]; /*wants to declare array this way*/
    int localdata[4]; /*without using pointers*/

    int i;
    if (rank == 0) {

        for (i = 0; i < size; i++)
            globaldata[i] = i;

        printf("1. Processor %d has data: ", rank);
        for (i = 0; i < size; i++)
            printf("%d ", globaldata[i]);
        printf("\n");
    }

    MPI_Scatter(globaldata, 1, MPI_INT, &localdata, 1, MPI_INT, 0, MPI_COMM_WORLD);

    printf("2. Processor %d has data %d\n", rank, localdata[rank]);
    localdata[rank]= 5;
    printf("3. Processor %d now has %d\n", rank, localdata[rank]);

    MPI_Gather(&localdata, 1, MPI_INT, globaldata, 1, MPI_INT, 0, MPI_COMM_WORLD);

    if (rank == 0) {
        printf("4. Processor %d has data: ", rank);
        for (i = 0; i < size; i++)
            printf("%d ", globaldata[i]);
        printf("\n");
    }


    MPI_Finalize();
    return 0;
}

Solution

Your setup and your scatter is in principle ok. Your problem is in printing, as you have misunderstood a detail of scatter/gather here.

When scattering the 4-element array, each process gets only one element (as you define with the 2nd and 5th arguments of the MPI_Scatter call()). This element is stored in the 0-index of the local array. It is actually a scalar.

In general, you may scatter very big arrays and each process may still have to process a big local array. In these cases it is essential to correctly calculate the global indices and the local indices.

Assume the following toy problem: you want to scatter the array [1 2 3 4 5 6] to two processes. Proc0 should have the [1 2 3] part and the Proc1 should have the [4 5 6] part. In this case, the global array has size 6 and the local arrays have size 3. The Proc0 gets the global elements 0, 1, 2 and assigns them to its local 0, 1, 2. The Proc1 gets the global elements 3, 4, 5 and assigns them to its local 0, 1, 2.

Probably you will understand this concept better when you learn about the MPI_Scatterv which doesn't assume the same number of local elements for every process.

This version of your code seems to work:

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    int size, rank;

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    int globaldata[4];/*wants to declare array this way*/
    int localdata;/*without using pointers*/

    int i;
    if (rank == 0) {

        for (i=0; i<size; i++)
            globaldata[i] = i;

        printf("1. Processor %d has data: ", rank);
        for (i=0; i<size; i++)
            printf("%d ", globaldata[i]);
        printf("\n");
    }

    MPI_Scatter(globaldata, 1, MPI_INT, &localdata, 1, MPI_INT, 0, MPI_COMM_WORLD);

    printf("2. Processor %d has data %d\n", rank, localdata);
    localdata= 5;
    printf("3. Processor %d now has %d\n", rank, localdata);

    MPI_Gather(&localdata, 1, MPI_INT, globaldata, 1, MPI_INT, 0, MPI_COMM_WORLD);

    if (rank == 0) {
        printf("4. Processor %d has data: ", rank);
        for (i=0; i<size; i++)
            printf("%d ", globaldata[i]);
        printf("\n");
    }


    MPI_Finalize();
    return 0;
}

Enjoy learning MPI! :-)

Comments

[Newlearner]

Thank you so much for showing with an example. Everything was clear and concept and all is well understood. Thank you.

[Newlearner]

I have a similar question in the following post. Can you please help me out with that too: stackoverflow.com/questions/40083332/…