CHAPTER 2: A Simple Parallel Program

2.1 The SPMD Patternlet

The first directory we will explore is the patternlets directory. A patternlet is a small program that succinctly illustrates common patterns in parallel programming. The first patternlet we will study is Single Program, Multiple Data (SPMD). Let’s explore this application by traveling to the spmd directory under the patternlets folder:

cd patternlets/spmd
ls

Typing ls should reveal the files:

  • Makefile

  • spmd2.c

Let’s start by examining spmd2.c, a C program that uses OpenMP pragmas to make it easy to run a portion of the program on multiple threads:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
/* spmd2.c
* ... illustrates the SPMD pattern in OpenMP,
* using the commandline arguments
* to control the number of threads.
*
* Joel Adams, Calvin College, November 2009.
*
* Usage: ./spmd2 [numThreads]
*
* Exercise:
* - Compile & run with no commandline args
* - Rerun with different commandline args,
* until you see a problem with thread ids
* - Fix the race condition
* (if necessary, compare to 02.spmd)
*/

#include <stdio.h>
#include <omp.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    int id, numThreads;

    printf("\n");
    if (argc > 1) {
        omp_set_num_threads( atoi(argv[1]) );
    }

    #pragma omp parallel
    {
        id = omp_get_thread_num();
        numThreads = omp_get_num_threads();
        printf("Hello from thread %d of %d\n", id, numThreads);
    }

    printf("\n");
    return 0;
}

The omp parallel pragma on line 30 directs that the block of code within the curly braces be run on separate threads. Prior to Line 30, the program is run serially. When line 30 executes, OpenMP generates a a team of threads (known as forking). Each thread is assigned its own id and runs separate copies of the code between the curly braces. At the end of the pragma’s scope (line 35), OpenMP combines all the threads together to a single-threaded process (known as joining). Conceptually, the process looks like the following:

../_images/ForkJoin_SPMD.png

The code in main() up until line 30 is run in one thread on one core; the fork that occurs between lines 31 and 35 is shown in the middle of the diagram.

The last couple of lines of code (lines 37-28) are run as a single-threaded process after all the threads have joined back together.

2.2 Running the Program

To run the code, we must first compile it. Use the make command in the spmd directory to compile the code:

make

The make command compiles the code in spmd2.c according to the commands in the Makefile, producing an executable called spmd2. To run the spmd2 program on 4 threads, use the following command:

./spmd2 4

2.3 Race Conditions

The following video will help you understand what is going on:

Try and answer the following question:

2.4 Fixing the code

In this case, the race condition can be avoided by ensuring that each thread has its own private copy of the id and numThreads variables.

Edit the spmd2.c file in the spmd directory (the full path is shown below):

CSinParallel/RaspberryPiBasics/patternlets/spmd/spmd2.c

The steps to fix the code are:

  1. comment out line 23 by adding a // at the very beginning of the line

  2. fix lines 32 and 33 to be full variable declarations (see listing below)

Your changed code should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
/* spmd2.c
* ... illustrates the SPMD pattern in OpenMP,
* using the commandline arguments
* to control the number of threads.
*
* Joel Adams, Calvin College, November 2009.
*
* Usage: ./spmd2 [numThreads]
*
* Exercise:
* - Compile & run with no commandline args
* - Rerun with different commandline args,
* until you see a problem with thread ids
* - Fix the race condition
* (if necessary, compare to 02.spmd)
*/

#include <stdio.h>
#include <omp.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    //int id, numThreads;

    printf("\n");
    if (argc > 1) {
        omp_set_num_threads( atoi(argv[1]) );
    }

    #pragma omp parallel
    {
        int id = omp_get_thread_num();
        int numThreads = omp_get_num_threads();
        printf("Hello from thread %d of %d\n", id, numThreads);
    }

    printf("\n");
    return 0;
}

You can recompile and re-run the code by re-using the make command and invoking the spmd2 executable with 4 threads:

make
./spmd2 4
You have attempted of activities on this page