NPRG058: Advanced Programming in Parallel Environment

Lab practice 08 - MPI

Please, login to parlab (there is no need for GPUs today).

Hello world

Copy code and scripts from /home/_teaching/advpara/labs/08-mpi-intro to your home. Use attached ./build.sh script to build the "Hello world" testing application.

Checkout attached ./run.sh script and try to execute it. Adjust HOSTS and PROCS variables at will. Note that there are several important SLURM arguments, most notably:

• -n how many processes should be spawned
• -N how many host nodes will be allocated for the processes (each host will accommodate n/N processes)
• -c number of CPUs allocated for a process (usually 1, unless you combine MPI with another parallel technology)

Also make sure you allocate memory (or memory-per-cpu properly) and attribute -m affects how the processes are distributed over the nodes.

Manual reduction

Modify the hello world application to compute a trivial reduction. Generate a vector of sufficient size (e.g., 1M) in each process (optionally, use rank ID as the random seed). Compute minimum from all the processes.

The minimum should be aggregated on rank #0 (send local minima from all processes to #0 and perform manual reduction there) and printed to stdout.

Note that the minimum may be dependent on how many processes you are spawning. Use debug printouts to find out your code is working correctly.

Stretch goal 1: Experiment with asynchronous message API.

Stretch goal 2: Find two smallest numbers from the same (generated) dataset.