Technical background

In our previous blog, we introduced concurrent The reason why we are considering MPI and other schemes to realize python parallel computing is to implement python computing tasks and parallel computing task scheduling hierarchically. In concurrent and multiprocessing schemes, our python computing tasks and scheduling tasks are integrated, and there is a big limitation that it is impossible to operate across nodes, which requires a high degree of customization of tasks and environment. The MPI scheme considers the problem of communication between multiple nodes at the early stage of design, and this hierarchical task scheduling solution is actually more reasonable in terms of architecture. Computing people only need to consider how to execute tasks in a single process. As for how to parallel and schedule tasks, that's what the upper MPI should do.

Installation of mpi4py

It is recommended to use conda for direct installation. If pip is used for installation, some environment dependent problems may occur:

$ conda install mpi4py
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/dechin/anaconda3

  added / updated specs:
    - mpi4py


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    mpi-1.0                    |            mpich          13 KB  defaults
    mpi4py-3.0.3               |   py38h028fd6f_0         572 KB  defaults
    mpich-3.3.2                |       hc856adb_0         3.8 MB  defaults
    ------------------------------------------------------------
                                           Total:         4.4 MB

The following NEW packages will be INSTALLED:

  mpi                pkgs/main/linux-64::mpi-1.0-mpich
  mpi4py             pkgs/main/linux-64::mpi4py-3.0.3-py38h028fd6f_0
  mpich              pkgs/main/linux-64::mpich-3.3.2-hc856adb_0

The following packages will be UPDATED:

  ca-certificates                      2021.9.30-h06a4308_1 --> 2021.10.26-h06a4308_2


Proceed ([y]/n)? y


Downloading and Extracting Packages
mpi4py-3.0.3         | 572 KB    | ############################################## | 100% 
mpich-3.3.2          | 3.8 MB    | ############################################## | 100% 
mpi-1.0              | 13 KB     | ############################################## | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

After installation, you can check whether the installation is successful through python3 -c "from mpi4py import MPI". Let's take a look at some specific use cases.

Use case

First, understand the basic usage of mpi. If we use mpirun -n 3 python3 test.py to run a program, we will send a different rank to each different test.py. The range of this rank starts from 0. For example, in the following case, we use get_ The rank () method can obtain the rank id passed by mpi, so that the process can know its process number. We can write the tasks to be performed under each number:

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
print('My rank is ',rank)

The implementation effect is as follows:

$ mpirun -n 4 python3 mpi_test.py
My rank is  2
My rank is  1
My rank is  0
My rank is  3

Of course, because each task is at the same level, the rank id of mpi is also issued randomly, which can not be controlled, but we can use the following methods to communicate between processes:

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

if rank == 0:
    idata = 1
    comm.send(idata, dest=1)
    print ('This is process {}'.format(rank), '\nData send to process 1 successfully!')
elif rank == 1:
    idata = comm.recv(source=0)
    print ('This is process {}, data is '.format(rank),idata)

In this case, we sent the data of an integer variable from the process with rank id 0 to the process with rank id 1. Because we don't know when this script will be assigned to rank 0 and rank 1, we need to deal with these two possible situations respectively in the same script. The operation results are as follows:

$ mpirun -n 2 python3 mpi_test.py
This is process 0 
Data send to process 1 successfully!
This is process 1, data is  1

Integer variables are successfully transferred. Of course, there is a more important meaning in this. The process with rank 1 actually depends on the process with rank 0 in time series. We must complete the tasks in rank 0 before we can perform the tasks in rank 1. This function may be frequently used in practical applications, Especially when tasks are interdependent. Of course, the communication between processes can pass not only integer variables, but also other types, such as a dictionary or a numpy array:

from mpi4py import MPI
import numpy as np

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

if rank == 0:
    numData = 10
    comm.send(numData, dest=1)
    data = np.linspace(0.0, 3.14, numData)
    comm.Send(data, dest=1)
    print ('This is process {}'.format(rank), '\nData send to process 1 successfully!')

elif rank == 1:
    numData = comm.recv(source=0)
    print('Number of data to receive: ', numData)

    data = np.empty(numData, dtype='d')
    comm.Recv(data, source=0)
    print ('This is process {}, data is '.format(rank),data)

The operation results are as follows:

$ mpirun -n 2 python3 mpi_test.py
This is process 0 
Data send to process 1 successfully!
Number of data to receive:  10
This is process 1, data is  [0.         0.34888889 0.69777778 1.04666667 1.39555556 1.74444444
 2.09333333 2.44222222 2.79111111 3.14      ]

You can see that the result was successfully received.

Summary summary

In this article, we do not introduce many applications of MPI. In fact, in ordinary parallel or distributed tasks, it is enough to master the task processing process of each process and the communication method between processes. Generally speaking, MPI is a very general and efficient parallel computing software. With these professional parallel task scheduling software, we can focus on the code and algorithm of professional tasks without paying too much attention to the scheduling and allocation of parallel tasks.

Copyright notice

The starting link of this article is: https://www.cnblogs.com/dechinphy/p/mpi4py.html

Author ID: DechinPhy

For more original articles, please refer to: https://www.cnblogs.com/dechinphy/

Special links for rewards: https://www.cnblogs.com/dechinphy/gallery/image/379634.html

Tencent cloud column synchronization: https://cloud.tencent.com/developer/column/91958