Bureaucrats, cc_docs_admin, cc_staff, rsnt_translations
2,837
edits
MPI: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 311: | Line 311: | ||
|lang="c" | |lang="c" | ||
|contents= | |contents= | ||
#include <stdio.h> | |||
#include <mpi.h> | |||
#define BUFMAX 81 | |||
int main(int argc, char *argv[]) | |||
{ | |||
char outbuf[BUFMAX], inbuf[BUFMAX]; | char outbuf[BUFMAX], inbuf[BUFMAX]; | ||
int rank, size; | int rank, size; | ||
| Line 339: | Line 339: | ||
MPI_Finalize(); | MPI_Finalize(); | ||
return(0); | return(0); | ||
} | |||
}} | }} | ||
</tab> | </tab> | ||
| Line 347: | Line 347: | ||
|lang="fortran" | |lang="fortran" | ||
|contents= | |contents= | ||
program phello2 | |||
implicit none | implicit none | ||
| Line 377: | Line 377: | ||
call MPI_FINALIZE(ierr) | call MPI_FINALIZE(ierr) | ||
end program phello2 | |||
}} | }} | ||
</tab> | </tab> | ||
| Line 385: | Line 385: | ||
Compile this program and run it using 2, 4, and 8 processes. While it certainly seems to be working as intended, there is a hidden problem here. The MPI standard does not ''guarantee'' that <tt>MPI_Send</tt> returns before the message has been delivered. Most implementations ''buffer'' the data from <tt>MPI_Send</tt> and return without waiting for it to be delivered. But if it were not buffered, the code we've written would deadlock: Each process would call <tt>MPI_Send</tt> and then wait for its neighbour process to call <tt>MPI_Recv</tt>. Since the neighbour would also be waiting at the <tt>MPI_Send</tt> stage, they would all wait forever. Clearly there ''is'' buffering in the libraries on our systems since the code did not deadlock, but it is poor design to rely on this. The code could fail if used on a system in which there is no buffering provided by the library. Even where buffering is provided, the call might still block if the buffer fills up. | Compile this program and run it using 2, 4, and 8 processes. While it certainly seems to be working as intended, there is a hidden problem here. The MPI standard does not ''guarantee'' that <tt>MPI_Send</tt> returns before the message has been delivered. Most implementations ''buffer'' the data from <tt>MPI_Send</tt> and return without waiting for it to be delivered. But if it were not buffered, the code we've written would deadlock: Each process would call <tt>MPI_Send</tt> and then wait for its neighbour process to call <tt>MPI_Recv</tt>. Since the neighbour would also be waiting at the <tt>MPI_Send</tt> stage, they would all wait forever. Clearly there ''is'' buffering in the libraries on our systems since the code did not deadlock, but it is poor design to rely on this. The code could fail if used on a system in which there is no buffering provided by the library. Even where buffering is provided, the call might still block if the buffer fills up. | ||
[ | [~]$ mpicc -Wall phello2.c -o phello2 | ||
[ | [~]$ mpirun -np 4 ./phello2 | ||
[P_0] process 3 said: "Hello, world! from process 3 of 4"] | [P_0] process 3 said: "Hello, world! from process 3 of 4"] | ||
[P_1] process 0 said: "Hello, world! from process 0 of 4"] | [P_1] process 0 said: "Hello, world! from process 0 of 4"] | ||
| Line 456: | Line 456: | ||
How do we rewrite our "Hello, World!" program to make it safe? A common solution to this kind of problem is to adopt an odd-even pairing and perform the communication in two steps. Since in our example communication is a rotation of data one rank to the right, we should end up with a safe program if all even ranked processes execute a send followed by a receive, while all odd ranked processes execute a receive followed by a send. The reader can easily verify that the sends and receives are properly paired avoiding any possibility of deadlock. | How do we rewrite our "Hello, World!" program to make it safe? A common solution to this kind of problem is to adopt an odd-even pairing and perform the communication in two steps. Since in our example communication is a rotation of data one rank to the right, we should end up with a safe program if all even ranked processes execute a send followed by a receive, while all odd ranked processes execute a receive followed by a send. The reader can easily verify that the sends and receives are properly paired avoiding any possibility of deadlock. | ||
<tabs> | |||
<tab name="C"> | |||
| | {{File | ||
| | |name=phello3.c | ||
|lang="c" | |||
|contents= | |||
#include <stdio.h> | #include <stdio.h> | ||
#include <mpi.h> | #include <mpi.h> | ||
| Line 499: | Line 501: | ||
return(0); | return(0); | ||
} | } | ||
</ | }} | ||
</tab> | |||
<tab name="Fortran"> | |||
{{File | |||
|name=phello3.f90 | |||
| | |lang="fortran" | ||
| | |contents= | ||
program phello3 | program phello3 | ||
| Line 544: | Line 546: | ||
end program phello3 | end program phello3 | ||
</ | }} | ||
</tab> | |||
</tabs> | |||
Is there still a problem here if the number of processors is odd? It might seem so at first as process 0 (which is even) will be sending while process N-1 (also even) is trying to send to 0. But process 0 is originating a send that is correctly paired with a receive at process 1. Since process 1 (odd) begins with a receive, that transaction is guaranteed to complete. When it does, process 0 will proceed to receive the message from process N-1. There may be a (very small!) delay, but there is no chance of a deadlock. | Is there still a problem here if the number of processors is odd? It might seem so at first as process 0 (which is even) will be sending while process N-1 (also even) is trying to send to 0. But process 0 is originating a send that is correctly paired with a receive at process 1. Since process 1 (odd) begins with a receive, that transaction is guaranteed to complete. When it does, process 0 will proceed to receive the message from process N-1. There may be a (very small!) delay, but there is no chance of a deadlock. | ||
[ | [~]$ mpicc -Wall phello3.c -o phello3 | ||
[ | [~]$ mpirun -np 16 ./phello3 | ||
[P_1] process 0 said: "Hello, world! from process 0 of 16"] | [P_1] process 0 said: "Hello, world! from process 0 of 16"] | ||
[P_2] process 1 said: "Hello, world! from process 1 of 16"] | [P_2] process 1 said: "Hello, world! from process 1 of 16"] | ||