Bureaucrats, cc_docs_admin, cc_staff
2,318
edits
MPI-IO: Difference between revisions
Jump to navigation
Jump to search
Marked this version for translation
No edit summary |
(Marked this version for translation) |
||
| Line 2: | Line 2: | ||
<translate> | <translate> | ||
== Description == | == Description == <!--T:1--> | ||
'''MPI-IO''' is a family of [[MPI]] routines that makes it possible to do file read and write operations in parallel. MPI-IO is a part of the MPI-2 standard. The main advantage of MPI-IO is that it allows, in a simple and efficient fashion, to read and to write data that is partitioned on multiple processes, to one file that is common to all processes. This is particularly useful when the manipulated data are vectors or matrices that are cut up in a structured manner between the different processes involved. This page gives a few guidelines on the use of MPI-IO and some references to more complete documentation. | '''MPI-IO''' is a family of [[MPI]] routines that makes it possible to do file read and write operations in parallel. MPI-IO is a part of the MPI-2 standard. The main advantage of MPI-IO is that it allows, in a simple and efficient fashion, to read and to write data that is partitioned on multiple processes, to one file that is common to all processes. This is particularly useful when the manipulated data are vectors or matrices that are cut up in a structured manner between the different processes involved. This page gives a few guidelines on the use of MPI-IO and some references to more complete documentation. | ||
== Using MPI-IO == | == Using MPI-IO == <!--T:2--> | ||
=== Operations through offsets === | === Operations through offsets === <!--T:3--> | ||
<!--T:4--> | |||
The simplest way to perform parallel read and write operations is to use offsets. Each process can read from or write to the file with a defined offset. This can be done in two operations ([http://www.open-mpi.org/doc/current/man3/MPI_File_seek.3.php MPI_File_seek] followed by [http://www.open-mpi.org/doc/current/man3/MPI_File_read.3.php MPI_File_read] or by [http://www.open-mpi.org/doc/current/man3/MPI_File_write.3.php MPI_File_write]), or even in a single operation ([http://www.open-mpi.org/doc/current/man3/MPI_File_read_at.3.php MPI_File_read_at] or [http://www.open-mpi.org/doc/current/man3/MPI_File_write_at.3.php MPI_File_write_at]). Usually the offset is computed as a function of the process rank. | The simplest way to perform parallel read and write operations is to use offsets. Each process can read from or write to the file with a defined offset. This can be done in two operations ([http://www.open-mpi.org/doc/current/man3/MPI_File_seek.3.php MPI_File_seek] followed by [http://www.open-mpi.org/doc/current/man3/MPI_File_read.3.php MPI_File_read] or by [http://www.open-mpi.org/doc/current/man3/MPI_File_write.3.php MPI_File_write]), or even in a single operation ([http://www.open-mpi.org/doc/current/man3/MPI_File_read_at.3.php MPI_File_read_at] or [http://www.open-mpi.org/doc/current/man3/MPI_File_write_at.3.php MPI_File_write_at]). Usually the offset is computed as a function of the process rank. | ||
</translate> | </translate> | ||
| Line 63: | Line 64: | ||
}} | }} | ||
<translate> | <translate> | ||
=== Using views === | === Using views === <!--T:5--> | ||
<!--T:6--> | |||
Using views, each process can ''see'' a section of the file, as if it were the entire file. In this way it is no longer necessary to compute the file offsets as a function of the process rank. Once the view is defined, it is then a lot simpler to perform operations on this file, without risking conflicts with operations performed by other processes. A view is defined using the function [http://www.open-mpi.org/doc/current/man3/MPI_File_set_view.3.php MPI_File_set_view]. Here is a program identical to the previous example, but using views instead. | Using views, each process can ''see'' a section of the file, as if it were the entire file. In this way it is no longer necessary to compute the file offsets as a function of the process rank. Once the view is defined, it is then a lot simpler to perform operations on this file, without risking conflicts with operations performed by other processes. A view is defined using the function [http://www.open-mpi.org/doc/current/man3/MPI_File_set_view.3.php MPI_File_set_view]. Here is a program identical to the previous example, but using views instead. | ||
</translate> | </translate> | ||
| Line 132: | Line 134: | ||
}} | }} | ||
<translate> | <translate> | ||
<!--T:7--> | |||
'''Warning!''' Some file systems do not support file locks. Consequently some operations are not possible, in particular using views on disjoint file sections. | '''Warning!''' Some file systems do not support file locks. Consequently some operations are not possible, in particular using views on disjoint file sections. | ||
== References == | == References == <!--T:8--> | ||
<!--T:9--> | |||
* [http://www.open-mpi.org/doc/current/ OpenMPI documentation] | * [http://www.open-mpi.org/doc/current/ OpenMPI documentation] | ||
* [https://support.scinet.utoronto.ca/wiki/images/0/01/Parallel_io_course.pdf Course on parallel I/O] (PDF) | * [https://support.scinet.utoronto.ca/wiki/images/0/01/Parallel_io_course.pdf Course on parallel I/O] (PDF) | ||
</translate> | </translate> | ||