Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
NetCDF: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
No edit summary |
||
| Line 3: | Line 3: | ||
= General = | = General = | ||
NetCDF (Network Common Data Form) is an interface for array-oriented data access and a library that provides an implementation of the interface. The NetCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. | NetCDF (Network Common Data Form) is: | ||
* an interface for array-oriented data access, and | |||
* a library that provides an implementation of the interface. | |||
The NetCDF library also defines a machine-independent format for representing scientific data. Together, the interface, library, and format support the creation, access, and sharing of scientific data. | |||
There were significant changes to the library with the release of version 4.0. NetCDF 4.x is backwards compatible - it can read files produced by an earlier version but the opposite isn't true. Since | There were significant changes to the library with the release of version 4.0. NetCDF 4.x is backwards compatible - it can read files produced by an earlier version but the opposite isn't true. Since 4.0 was released in 2008, we will not discuss earlier versions here. | ||
* Project web site : https://www.unidata.ucar.edu/software/netcdf | * Project web site : https://www.unidata.ucar.edu/software/netcdf | ||
| Line 14: | Line 17: | ||
== Strengths == | == Strengths == | ||
* The data are independent of the processor architecture ([https://en.wikipedia.org/wiki/Endianness endianness]). | * The data are independent of the processor architecture ([https://en.wikipedia.org/wiki/Endianness endianness]). | ||
* The data are structured in a way that keeps track of all the pertinent information | * The data are structured in a way that keeps track of all the pertinent information, e.g. physical units. | ||
* NetCDF4 can read and write in parallel, if it is built using a parallel version of [[HDF5]]. | * NetCDF4 can read and write in parallel, if it is built using a parallel version of [[HDF5]]. | ||
* Data can be compressed as it is written. | * Data can be compressed as it is written. | ||
| Line 29: | Line 32: | ||
== Environment modules == | == Environment modules == | ||
The following modules providing NetCDF are available on Compute Canada systems via CVMFS: | The following [[Utiliser des modules/en|modules]] providing NetCDF are available on Compute Canada systems via CVMFS: | ||
* '''netcdf''' | * '''netcdf''' | ||
** for linking with | ** for linking with programs containing C instructions only | ||
* '''netcdf-c++''' | * '''netcdf-c++''' | ||
** for linking with | ** for linking with programs containing both C and C++ instructions | ||
* '''netcdf-fortran''' | * '''netcdf-fortran''' | ||
** for linking with | ** for linking with programs containing Fortran instructions | ||
There is also a set of NetCDF modules compiled with MPI support for parallel I/O: | There is also a set of NetCDF modules compiled with [[MPI]] support for parallel I/O: | ||
* '''netcdf-mpi''' | * '''netcdf-mpi''' | ||
** for linking with | ** for linking with programs containing C instructions and calling MPI libraries | ||
* '''netcdf-c++-mpi''' | * '''netcdf-c++-mpi''' | ||
** for linking with | ** for linking with programs containing both C and C++ instructions and calling MPI | ||
* '''netcdf-fortran-mpi''' | * '''netcdf-fortran-mpi''' | ||
** for linking with | ** for linking with programs containing Fortran instructions and calling MPI | ||
Run <code>module avail netcdf</code> to see what versions are currently available with the compiler and MPI modules you have loaded. For a comprehensive list of NetCDF modules, run <code>module -r spider '.*netcdf.*'</code>. | |||
Use <code>module load netcdf/version</code> to set your environment to use your chosen version. For example, to load the NetCDF version 4.1.3 C-based library, do: | |||
{{Command|module load netcdf/4.1.3}} | |||
== Submission scripts == | == Submission scripts == | ||
Please refer to the page "[[Running_jobs|Running jobs]]" for | Please refer to the page "[[Running_jobs|Running jobs]]" for examples of job scripts for the Slurm workload manager. We recommend you include the <code>module load ...</code> command in your job script. | ||
== Linking | == Linking code to NetCDF libraries == | ||
Below are | Below are a few examples showing how to link NetCDF libraries to C and Fortran code: | ||
=== Serial NetCDF === | === Serial NetCDF === | ||
For C code, | |||
{{Commands | {{Commands | ||
|module load netcdf/4.4.1 | |module load netcdf/4.4.1 | ||
| Line 67: | Line 67: | ||
}} | }} | ||
For Fortran code, notice that two libraries are required and their order matters: | |||
{{Commands | {{Commands | ||
|module load netcdf-fortran | |module load netcdf-fortran | ||
| Line 74: | Line 74: | ||
=== Parallel NetCDF === | === Parallel NetCDF === | ||
For C code which calls MPI, | |||
{{Commands | {{Commands | ||
|module load netcdf-mpi | |module load netcdf-mpi | ||
| Line 81: | Line 81: | ||
=== Example === | === Example === | ||
An example demonstrating the use of the NetCDF can be found at the following [https://www.unidata.ucar.edu/software/netcdf/netcdf-4/newdocs/netcdf-tutorial/simple_005fxy_005fwr_002ec.html#simple_005fxy_005fwr_002ec link]. The example program creates a NetCDF file containing a single two-dimensional variable called "data", and labels the two dimensions "x" and "y". | |||
To compile the example: | |||
{{ | {{Commands | ||
|module load netcdf | |||
|gcc ex_netcdf4.c -lnetcdf | |gcc ex_netcdf4.c -lnetcdf | ||
}} | }} | ||