Bureaucrats, cc_docs_admin, cc_staff
2,306
edits
Niagara: Difference between revisions
Marked this version for translation
No edit summary |
(Marked this version for translation) |
||
| Line 12: | Line 12: | ||
scientific codes running at scale, energy efficiency, and network and storage preformance and capacity. | scientific codes running at scale, energy efficiency, and network and storage preformance and capacity. | ||
<!--T:4--> | |||
The user experience on Niagara will be similar to that on Graham | The user experience on Niagara will be similar to that on Graham | ||
and Cedar, but specific instructions on how to use the Niagara system | and Cedar, but specific instructions on how to use the Niagara system | ||
| Line 17: | Line 18: | ||
flux at present (February 2018). | flux at present (February 2018). | ||
<!--T:5--> | |||
Niagara is an allocatable resource in the 2018 [https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions/ Resource Allocation Competition] (RAC 2018), which comes into effect on April 4, 2018. | Niagara is an allocatable resource in the 2018 [https://www.computecanada.ca/research-portal/accessing-resources/resource-allocation-competitions/ Resource Allocation Competition] (RAC 2018), which comes into effect on April 4, 2018. | ||
<!--T:6--> | |||
[https://youtu.be/EpIcl-iUCV8 Niagara installation update at the SciNet User Group Meeting on February 14th, 2018] | [https://youtu.be/EpIcl-iUCV8 Niagara installation update at the SciNet User Group Meeting on February 14th, 2018] | ||
<!--T:7--> | |||
[https://youtu.be/EpIcl-iUCV8 Niagara installation time-lag video] | [https://youtu.be/EpIcl-iUCV8 Niagara installation time-lag video] | ||
<!--T:3--> | =Niagara system specifications= <!--T:3--> | ||
<!--T:8--> | |||
* 1500 nodes, each with 40 Intel Skylake cores at 2.4GHz, for a total of 60,000 cores. | * 1500 nodes, each with 40 Intel Skylake cores at 2.4GHz, for a total of 60,000 cores. | ||
* 192 GB of RAM per node. | * 192 GB of RAM per node. | ||
| Line 36: | Line 40: | ||
* 685 kW power consumption. | * 685 kW power consumption. | ||
=Attached storage systems= | =Attached storage systems= <!--T:9--> | ||
{| class="wikitable sortable" | {| class="wikitable sortable" | ||
|- | |- | ||
| Line 66: | Line 70: | ||
|} | |} | ||
=High-performance interconnect= | =High-performance interconnect= <!--T:10--> | ||
<!--T:11--> | |||
The Niagara system has an EDR Infiniband network in a so-called | The Niagara system has an EDR Infiniband network in a so-called | ||
'Dragonfly+' topology, with four wings. Each wing (of 375 nodes) has | 'Dragonfly+' topology, with four wings. Each wing (of 375 nodes) has | ||
| Line 73: | Line 78: | ||
adaptive routing, which alleviates network congestion. | adaptive routing, which alleviates network congestion. | ||
=Node characteristics= | =Node characteristics= <!--T:12--> | ||
<!--T:13--> | |||
* CPU: 2 sockets with 20 Intel Skylake cores (2.4GHz, AVX512), for a total of 40 cores per node | * CPU: 2 sockets with 20 Intel Skylake cores (2.4GHz, AVX512), for a total of 40 cores per node | ||
* Computational perfomance: 3 TFlops (theoretical maximum) | * Computational perfomance: 3 TFlops (theoretical maximum) | ||
| Line 82: | Line 88: | ||
* Operating system: Linux CentOS 7 | * Operating system: Linux CentOS 7 | ||
=Scheduling= | =Scheduling= <!--T:14--> | ||
<!--T:15--> | |||
The Niagara system will use the slurm scheduler to run jobs. The basic scheduling commands will therefore be similar as those for Cedar and Graham, with a few differences: | The Niagara system will use the slurm scheduler to run jobs. The basic scheduling commands will therefore be similar as those for Cedar and Graham, with a few differences: | ||
<!--T:16--> | |||
* Scheduling will be by node only. This means jobs will always need to use 40 cores per job. | * Scheduling will be by node only. This means jobs will always need to use 40 cores per job. | ||
* Asking for specific amounts of memory will not be necessary and is discouraged; all nodes have the same amount of memory (192GB minus some operating system overhead). | * Asking for specific amounts of memory will not be necessary and is discouraged; all nodes have the same amount of memory (192GB minus some operating system overhead). | ||
<!--T:17--> | |||
Details, such as how to request burst buffer usagein jobs, are still being worked out. | Details, such as how to request burst buffer usagein jobs, are still being worked out. | ||
=Software= | =Software= <!--T:18--> | ||
<!--T:19--> | |||
* Module-based software stack. | * Module-based software stack. | ||
* Both the standard Compute Canada software stack as well as system-specific software tuned for the system will be available. | * Both the standard Compute Canada software stack as well as system-specific software tuned for the system will be available. | ||