Bureaucrats, cc_docs_admin, cc_staff
2,306
edits
Narval/en: Difference between revisions
no edit summary
(Created page with "=Node Characteristics= {| class="wikitable sortable" ! nodes !! cores !! available memory !! CPU !! storage !! GPU |- | 1109 || rowspan="2"|64 || ~256000M || rowspan="2"|2 x...") |
No edit summary |
||
| Line 56: | Line 56: | ||
=High-performance interconnect= | =High-performance interconnect= | ||
The [InfiniBand] [https://www.nvidia.com/en-us/networking/infiniband/qm8700/ Mellanox HDR] network links together all of the nodes of the cluster. Each hub of 40 HDR ports (200 Gb/s) can connect up to 66 nodes with HDR100 (100 Gb/s) with 33 HDR links divided in two (2) by special cables. The seven (7) remaining HDR links allow the hub to be connected to a rack containing the seven (7) cental HDR InfiniBand hubs. The islands of nodes are therefore connected by a maximum blocking factor of 33:7 (4.7:1). In contrast, the storage servers are connected by a non-blocking network. | The [https://en.wikipedia.org/wiki/InfiniBand InfiniBand] [https://www.nvidia.com/en-us/networking/infiniband/qm8700/ Mellanox HDR] network links together all of the nodes of the cluster. Each hub of 40 HDR ports (200 Gb/s) can connect up to 66 nodes with HDR100 (100 Gb/s) with 33 HDR links divided in two (2) by special cables. The seven (7) remaining HDR links allow the hub to be connected to a rack containing the seven (7) cental HDR InfiniBand hubs. The islands of nodes are therefore connected by a maximum blocking factor of 33:7 (4.7:1). In contrast, the storage servers are connected by a non-blocking network. | ||
In practice the Narval racks contain islands of 48 ou 56 regular CPU nodes. It is therefore possible to run parallel jobs using up to 3584 cores with a non-blocking network. For larger jobs or ones which are distributed in a fragmented manner across the network, the blocking factor is 4.7:1. The inter-connect remains a high-performance one nonetheless. | In practice the Narval racks contain islands of 48 ou 56 regular CPU nodes. It is therefore possible to run parallel jobs using up to 3584 cores with a non-blocking network. For larger jobs or ones which are distributed in a fragmented manner across the network, the blocking factor is 4.7:1. The inter-connect remains a high-performance one nonetheless. | ||