National systems
| Sharing: Public | New Systems | Author:CC Migration team |
Compute
Overview
GP2 and GP3 are almost identical systems with some minor differences in the actual mix of large memory, small memory and GPU nodes.
| Name | Description | Approximate Capacity | Availability |
|---|---|---|---|
| CC-Cloud Resources (GP1) | Cloud | 7,640 cores | In production (integrated with west.cloud) |
| GP2 | heterogeneous, general-purpose cluster (Serial and small parallel jobs) Small cloud partition |
25,000 cores | April, 2017 |
| GP3 | heterogeneous, general-purpose cluster (Serial and small parallel jobs) Small cloud partition |
25,000 cores | April, 2017 |
| LP | a cluster designed for large parallel jobs | 60,000 cores | Late 2017 |
Note that GP1, GP2 and LP will all have large, high-performance attached storage.
GP2 (SFU)
The GP2 system evaluation is not yet completed (as of early November 2016). Anticipated specifications, based on SFU's RFP and bids received, include the following. This information is not guaranteed and might not be complete. It is provided for planning purposes.
| Parallel filesystem: | Approximately 4PB usable capacity for temporary (/scratch) storage. Aggregate performance of approximately 40GB/s. Available to all nodes.
|
External persistent storage: 10PB or more of storage for persistent (/project and other) storage. Available to compute nodes, but not designed for parallel workloads.
High performance interconnect: Low-latency high-performance fabric connecting all nodes and temporary storage. The design of GP2 is to support multiple simultaneous parallel jobs of at least 1024 cores in a fully non-blocking manner. Jobs larger than 1024 cores would be less well-suited for the topology.
Node types and characteristics:
"Base" compute nodes: Over 500 nodes with 128GB of memory.
"Large" compute nodes: Over 100 nodes with 256GB of memory.
"Bigmem512" and "Bigmem1500" nodes, each with 512GB and 1.5TB of memory respectively. 24 of each type.
All of the above nodes will have 16 cores/socket (32 cores/node), with an anticipated frequency of 2.1Ghz.
"GPU base" nodes: Over 100 nodes with x4 GPUs. 12 cores/socket (24 cores/node) with an anticipated frequency of 2.2Ghz. GPUs on a dual PCI root.
"GPU large" nodes. Approximately 30 nodes with the same configuration as "GPU base," but a single PCI root.
"Bigmem3000" nodes, each with 3TB of memory. These are 4-socket nodes with 8 cores/socket. 4 nodes.
All of the above nodes will have local (on-node) storage.
Login nodes
Gateway and data transfer nodes
Compute Canada is not currently able to disclose the specific GPU model or specifications. The RFP used NVIDIA K80 as a baseline specification.
The total GP2 system is anticipated to have over 25,000 cores, 900 nodes, and 500 GPUs.
The delivery and installation schedule is not yet known, but the procurement team has confidence that the system will be in production for the start of the allocations year on April 1, 2017.
GP3 (Waterloo)
GP3 overview: GP3 system evaluation is not yet completed (as of early November 2016). Anticipated specifications, based on Waterloo's RFP and bids received, include the following. This information is NOT GUARANTEED and might not be complete. It is provided for planning purposes.
GP3 overview: A heterogeneous cluster providing capabilities for parallel and serial workloads, as well as a small cloud partition.
Parallel filesystem: Approximately 4PB usable capacity for temporary (/scratch) storage. Aggregate performance of approximately 40GB/s. Available to all nodes. External persistent storage: 10PB or more of storage for persistent (/project and other) storage. Available to compute nodes, but not designed for parallel workloads. High performance interconnect: Low-latency high-performance fabric connecting all nodes and temporary storage. The design of GP3 is to support multiple simultaneous parallel jobs of at least 1024 cores in a fully non-blocking manner. Jobs larger than 1024 cores would be less well-suited for the topology.
Node types and characteristics:
"Base" compute nodes: Over 500 nodes with 128GB of memory. "Large" compute nodes: Over 100 nodes with 256GB of memory. "Bigmem512" nodes, each with 512GB of memory "Bigmem3000" nodes, each with 3TB of memory
GPU nodes
All of the above nodes will have local (on-node) storage.
Login nodes
Gateway and data transfer nodes
Compute Canada is not currently able to disclose the specific GPU or CPU model or specifications. The RFP used NVIDIA K80 as a baseline specification.
The total GP3 system is anticipated to have over 20,000 cores.
The delivery and installation schedule is not yet known.
National Data Cyberinfrastructure (NDC)
| Type | Location | Capacity | Availability | Comments |
|---|---|---|---|---|
| Nearline (Long-term) File Storage |
SFU and Waterloo
|
15 PB each | Late Autumn 2016 |
|
| Object Store |
All 4 sites
|
Small to start (A few PB usable) |
Late 2017 |
|
| Special Purpose | All 4 sites | ~3.5 PB | Specialized migration plans |
|
Note that due to Silo decommissioning Dec.31/2016 it will be necessary to provide interim storage while the NDC is developed. See Migration2016:Silo for details.