National systems: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
m (trivial paste-o)
No edit summary
Line 32: Line 32:
Note that GP1, GP2 and LP will all have large, high-performance attached storage.
Note that GP1, GP2 and LP will all have large, high-performance attached storage.


===GP2 (SFU)===


The GP2 system evaluation is not yet completed (as of early November 2016).  Anticipated specifications, based on SFU's RFP and bids received, include the following.  This information is '''not guaranteed''' and might not be complete.  It is provided for planning purposes.
{{GP2}}


{| class="wikitable sortable"
{{GP3}}
|-
| '''Parallel filesystem:''' ||
Approximately 4PB usable capacity for temporary (<code>/scratch</code>) storage.<br />
Aggregate performance of approximately 40GB/s.  Available to all nodes.
|-
|'''External persistent storage:'''
||
10PB or more of storage for persistent (<code>/project</code> and other) storage.<br />
Available to compute nodes, but not designed for parallel workloads.
|-
|'''High performance interconnect:'''
||
Low-latency high-performance fabric connecting all nodes and temporary storage. <br />
The design of GP2 is to support multiple simultaneous parallel jobs of at least 1024 cores in a fully non-blocking manner. Jobs larger than 1024 cores would be less well-suited for the topology.
|}


====Node types and characteristics:====
{| class="wikitable sortable"
|-
| "Base" compute nodes: || Over 500 nodes ||  128GB of memory each
|-
| "Large" compute nodes: || Over 100 nodes || 256GB of memory each
|-
| "Bigmem512" || 24 nodes || 512 GB of memory each
|-
|"Bigmem1500" nodes || 24 nodes || 1.5TB of memory each
|}
All of the above nodes will have 16 cores/socket (32 cores/node), with an anticipated frequency of 2.1Ghz.
{| class="wikitable sortable"
|-
| "GPU base" nodes: || Over 100 nodes with 4 GPUs each. || 12 cores/socket (24 cores/node) with an anticipated frequency of 2.2Ghz.  GPUs on a dual PCI root.
|-
| "GPU large" nodes.  || Approximately 30 nodes || same configuration as "GPU base," but a single PCI root.
|-
| "Bigmem3000" nodes || 4 nodes || each with 3TB of memory. These are 4-socket nodes with 8 cores/socket.
|}
All of the above nodes will have local (on-node) storage.
Compute Canada is not currently able to disclose the specific GPU model or specifications.  The RFP used NVIDIA K80 as a baseline specification.
The total GP2 system is anticipated to have over 25,000 cores, 900 nodes, and 500 GPUs.
The delivery and installation schedule is not yet known, but the procurement team has confidence that the system will be in production for the start of the allocations year on April 1, 2017.
===GP3 (Waterloo)===
GP3 system evaluation is not yet completed (as of early November 2016).  Anticipated specifications, based on Waterloo's RFP and bids received, include the following.  This information is NOT GUARANTEED and might not be complete.  It is provided for planning purposes.
The parallel filesystem, interconnects and external persistent storage will be the same as GP2.
====Node types and characteristics:====
{| class="wikitable sortable"
|-
| "Base" compute nodes || Over 500 nodes ||  128GB of memory each
|-
| "Large" compute nodes || Over 100 nodes || 256GB of memory each
|-
| "Bigmem512" nodes|| TBD || 512GB of memory each
|-
| "Bigmem3000" nodes || TBD || 3 TB of memory each
|}
All of the above nodes will have local (on-node) storage.
Compute Canada is not currently able to disclose the specific CPU model or specifications.
GP3 will have GPU's, but Compute Canada is not currently able to disclose specific details. The RFP used NVIDIA K80 as a baseline specification.
The total GP3 system is anticipated to have over 25,000 cores.
The delivery and installation schedule is not yet known.


==National Data Cyberinfrastructure (NDC)==
==National Data Cyberinfrastructure (NDC)==

Revision as of 17:38, 7 November 2016

Sharing: PublicNew SystemsAuthor:CC Migration team

Compute

Overview

GP2 and GP3 are almost identical systems with some minor differences in the actual mix of large memory, small memory and GPU nodes.

Name Description Approximate Capacity Availability
CC-Cloud Resources (GP1) Cloud 7,640 cores In production (integrated with west.cloud)
GP2

heterogeneous, general-purpose cluster

  • Serial and small parallel jobs
  • Small cloud partition
 25,000 cores April, 2017
GP3

heterogeneous, general-purpose cluster

  • Serial and small parallel jobs
  • Small cloud partition
 25,000 cores April, 2017
LP a cluster designed for large parallel jobs 60,000 cores Late 2017

Note that GP1, GP2 and LP will all have large, high-performance attached storage.


Template:GP2

Template:GP3


National Data Cyberinfrastructure (NDC)

Type Location Capacity Availability Comments
Nearline (Long-term)
File Storage

SFU and Waterloo

  • NDC-SFU
  • NDC-Waterloo
 15 PB each Late Autumn 2016
  • Tape backup
  • Silo replacement
  • Aiming for redundant tape backup across sites
Object Store

All 4 sites

  • NDC-Object

Small to start (A few PB usable)

Late 2017

  • Fully distributed, redundant object storage
  • Accessible anywhere
  • Allows for redundant, high availability architectures
  • S3 and File interfaces
  • New service aimed at experimental and observational data
Special Purpose various ~3.5 PB Specialized migration plans
  • Special purpose for Atlas, LHC, SNO+, CANFAR
  • dCache and other customized configurations

Note that due to Silo decommissioning Dec.31/2016 it will be necessary to provide interim storage while the NDC is developed. See Migration2016:Silo for details.

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=National_systems&oldid=3496"