Infrastructure renewal: Difference between revisions
No edit summary |
m (replace external link with Wiki link) |
||
| (38 intermediate revisions by 7 users not shown) | |||
| Line 2: | Line 2: | ||
<translate> | <translate> | ||
<!--T:1--> | =Major upgrade of our Advanced Research Computing infrastructure= <!--T:1--> | ||
<!--T:2--> | <!--T:2--> | ||
Our Advanced Research Computing infrastructure is undergoing major changes to | Our Advanced Research Computing infrastructure is undergoing major changes in the winter of 2024-2025 and spring of 2025 to provide better High Performance Computing (HPC) and Cloud services for Canadian researchers. This page will be regularly updated to keep you informed of the activities concerning the transition to the new equipment. | ||
<!--T:31--> | |||
The infrastructure renewal will replace the nearly 80% of our current equipment that is approaching end-of-life. The new equipment will offer faster processing speeds, greater storage capacity, and improved reliability. | The infrastructure renewal will replace the nearly 80% of our current equipment that is approaching end-of-life. The new equipment will offer faster processing speeds, greater storage capacity, and improved reliability. | ||
<!--T:3--> | <!--T:3--> | ||
The systems involved are | The systems involved are | ||
*Arbutus, cloud | *[[Infrastructure renewal#Arbutus,_cloud|Arbutus]], cloud | ||
*Béluga, compute cluster only (not cloud) | *[[Infrastructure renewal#Béluga,_compute_cluster_only_(not_cloud)|Béluga, compute cluster only (not cloud)]] | ||
*Cedar, compute cluster and cloud | *[[Infrastructure renewal#Cedar,_compute_cluster_and_cloud|Cedar, compute cluster and cloud]] | ||
*Graham, compute cluster and cloud | *[[Infrastructure renewal#Graham,_compute_cluster_and_cloud|Graham, compute cluster and cloud]] | ||
*Niagara, compute cluster | *[[Infrastructure renewal#Niagara,_compute_cluster|Niagara, compute cluster]] | ||
<!--T:4--> | =Technical specifications= <!--T:4--> | ||
Technical specifications for each new system will be provided further down this page in future updates. Generally, they will be similar in architecture to the current systems, but with considerably increased capacity and performance. | |||
Technical specifications for each new system will be provided further down this page in future updates. Generally, they will be similar in architecture to the current systems, but with considerably increased capacity and performance. | |||
For example, we expect to have fewer compute nodes, but each node will have a significant increase in the number of its cores, for an overall increase in the total number of CPU cores. | For example, we expect to have fewer compute nodes, but each node will have a significant increase in the number of its cores, for an overall increase in the total number of CPU cores. | ||
<!--T:5--> | =Impacts= <!--T:5--> | ||
<!--T:6--> | ==System outages== <!--T:6--> | ||
During the installation and the transition to the new systems, outages will be unavoidable due to constraints on space and electrical power. | |||
We recommend that you consider the possibility of outages when you plan research programs, graduate examinations, etc. | We recommend that you consider the possibility of outages when you plan research programs, graduate examinations, etc. | ||
<!--T: | <!--T:29--> | ||
==Resource Allocation Competition (RAC)== | {| class="wikitable" | ||
|- | |||
| '''Start Time''' || '''End Time''' || '''System''' || '''Description''' | |||
|- | |||
| Nov 7, 2024 || Nov 8, 2024 (1 day) || Niagara || All systems and storage located at the SciNet Datacenter (Niagara, Mist, HPSS, Rouge, Teach, JupyterHub, Balam) will be unavailable from 7 a.m. to 5 p.m. ET. This outage is required to install new electrical equipment (UPS) for the upcoming systems refresh. The work is expected to be completed in one day. The scheduler will hold jobs that cannot finish before the start of the shutdown. Users are encouraged to submit small and short jobs that can take advantage of this, as the scheduler may be able to fit these jobs in before the maintenance on otherwise idle nodes. | |||
|- | |||
| Nov 7, 2024 6am PST || Nov 8, 2024 6am PST || Cedar || Cedar compute nodes will be unavailable during this period (jobs will not run). Cedar login nodes and storage, as well as Cedar cloud will remain online and are not affected by this work. | |||
|} | |||
==Resource Allocation Competition (RAC)== <!--T:7--> | |||
The [https://www.alliancecan.ca/en/services/advanced-research-computing/accessing-resources/resource-allocation-competition Resource Allocation Competition] will be impacted by this transition, but the application process remains the same. Application deadline this year is October 30, 2024.<br> | The [https://www.alliancecan.ca/en/services/advanced-research-computing/accessing-resources/resource-allocation-competition Resource Allocation Competition] will be impacted by this transition, but the application process remains the same. Application deadline this year is October 30, 2024.<br> | ||
2024/25 allocations will remain in effect on retiring clusters while each cluster remains in service. The 2025/26 allocations will be implemented everywhere once all new clusters are in service.<br> | 2024/25 allocations will remain in effect on retiring clusters while each cluster remains in service. The 2025/26 allocations will be implemented everywhere once all new clusters are in service.<br> | ||
Because the old clusters will mostly be out of service before all new ones are available, if you hold both a 2024 and a 2025 RAC award you will experience a period when neither award is available to you. You will be able to compute with your default allocation (<code>def-xxxxxx</code>) on each new cluster as soon as it goes into service, but the 2025 RAC allocations will only become available when all new clusters are in service. | Because the old clusters will mostly be out of service before all new ones are available, if you hold both a 2024 and a 2025 RAC award you will experience a period when neither award is available to you. You will be able to compute with your default allocation (<code>def-xxxxxx</code>) on each new cluster as soon as it goes into service, but the 2025 RAC allocations will only become available when all new clusters are in service. | ||
<!--T:8--> | =General progress updates= <!--T:8--> | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |||
| Oct 7, 2024 || Details for necessary infrastructure (power and cooling) upgrades are being worked out. Timelines are not yet available but we expect some outages of a day or more in November. | |||
|- | |- | ||
| Sep 13, 2024 || The RFP processes for all sites except for Rorqual (replacing Béluga) have been completed, and purchase orders have been sent to vendors. The Rorqual storage Request for Proposals is still open and is scheduled to complete on September 18. | | Sep 13, 2024 || The RFP processes for all sites except for Rorqual (replacing Béluga) have been completed, and purchase orders have been sent to vendors. The Rorqual storage Request for Proposals is still open and is scheduled to complete on September 18. | ||
| Line 46: | Line 54: | ||
|} | |} | ||
<!--T:9--> | =Activities by system= <!--T:9--> | ||
<!--T:10--> | ==Arbutus, cloud== <!--T:10--> | ||
[[Cloud resources#Arbutus cloud|Arbutus]] | |||
<i>coming soon</i> | <i>coming soon</i> | ||
==Béluga, compute cluster only (not cloud)== <!--T:11--> | |||
<!--T: | <!--T:30--> | ||
The cluster replacing [[Beluga/en|Béluga]] will be named <b>Rorqual</b>. Specifications are available on [[Rorqual/en|this page]]. | |||
= | ==Cedar, compute cluster and cloud== <!--T:12--> | ||
The cluster replacing [[Cedar]] will be named <b>Fir</b>. Specifications are available on [[Fir/en|this page]]. | |||
== | ==Graham, compute cluster and cloud== <!--T:13--> | ||
[[Graham]] | |||
<i>coming soon</i> | |||
< | |||
== Niagara == <!--T: | ==Niagara, compute cluster== <!--T:14--> | ||
The cluster replacing [[Niagara]] & [[Mist]] in early 2025 will be named [[Trillium]]. Specifications are available on [[Trillium|this page]]. Hardware delivery starts late 2024, and the new cluster will be available for users in the spring of 2025. To make room, half of Niagara will be decommissioned starting in December 2024 or January 2025. We’ll update you when we have a better idea of Trillium’s installation schedule. | |||
= Frequently asked questions = <!--T: | = Frequently asked questions = <!--T:15--> | ||
== Will data be copied to | == Will my data be copied to its new system? == <!--T:16--> | ||
Data migration to the new systems is | Data migration to the new systems is the responsibility of each National Host Site who will inform you of what you need to do. | ||
== When will outages occur? == <!--T: | == When will outages occur? == <!--T:17--> | ||
Each | Each National Host Site will have its own schedule for outages as the installation of and transition to new equipment proceeds. As usual, specific outages will be described on [https://status.alliancecan.ca our system status web page]. We will provide more general updates on this wiki page and you will periodically receive emails with updates and outage notices. | ||
== Who | == Who can I contact for questions about the transition? == <!--T:18--> | ||
Contact our [[ | Contact our [[technical support]]. They will try their best to answer any questions they can. | ||
== Will my jobs | == Will my jobs and applications still be able to run on the new system? == <!--T:19--> | ||
Generally yes, but | Generally yes, but the new CPUs and GPUs may require recompilation or reconfiguration of some applications. More details will be provided as the transition unfolds. | ||
== Will the software from the | == Will the software from the current systems still be available? == <!--T:20--> | ||
Yes, our [[Standard software environments|standard software environment]] will be available on the new systems. | Yes, our [[Standard software environments|standard software environment]] will be available on the new systems. | ||
== Will there be staggered outages? == <!--T: | == Will there be staggered outages? == <!--T:21--> | ||
We will do our best to limit overlapping outages, but we are very constrained by delivery schedules and funding deadlines | We will do our best to limit overlapping outages, but because we are very constrained by delivery schedules and funding deadlines, there will probably be periods when several of our systems are simultaneously offline. Outages will be announced as early as possible. | ||
== Can I purchase old hardware after equipment upgrades? == <!--T:28--> | |||
Most of the equipment is legally the property of the hosting institution. When the equipment is retired, the host institution manages its disposal following that institution's guidelines. This typically involves "e-cycling"--- recycling the equipment rather than selling it. If you're looking to acquire the old hardware, it's best to contact the host institution directly, as they may have specific policies or options for selling equipment. | |||
</translate> | </translate> | ||
Latest revision as of 13:46, 6 November 2024
Major upgrade of our Advanced Research Computing infrastructure[edit]
Our Advanced Research Computing infrastructure is undergoing major changes in the winter of 2024-2025 and spring of 2025 to provide better High Performance Computing (HPC) and Cloud services for Canadian researchers. This page will be regularly updated to keep you informed of the activities concerning the transition to the new equipment.
The infrastructure renewal will replace the nearly 80% of our current equipment that is approaching end-of-life. The new equipment will offer faster processing speeds, greater storage capacity, and improved reliability.
The systems involved are
- Arbutus, cloud
- Béluga, compute cluster only (not cloud)
- Cedar, compute cluster and cloud
- Graham, compute cluster and cloud
- Niagara, compute cluster
Technical specifications[edit]
Technical specifications for each new system will be provided further down this page in future updates. Generally, they will be similar in architecture to the current systems, but with considerably increased capacity and performance. For example, we expect to have fewer compute nodes, but each node will have a significant increase in the number of its cores, for an overall increase in the total number of CPU cores.
Impacts[edit]
System outages[edit]
During the installation and the transition to the new systems, outages will be unavoidable due to constraints on space and electrical power. We recommend that you consider the possibility of outages when you plan research programs, graduate examinations, etc.
| Start Time | End Time | System | Description |
| Nov 7, 2024 | Nov 8, 2024 (1 day) | Niagara | All systems and storage located at the SciNet Datacenter (Niagara, Mist, HPSS, Rouge, Teach, JupyterHub, Balam) will be unavailable from 7 a.m. to 5 p.m. ET. This outage is required to install new electrical equipment (UPS) for the upcoming systems refresh. The work is expected to be completed in one day. The scheduler will hold jobs that cannot finish before the start of the shutdown. Users are encouraged to submit small and short jobs that can take advantage of this, as the scheduler may be able to fit these jobs in before the maintenance on otherwise idle nodes. |
| Nov 7, 2024 6am PST | Nov 8, 2024 6am PST | Cedar | Cedar compute nodes will be unavailable during this period (jobs will not run). Cedar login nodes and storage, as well as Cedar cloud will remain online and are not affected by this work. |
Resource Allocation Competition (RAC)[edit]
The Resource Allocation Competition will be impacted by this transition, but the application process remains the same. Application deadline this year is October 30, 2024.
2024/25 allocations will remain in effect on retiring clusters while each cluster remains in service. The 2025/26 allocations will be implemented everywhere once all new clusters are in service.
Because the old clusters will mostly be out of service before all new ones are available, if you hold both a 2024 and a 2025 RAC award you will experience a period when neither award is available to you. You will be able to compute with your default allocation (def-xxxxxx) on each new cluster as soon as it goes into service, but the 2025 RAC allocations will only become available when all new clusters are in service.
General progress updates[edit]
| Oct 7, 2024 | Details for necessary infrastructure (power and cooling) upgrades are being worked out. Timelines are not yet available but we expect some outages of a day or more in November. |
| Sep 13, 2024 | The RFP processes for all sites except for Rorqual (replacing Béluga) have been completed, and purchase orders have been sent to vendors. The Rorqual storage Request for Proposals is still open and is scheduled to complete on September 18.
All sites are working on infrastructure design (power and cooling) and implementation. We are expecting some outages throughout the fall for cabling and plumbing upgrades. |
| Sep 3, 2024 | All sites have completed their Requests for Proposals, and are working with the vendors on deliverables and purchase orders. |
Activities by system[edit]
Arbutus, cloud[edit]
Arbutus coming soon
Béluga, compute cluster only (not cloud)[edit]
The cluster replacing Béluga will be named Rorqual. Specifications are available on this page.
Cedar, compute cluster and cloud[edit]
The cluster replacing Cedar will be named Fir. Specifications are available on this page.
Graham, compute cluster and cloud[edit]
Graham coming soon
Niagara, compute cluster[edit]
The cluster replacing Niagara & Mist in early 2025 will be named Trillium. Specifications are available on this page. Hardware delivery starts late 2024, and the new cluster will be available for users in the spring of 2025. To make room, half of Niagara will be decommissioned starting in December 2024 or January 2025. We’ll update you when we have a better idea of Trillium’s installation schedule.
Frequently asked questions[edit]
Will my data be copied to its new system?[edit]
Data migration to the new systems is the responsibility of each National Host Site who will inform you of what you need to do.
When will outages occur?[edit]
Each National Host Site will have its own schedule for outages as the installation of and transition to new equipment proceeds. As usual, specific outages will be described on our system status web page. We will provide more general updates on this wiki page and you will periodically receive emails with updates and outage notices.
Who can I contact for questions about the transition?[edit]
Contact our technical support. They will try their best to answer any questions they can.
Will my jobs and applications still be able to run on the new system?[edit]
Generally yes, but the new CPUs and GPUs may require recompilation or reconfiguration of some applications. More details will be provided as the transition unfolds.
Will the software from the current systems still be available?[edit]
Yes, our standard software environment will be available on the new systems.
Will there be staggered outages?[edit]
We will do our best to limit overlapping outages, but because we are very constrained by delivery schedules and funding deadlines, there will probably be periods when several of our systems are simultaneously offline. Outages will be announced as early as possible.
Can I purchase old hardware after equipment upgrades?[edit]
Most of the equipment is legally the property of the hosting institution. When the equipment is retired, the host institution manages its disposal following that institution's guidelines. This typically involves "e-cycling"--- recycling the equipment rather than selling it. If you're looking to acquire the old hardware, it's best to contact the host institution directly, as they may have specific policies or options for selling equipment.