Getting started: Difference between revisions
Jump to navigation
Jump to search
(broken link) |
|||
| (23 intermediate revisions by 8 users not shown) | |||
| Line 5: | Line 5: | ||
==What do you want to do?== <!--T:3--> | ==What do you want to do?== <!--T:3--> | ||
* If you don't already have an account, see | * If you don't already have an account, see | ||
** [https://alliancecan.ca/en/services/advanced-research-computing/account-management/apply-account Apply for | ** [https://alliancecan.ca/en/services/advanced-research-computing/account-management/apply-account Apply for an account] | ||
** [[Frequently Asked Questions about the CCDB]] | ** [[Frequently Asked Questions about the CCDB]] | ||
* If you are an experienced HPC user and are ready to log onto a cluster, you probably want to know | * If you are an experienced HPC user and are ready to log onto a cluster, you probably want to know | ||
** what [[#What | ** what [[#What systems are available?|systems]] are available; | ||
** what [[Available software|software]] is available, and how [[Utiliser des modules/en|environment modules]] work; | ** what [[Available software|software]] is available, and how [[Utiliser des modules/en|environment modules]] work; | ||
** how to [[Running jobs|submit jobs]]; | ** how to [[Running jobs|submit jobs]]; | ||
** how the [[Storage and file management| | ** how the [[Storage and file management|filesystems]] are organized. | ||
* If you are new to HPC | * If you are new to HPC, you can | ||
** read about how to connect to our HPC systems with [[SSH|SSH]]; | ** read about how to connect to our HPC systems with [[SSH|SSH]]; | ||
** read an [[Linux introduction|introduction to Linux]] systems; | ** read an [[Linux introduction|introduction to Linux]] systems; | ||
| Line 27: | Line 26: | ||
* If you have hundreds of gigabytes of data to move across the network, read about the [[Globus]] file transfer service. | * If you have hundreds of gigabytes of data to move across the network, read about the [[Globus]] file transfer service. | ||
* Python users can learn how to [[Python#Creating_and_using_a_virtual_environment|install modules in a virtual environment]] and R users how to [[R|install packages]]. | * Python users can learn how to [[Python#Creating_and_using_a_virtual_environment|install modules in a virtual environment]] and R users how to [[R|install packages]]. | ||
* If you want to experiment with software that doesn’t run well on our traditional HPC | * If you want to experiment with software that doesn’t run well on our traditional HPC clusters, please read about [[Cloud|our cloud resources]]. | ||
<!--T:4--> | <!--T:4--> | ||
For any other questions, you might try the | For any other questions, you might try the <i>Search</i> box in the upper right corner of this page, the main page for [[Technical documentation|our technical documentation]] or [[Technical support|contact us]] by email. | ||
==What | ==What systems are available?== <!--T:5--> | ||
<!--T:6--> | <!--T:6--> | ||
The renewal of the national infrastructure began in 2016. The deployment currently consists of five | The renewal of the national infrastructure began in 2016. The deployment currently consists of five systems, called [[Cloud resources|Arbutus]], [[Béluga/en|Béluga]], [[Narval/en|Narval]], [[Cedar]], [[Graham]], and [[Niagara]]. | ||
<!--T:18--> | <!--T:18--> | ||
| Line 41: | Line 40: | ||
<!--T:19--> | <!--T:19--> | ||
[[Béluga/en|Béluga]], [[Cedar]], [[Narval/en|Narval]] and [[Graham]] are | [[Béluga/en|Béluga]], [[Cedar]], [[Narval/en|Narval]] and [[Graham]] are <b>general-purpose clusters</b> composed of a variety of nodes including large memory nodes and nodes with accelerators such as GPUs. You can log into any of these using [[SSH]]. A /home directory will be automatically created for you the first time you log in. | ||
<!--T:20--> | <!--T:20--> | ||
[[Niagara]] is a homogeneous cluster designed for | [[Niagara]] is a homogeneous cluster designed for <b>large parallel</b> jobs (>1000 cores). To obtain access to Niagara, visit the [https://ccdb.alliancecan.ca/services/opt_in Available Services] page. | ||
<!--T:21--> | <!--T:21--> | ||
Your | Your <b>password</b> to log in to all new national systems is the same one you use to log into [https://ccdb.alliancecan.ca/ CCDB]. Your <b>username</b> will be displayed at the top of the page once you've logged in. | ||
==What training is available?== | ==What training is available?== <!--T:22--> | ||
<!--T:23--> | |||
Most workshops are organized by the Alliance's regional partners; both online and in-person training opportunities exist on a wide variety of subjects and at different levels of sophistication. We invite you to consult the following regional training calendars and websites for more information, | Most workshops are organized by the Alliance's regional partners; both online and in-person training opportunities exist on a wide variety of subjects and at different levels of sophistication. We invite you to consult the following regional training calendars and websites for more information, | ||
* | * WestDRI (Western Canada Research Computing covering both BC and the Prairies regions) | ||
** [https:// | ** [https://training.westdri.ca Training Materials website] - click on <i>Upcoming sessions</i> or browse the menu at the top for recorded webinars | ||
** [https://www. | ** [https://www.ualberta.ca/information-services-and-technology/research-computing/bootcamps.html UAlberta ARC Bootcamp] - videos of previous sessions available | ||
* | * [https://www.sharcnet.ca SHARCNET] | ||
* SciNet | ** [https://www.sharcnet.ca/my/news/calendar Training Events Calendar] | ||
** [http://youtube.sharcnet.ca/ YouTube Channel] | |||
** [https://training.sharcnet.ca/ Online Workshops] | |||
* [https://www.scinethpc.ca SciNet] | |||
** [https://education.scinet.utoronto.ca SciNet Education Site] | ** [https://education.scinet.utoronto.ca SciNet Education Site] | ||
** [https://www.youtube.com/c/SciNetHPCattheUniversityofToronto SciNet YouTube Channel] | ** [https://www.youtube.com/c/SciNetHPCattheUniversityofToronto SciNet YouTube Channel] | ||
* | * [https://www.calculquebec.ca/en/ Calcul Québec] | ||
* | ** [https://calculquebec.eventbrite.ca/ Workshops] | ||
** [https://training. | ** [https://www.calculquebec.ca/en/academic-research-services/training/ Training information] | ||
* [https://www.ace-net.ca/ ACENET] | |||
** [https://www.ace-net.ca/training.html Training information] | |||
** [https://www.youtube.com/@ACENETDRI ACENET YouTube Channel] | |||
One can also find a shared calendar of [https://alliancecan.ca/en/services/advanced-research-computing/technical-support/training-calendar upcoming workshops]. | One can also find a shared calendar of [https://alliancecan.ca/en/services/advanced-research-computing/technical-support/training-calendar upcoming workshops]. | ||
==What | ==What system should I use?== <!--T:12--> | ||
This question is hard to answer because of the range of needs we serve and the wide variety of resources we have available. If the descriptions above are insufficient, contact our [[ | This question is hard to answer because of the range of needs we serve and the wide variety of resources we have available. If the descriptions above are insufficient, contact our [[technical support]]. | ||
<!--T:13--> | <!--T:13--> | ||
| Line 74: | Line 79: | ||
** Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface? | ** Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface? | ||
** Can it run on the Linux operating system? | ** Can it run on the Linux operating system? | ||
* How much memory, time, computing power, accelerators, storage, network bandwidth and so | * How much memory, time, computing power, accelerators, storage, network bandwidth and so forth—are required by a typical job? Rough estimates are fine. | ||
* How frequently will you need to run this type of job? | * How frequently will you need to run this type of job? | ||
Latest revision as of 14:13, 21 November 2023
What do you want to do?
- If you don't already have an account, see
- If you are an experienced HPC user and are ready to log onto a cluster, you probably want to know
- what systems are available;
- what software is available, and how environment modules work;
- how to submit jobs;
- how the filesystems are organized.
- If you are new to HPC, you can
- read about how to connect to our HPC systems with SSH;
- read an introduction to Linux systems;
- read about how to transfer files to and from our systems;
- If you want to know which software and hardware are available for a specific discipline, a series of discipline guides is in preparation. At this time, you can consult the guides on
- If you have hundreds of gigabytes of data to move across the network, read about the Globus file transfer service.
- Python users can learn how to install modules in a virtual environment and R users how to install packages.
- If you want to experiment with software that doesn’t run well on our traditional HPC clusters, please read about our cloud resources.
For any other questions, you might try the Search box in the upper right corner of this page, the main page for our technical documentation or contact us by email.
What systems are available?
The renewal of the national infrastructure began in 2016. The deployment currently consists of five systems, called Arbutus, Béluga, Narval, Cedar, Graham, and Niagara.
Arbutus is a cloud site, which allows users to launch and customize virtual machines. See Cloud for how to obtain access to Arbutus.
Béluga, Cedar, Narval and Graham are general-purpose clusters composed of a variety of nodes including large memory nodes and nodes with accelerators such as GPUs. You can log into any of these using SSH. A /home directory will be automatically created for you the first time you log in.
Niagara is a homogeneous cluster designed for large parallel jobs (>1000 cores). To obtain access to Niagara, visit the Available Services page.
Your password to log in to all new national systems is the same one you use to log into CCDB. Your username will be displayed at the top of the page once you've logged in.
What training is available?
Most workshops are organized by the Alliance's regional partners; both online and in-person training opportunities exist on a wide variety of subjects and at different levels of sophistication. We invite you to consult the following regional training calendars and websites for more information,
- WestDRI (Western Canada Research Computing covering both BC and the Prairies regions)
- Training Materials website - click on Upcoming sessions or browse the menu at the top for recorded webinars
- UAlberta ARC Bootcamp - videos of previous sessions available
- SHARCNET
- SciNet
- Calcul Québec
- ACENET
One can also find a shared calendar of upcoming workshops.
What system should I use?
This question is hard to answer because of the range of needs we serve and the wide variety of resources we have available. If the descriptions above are insufficient, contact our technical support.
In order to identify the best resource to use, we may ask specific questions, such as:
- What software do you want to use?
- Does the software require a commercial license?
- Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface?
- Can it run on the Linux operating system?
- How much memory, time, computing power, accelerators, storage, network bandwidth and so forth—are required by a typical job? Rough estimates are fine.
- How frequently will you need to run this type of job?
You may know the answer to these questions or not. If you do not, our technical support team is there to help you find the answers. Then they will be able to direct you to the most appropriate resources for your needs.