Getting started: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
No edit summary
(broken link)
 
(68 intermediate revisions by 15 users not shown)
Line 2: Line 2:


<translate>
<translate>
<!--T:1-->
You have just received your Compute Canada account. Welcome! Now what do you do? This page is intended to help you find your way through the technical documentation on Compute Canada services and systems.
<!--T:2-->
{| class="wikitable"
|-
| If you don't already have a Compute Canada account, see [https://www.computecanada.ca/research-portal/account-management/apply-for-an-account/ Apply for an account].
|}


==What do you want to do?== <!--T:3-->
==What do you want to do?== <!--T:3-->
* If you don't already have an account, see
** [https://alliancecan.ca/en/services/advanced-research-computing/account-management/apply-account Apply for an account]
** [[Frequently Asked Questions about the CCDB]]
* If you are an experienced HPC user and are ready to log onto a cluster, you probably want to know
* If you are an experienced HPC user and are ready to log onto a cluster, you probably want to know
** what [[#What resources are available?|clusters]] are available;
** what [[#What systems are available?|systems]] are available;
** what [[Available software|software]] is available, and how [[Utiliser des modules/en|environment modules]] work;
** what [[Available software|software]] is available, and how [[Utiliser des modules/en|environment modules]] work;
** how to submit [[Running jobs|jobs]];
** how to [[Running jobs|submit jobs]];
** how the [[Storage and file management|filesystem]] is organized.
** how the [[Storage and file management|filesystems]] are organized.
* If you are new to HPC or would like some training, you can
* If you are new to HPC, you can
** read about how to connect to our HPC systems with [[SSH|SSH]];
** read about how to connect to our HPC systems with [[SSH|SSH]];
** read an introduction to [[Linux introduction|Linux]] systems;
** read an [[Linux introduction|introduction to Linux]] systems;
** read about how to [[Transferring data|transfer files]] to and from Compute Canada systems;  
** read about how to [[Transferring data|transfer files]] to and from our systems;  
** read about other national training offerings [https://www.computecanada.ca/research-portal/technical-support/training/ here].
* If you want to know which software and hardware are available for a specific discipline, a series of discipline guides is in preparation. At this time, you can consult the guides on  
* If you want to know which software and hardware are available for a specific discipline, a series of discipline guides is in preparation. At this time, you can consult the guide on  
** [[AI and Machine Learning]]
** [[Visualization]].
** [[Bioinformatics]]
* If you have hundreds of gigabytes of data to move across the network, read about the [[Globus]] file transfer services.
** [[Biomolecular simulation]]
** [[Computational chemistry]]
** [[Computational fluid dynamics]] ([[CFD]])
** [[Geographic information systems]] ([[GIS]])
** [[Visualization]]
* If you have hundreds of gigabytes of data to move across the network, read about the [[Globus]] file transfer service.
* Python users can learn how to [[Python#Creating_and_using_a_virtual_environment|install modules in a virtual environment]] and R users how to [[R|install packages]].
* Python users can learn how to [[Python#Creating_and_using_a_virtual_environment|install modules in a virtual environment]] and R users how to [[R|install packages]].
* If you want to experiment with software that doesn’t run well on our traditional HPC systems, please read about Compute Canada [[Cloud|Cloud resources]].
* If you want to experiment with software that doesn’t run well on our traditional HPC clusters, please read about [[Cloud|our cloud resources]].
* If you prefer a video webinar format, please check our [[Getting Started with the new National Systems]] mini-webinar series.


<!--T:4-->
<!--T:4-->
For any other questions, you might try the ''Search'' box in the upper right corner of this page, the main page for [[Compute Canada Documentation|Compute Canada Documentation]], or [[Technical support|contact us]] by email.
For any other questions, you might try the <i>Search</i> box in the upper right corner of this page, the main page for [[Technical documentation|our technical documentation]] or [[Technical support|contact us]] by email.


==What resources are available?== <!--T:5-->
==What systems are available?== <!--T:5-->
Compute Canada is currently installing several million dollars’ worth of new computers while simultaneously retiring many old computers. During the transition period (2016-2018), a changing mix of old and new computers will be available to you. You can read about the progress of the migration from old to new systems [https://www.computecanada.ca/research-portal/accessing-resources/migration/ here].


===New resources (deployed in 2016 or after)=== <!--T:6-->
<!--T:6-->
Compute Canada began to renew its infrastructure in 2016. The first phase of the new deployment is composed of four new clusters, called [[CC-Cloud Resources|Arbutus]], [[Cedar]], [[Graham]], and [[Niagara]].
The renewal of the national infrastructure began in 2016. The deployment currently consists of five systems, called [[Cloud resources|Arbutus]], [[Béluga/en|Béluga]], [[Narval/en|Narval]], [[Cedar]], [[Graham]], and [[Niagara]].


<!--T:18-->
<!--T:18-->
[[CC-Cloud Resources|Arbutus]] (formerly known as GP1) is an extension of the West cloud. Arbutus went into service in September 2016.
[[Cloud resources|Arbutus]] is a [[cloud]] site, which allows users to launch and customize virtual machines. See [[Cloud]] for how to obtain access to Arbutus.


<!--T:19-->
<!--T:19-->
[[Cedar]] (GP2) and [[Graham]] (GP3) are general purpose clusters composed of a variety of nodes including large memory nodes and nodes with accelerators. They are entering service in the summer of 2017 and to access them you must login to the [https://ccdb.computecanada.ca/services/resources Resource Access page]. On this page you can click on the link "Grant access" for the cluster that you wish to use, which will allow you immediate access, for instance using [[SSH]]. Once you have access to a cluster, you can suspend it by clicking on "Revoke access"; note that this will not lead to the loss of any of your data.  
[[Béluga/en|Béluga]], [[Cedar]], [[Narval/en|Narval]] and [[Graham]] are <b>general-purpose clusters</b> composed of a variety of nodes including large memory nodes and nodes with accelerators such as GPUs. You can log into any of these using [[SSH]]. A /home directory will be automatically created for you the first time you log in.


<!--T:20-->
<!--T:20-->
[[Niagara]] (LP) will be a large parallel cluster with nodes interconnected by a fast network planned to enter service in early 2018.
[[Niagara]] is a homogeneous cluster designed for <b>large parallel</b> jobs (>1000 cores). To obtain access to Niagara, visit the [https://ccdb.alliancecan.ca/services/opt_in Available Services] page.


<!--T:21-->
<!--T:21-->
Your '''password''' to log in to all new national systems are the same one you use to log in to [https://ccdb.computecanada.ca/ ccdb.computecanada.ca]. Your '''username''' will be displayed at the top of your home page at [https://ccdb.computecanada.ca/ ccdb.computecanada.ca] once you've logged in there.
Your <b>password</b> to log in to all new national systems is the same one you use to log into [https://ccdb.alliancecan.ca/ CCDB]. Your <b>username</b> will be displayed at the top of the page once you've logged in.
 
===Legacy resources (deployed before 2016)=== <!--T:7-->
Computing and storage resources which were installed between 2004 and 2015 and scheduled to be decommissioned in the next few years are referred to as ''legacy resources''. The legacy resources are administered by regional organizations, one of [http://www.ace-net.ca/ ACENET], [http://cac.queensu.ca/ the Centre for Advanced Computing], [http://www.calculquebec.ca/en/ Calcul Québec], [http://www.scinethpc.ca/ SciNet], [https://www.sharcnet.ca/my/front/ SHARCNET], and [https://www.westgrid.ca/ WestGrid]. To use a legacy resource you must have an account with one of these entities; you can apply for an account through [https://ccdb.computecanada.ca/me/facilities CCDB]. Resources deployed during and after 2016 will not require this step, nor will the two [[CC-Cloud|clouds]].
 
<!--T:8-->
Most legacy clusters are classified as either capacity clusters or capability clusters. ''Capacity clusters'' contain nodes connected to each other by a relatively slow Ethernet network, while the ''capability clusters'' have a fast network, usually InfiniBand. Large parallel jobs will run better on capability clusters than capacity clusters, while smaller jobs will run almost anywhere.  


<!--T:9-->
==What training is available?== <!--T:22-->
There are some specialty clusters among the legacy resources. Applications which require more than 512 GB of memory per node require ''large shared memory systems''. Compute Canada has four such systems:
* Hungabee hosted by WestGrid
* M9000 hosted by the Centre for Advanced Computing
* Guillimin-ScaleMP hosted by Calcul Québec
* Iqaluk, Wobbie, Mosaic hosted by SHARCNET


<!--T:10-->
<!--T:23-->
Compute Canada also has clusters equipped with accelerators such as [https://en.wikipedia.org/wiki/General-purpose_computing_on_graphics_processing_units GPUs] and Intel [https://en.wikipedia.org/wiki/Xeon_Phi Xeon Phis]. If your application calls for from such accelerators, in addition to the new systems, you will find them on the following legacy systems:
Most workshops are organized by the Alliance's regional partners; both online and in-person training opportunities exist on a wide variety of subjects and at different levels of sophistication. We invite you to consult the following regional training calendars and websites for more information,
* Helios, Hades and Guillimin, hosted by Calcul Québec
* WestDRI (Western Canada Research Computing covering both BC and the Prairies regions)
* Parallel, hosted by WestGrid
** [https://training.westdri.ca Training Materials website] - click on <i>Upcoming sessions</i> or browse the menu at the top for recorded webinars
* Monk, hosted by SHARCNET
** [https://www.ualberta.ca/information-services-and-technology/research-computing/bootcamps.html UAlberta ARC Bootcamp] - videos of previous sessions available
* Accelerator Research Cluster, hosted by SciNet
* [https://www.sharcnet.ca SHARCNET]
All of these have NVidia GPUs. Guillimin also has Intel Xeon Phis.  
** [https://www.sharcnet.ca/my/news/calendar Training Events Calendar]
** [http://youtube.sharcnet.ca/ YouTube Channel]
** [https://training.sharcnet.ca/ Online Workshops]
* [https://www.scinethpc.ca SciNet]
** [https://education.scinet.utoronto.ca SciNet Education Site]
** [https://www.youtube.com/c/SciNetHPCattheUniversityofToronto SciNet YouTube Channel]
* [https://www.calculquebec.ca/en/ Calcul Québec]
** [https://calculquebec.eventbrite.ca/ Workshops]
** [https://www.calculquebec.ca/en/academic-research-services/training/ Training information]
* [https://www.ace-net.ca/ ACENET]
** [https://www.ace-net.ca/training.html Training information]
** [https://www.youtube.com/@ACENETDRI ACENET YouTube Channel]
One can also find a shared calendar of [https://alliancecan.ca/en/services/advanced-research-computing/technical-support/training-calendar upcoming workshops].


==What resources should I use?== <!--T:12-->
==What system should I use?== <!--T:12-->
This question is hard to answer because of the range of needs Compute Canada serves, and because of the enormous variety of resources we have available --- especially during the 2016-2018 renewal period. If the descriptions above are insufficient, contact Compute Canada’s [[technical support]] or your [https://www.computecanada.ca/about/partners/ regional support].  
This question is hard to answer because of the range of needs we serve and the wide variety of resources we have available. If the descriptions above are insufficient, contact our [[technical support]].


<!--T:13-->
<!--T:13-->
Line 81: Line 79:
** Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface?  
** Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface?  
** Can it run on the Linux operating system?
** Can it run on the Linux operating system?
* How much memory, time, computing power, accelerators, storage, network bandwidth and so forth --- are required by a typical job? Rough estimates are fine.
* How much memory, time, computing power, accelerators, storage, network bandwidth and so forth—are required by a typical job? Rough estimates are fine.
* How frequently will you need to run this type of job?
* How frequently will you need to run this type of job?


Latest revision as of 14:13, 21 November 2023

Other languages:


What do you want to do?

For any other questions, you might try the Search box in the upper right corner of this page, the main page for our technical documentation or contact us by email.

What systems are available?

The renewal of the national infrastructure began in 2016. The deployment currently consists of five systems, called Arbutus, Béluga, Narval, Cedar, Graham, and Niagara.

Arbutus is a cloud site, which allows users to launch and customize virtual machines. See Cloud for how to obtain access to Arbutus.

Béluga, Cedar, Narval and Graham are general-purpose clusters composed of a variety of nodes including large memory nodes and nodes with accelerators such as GPUs. You can log into any of these using SSH. A /home directory will be automatically created for you the first time you log in.

Niagara is a homogeneous cluster designed for large parallel jobs (>1000 cores). To obtain access to Niagara, visit the Available Services page.

Your password to log in to all new national systems is the same one you use to log into CCDB. Your username will be displayed at the top of the page once you've logged in.

What training is available?

Most workshops are organized by the Alliance's regional partners; both online and in-person training opportunities exist on a wide variety of subjects and at different levels of sophistication. We invite you to consult the following regional training calendars and websites for more information,

One can also find a shared calendar of upcoming workshops.

What system should I use?

This question is hard to answer because of the range of needs we serve and the wide variety of resources we have available. If the descriptions above are insufficient, contact our technical support.

In order to identify the best resource to use, we may ask specific questions, such as:

  • What software do you want to use?
    • Does the software require a commercial license?
    • Can the software be used non-interactively? That is, can it be controlled from a file prepared prior to its execution rather than through the graphical interface?
    • Can it run on the Linux operating system?
  • How much memory, time, computing power, accelerators, storage, network bandwidth and so forth—are required by a typical job? Rough estimates are fine.
  • How frequently will you need to run this type of job?

You may know the answer to these questions or not. If you do not, our technical support team is there to help you find the answers. Then they will be able to direct you to the most appropriate resources for your needs.

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Getting_started&oldid=146906"