38,760
edits
Allocations and compute scheduling/en: Difference between revisions
Allocations and compute scheduling/en (view source)
Revision as of 14:45, 31 March 2023
, 1 year agoUpdating to match new version of source page
(Updating to match new version of source page) |
(Updating to match new version of source page) |
||
| Line 65: | Line 65: | ||
=How does resource use affect priority?= | =How does resource use affect priority?= | ||
The overarching principle governing the calculation of priority on | The overarching principle governing the calculation of priority on Alliance's national clusters is that compute-based jobs are considered in the calculation based on the resources that others are prevented from using and not on the resources actually used. | ||
The most common example of unused cores contributing to a priority calculation occurs when a submitted job requests multiple cores but uses fewer cores than requested when run. The usage that will affect the priority of future jobs is the number of cores requested, not the number of cores the application actually used. This is because the unused cores were unavailable to others to use during the job. | The most common example of unused cores contributing to a priority calculation occurs when a submitted job requests multiple cores but uses fewer cores than requested when run. The usage that will affect the priority of future jobs is the number of cores requested, not the number of cores the application actually used. This is because the unused cores were unavailable to others to use during the job. | ||
| Line 81: | Line 81: | ||
Cedar and Graham are considered to provide 4GB per core, since this corresponds to the most common node type in those clusters, making a core equivalent on these systems a core-memory bundle of 4GB per core. Niagara is considered to provide 4.8GB of memory per core, making a core equivalent on it a core-memory bundle of 4.8GB per core. Jobs are charged in terms of core equivalent usage at the rate of 4 or 4.8 GB per core, as explained above. See Figure 1. | Cedar and Graham are considered to provide 4GB per core, since this corresponds to the most common node type in those clusters, making a core equivalent on these systems a core-memory bundle of 4GB per core. Niagara is considered to provide 4.8GB of memory per core, making a core equivalent on it a core-memory bundle of 4.8GB per core. Jobs are charged in terms of core equivalent usage at the rate of 4 or 4.8 GB per core, as explained above. See Figure 1. | ||
Allocation target tracking is straightforward when requests to use resources on the clusters are made entirely of core and memory amounts that can be portioned only into complete equivalent cores. Things become more complicated when jobs request portions of a core equivalent because it is possible to have many points counted against a research group’s allocation, even when they are using only portions of core equivalents. In practice, the method used by | Allocation target tracking is straightforward when requests to use resources on the clusters are made entirely of core and memory amounts that can be portioned only into complete equivalent cores. Things become more complicated when jobs request portions of a core equivalent because it is possible to have many points counted against a research group’s allocation, even when they are using only portions of core equivalents. In practice, the method used by Alliance to account for system usage solves problems about fairness and perceptions of fairness but unfortunately the method is not initially intuitive. | ||
Research groups are charged for the maximum number of core equivalents they take from the resources. Assuming a core equivalent of 1 core and 4GB of memory: | Research groups are charged for the maximum number of core equivalents they take from the resources. Assuming a core equivalent of 1 core and 4GB of memory: | ||
| Line 107: | Line 107: | ||
== Ratios: GPU / CPU Cores / System-memory == | == Ratios: GPU / CPU Cores / System-memory == | ||
Alliance systems have the following GPU-core-memory bundle characteristics: | |||
* [[Béluga/en#Node_Characteristics|Béluga]]: | * [[Béluga/en#Node_Characteristics|Béluga]]: | ||
** V100/16GB nodes: 1 GPU / 10 cores / 47000 MB | ** V100/16GB nodes: 1 GPU / 10 cores / 47000 MB | ||