Known issues/en: Difference between revisions

= Intro =

* Please report issues to [mailto:support@computecanada.ca support@computecanada.ca]

* Utilization accounting is not currently being forwarded to the [https://ccdb.computecanada.ca Compute Canada database]

* The CC slurm configuration preferentially encourages whole-node jobs. Users should, if appropriate, request whole-node rather than per-core resources. See [[Job_scheduling_policies#Whole_nodes_versus_cores;|Job Scheduling - Whole Node Scheduling]] ([[User:Pjmann|Patrick Mann]] 20:15, 17 July 2017 (UTC))

* Cpu and Gpu backfill partitions have been created on both clusters. If a job is submitted with <24hr runtime, it will be automatically entered into the cluster-wide backfill partition. This partition has a low priority, but will allow increased utilization of the cluster by serial jobs. ([[User:Nathanw|Nathan Wielenga]])

* SLURM epilog does not fully clean up processes from ended jobs, especially if the job did not exit normally.   ([[User:Gbnewby|Greg Newby]]) Fri Jul 14 19:32:48 UTC 2017)

** This has been greatly improved after the addition of the epilog.clean script, but there are still nodes occasionally marked down for epilog failure. (NW)

* By default, the job receives environment settings from the submitting shell. This can lead to irreproducible results if it's not what you expect. To force the job to run with a fresh-like-login environment, you can submit with <tt>--export=none</tt> or add <tt>#SBATCH --export=NONE</tt> to your job script.

Sometimes, users will see quota error on their project folders. This may happen when files are owned by a group other than the project group. You can change the group which owns files using the command

{{Command|chgrp -R <group> <folder>}}

To see what <group> should be, run the following command :

Only the owner of the files can run the <tt>chgrp</tt> command. To ask us to correct the group owner for many users, write to support@computecanada.ca

* "Nearline" capabilities are not yet available (see https://docs.computecanada.ca/wiki/National_Data_Cyberinfrastructure for a brief description of the intended functionality)

** Update July 17: still not working. If you need your nearline RAC2017 quota then please ask [mailto:support@computecanada.ca CC support]. ([[User:Pjmann|Patrick Mann]] 20:45, 17 July 2017 (UTC))

* Upon login to the new clusters, symbolic links are supposed to be created in the user's account, as described in [[Project layout]]. Sometimes, it does not happen. If this is the case, please verify that your access to the cluster is enabled on this page [https://ccdb.computecanada.ca/services/resources https://ccdb.computecanada.ca/services/resources]

* SLURM operations will occasionally time out with a message like "Socket timed out on send/recv operation" or "Unable to contact slurm controller (connect failure)". As a temporary workaround, attempt to resubmit your jobs/commands, they should go through in a few seconds. ([[User:Nathanw|Nathan Wielenga]]) 08:50, 18 July 2017 (MDT))

** Should be resolved after a VHD migration to a new backend for slurmctl. (NW)

*Some people are getting an error "error: Job submit/allocate failed: Invalid account or account/partition combination specified"

**They need to specify '--account=<accounting group>'

** Workaround: use the /project or /scratch filesystems instead

** We're finding out whether this can be fixed through an update or reconfiguration.

** Solution: Request exceptions to be made at support@computecanada.ca   Describe what you need to access and why.

* Crontab is not offered on Graham. When attempting adding a new item there is an error during saving:

Crontab does work on Cedar. So, there must be some kind of a common approach on CC system.

Clearly, the main issue is how to handle user's crontabs on multiple login nodes.

But it's not clear whether we even want to do so.

#modules don't work for shells other than bash(sh) and tcsh

#*Workaround (this appears to work but not tested extensively)