Known issues/en: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Updating to match new version of source page)
(Updating to match new version of source page)
 
(41 intermediate revisions by the same user not shown)
Line 1: Line 1:
<languages />
<languages />
= Report an issue =
* Please report issues to [mailto:support@computecanada.ca support@computecanada.ca].


= Shared issues =
Problems that affect many users and are being investigated, such as a cluster-wide malfunction or outage, are on [https://status.alliancecan.ca/ the Alliance Status page]. This "Known issues" page describes problems that affect many users but that may take some time to repair, or are not planned for repair at this time. Problems that only affect a specific software package are described on the wiki page for that software package.  
* The status page at http://status.computecanada.ca/ is not updated automatically yet, so may lag in showing current status.
* Utilization accounting is not currently being forwarded to the [https://ccdb.computecanada.ca Compute Canada database].


== Scheduler problems ==
==Report an issue==
* The CC Slurm configuration encourages whole-node jobs. When appropriate, users should request whole-node rather than per-core resources. See [[Job_scheduling_policies#Whole_nodes_versus_cores;|Job Scheduling - Whole Node Scheduling]].
Please report issues to the [[Technical Support|technical support]] team.
* CPU and GPU backfill partitions have been created on the Cedar and Graham clusters. If a job is submitted with <24hr runtime, it will be automatically entered into the cluster-wide backfill partition. This partition has low priority, but will allow increased utilization of the cluster by serial jobs.
* Slurm epilog does not fully clean up processes from ended jobs, especially if the job did not exit normally. 
** This has been greatly improved after the addition of the epilog.clean script, but there are still nodes occasionally marked down for epilog failure.
* By default, the job receives environment settings from the submitting shell. This can lead to irreproducible results if it's not what you expect. To force the job to run with a fresh-like login environment, you can submit with <tt>--export=none</tt> or add <tt>#SBATCH --export=NONE</tt> to your job script.


==Shared issues==
The [https://status.alliancecan.ca/ status page] is updated manually, so there may be a delay between when a problem begins and when it is posted to the status page.


== Quota and filesystem problems ==
===Scheduler issues===
=== Quota errors on /project filesystem ===
Users will sometimes see a quota error on their project folders. This may happen when files are owned by a group other than the project group. You can change the group which owns files using the command
{{Command|chgrp -R <group> <folder>}}


To see what the value of <group> should be, run the following command:
No known issues.
{{Command|stat -c %G $HOME/projects/*/}}


Only the owner of the files can run the <tt>chgrp</tt> command. To ask us to correct the group owner for many users, write to  [mailto:support@computecanada.ca CC support].
===Quota and filesystem issues===


=== Nearline ===
====Missing project folder====
* Nearline capabilities are not yet available; see https://docs.computecanada.ca/wiki/National_Data_Cyberinfrastructure for a brief description of the intended functionality.
Upon creation of a new account for a Principal Investigator, the [[Project layout|<code>/project</code>]] storage space might not be allocated until the next business day.
** July 17 update: still not working. If you need your nearline RAC2017 quota, please ask [mailto:support@computecanada.ca CC support].  


=== Missing symbolic links to project folders ===
==Cluster-specific issues==
* Upon login to the new clusters, symbolic links are not always created in the user's account, as described in [[Project layout]]. If this is the case, please verify that your access to the cluster is enabled on this page [https://ccdb.computecanada.ca/services/resources https://ccdb.computecanada.ca/services/resources].


= Cedar only =
===Béluga===
* Some I/O operations on /project yield the error, "Cannot send after transport endpoint shutdown". We are working with the vendor to resolve this.
No known issues.
* <del>Slurm operations will occasionally time out with a message such as ''Socket timed out on send/recv operation'' or ''Unable to contact slurm controller (connect failure)''. As a temporary workaround, attempt to resubmit your jobs/commands; they should go through in a few seconds. </del>
** Should be resolved after a VHD migration to a new backend for slurmctl.
*Some people are getting the message ''error: Job submit/allocate failed: Invalid account or account/partition combination specified''.
** For users with only one account, this should be resolved in the latest version of the job submission script (Nathan Wielenga 2017-08-18).
** If you still see this, specify <tt>--account=<accounting group></tt> in you job submission. See [https://docs.computecanada.ca/wiki/Running_jobs#Accounts_and_projects https://docs.computecanada.ca/wiki/Running_jobs]


= Graham only =
===Cedar===
* /home is on an NFS appliance that does not support ACLs, so setfacl/getfacl doesn't work there.
No known issues.
** Workaround: use the /project or /scratch filesystems instead.
** Might be resolved by an update or reconfiguration.
* Compute nodes cannot access Internet
** Solution: Request exceptions to be made at  [mailto:support@computecanada.ca CC support] describing what you need to access and why.


* Crontab is not offered on Graham. When attempting the addition of a new item, there is an error during saving:
===Graham===
<pre>
Graham's /scratch is often slow; it will be replaced soon.
[rozmanov@gra-login1 ~]$ crontab -e
no crontab for rozmanov - using an empty one
crontab: installing new crontab
/var/spool/cron/#tmp.gra-login1.XXXXKsp8LU: Read-only file system
crontab: edits left in /tmp/crontab.u0ljzU
</pre>
As crontab works on Cedar, there must be some kind of common approach for CC systems.
Clearly, the main issue is how to handle user's crontabs on multiple login nodes; it is not clear however if this should be implemented.


= Other issues =
===Narval===
#Modules don't work for shells other than bash(sh) and tcsh.
No known issues.
#*Workaround: (this appears to work but not tested extensively)
#**<tt>source $LMOD_PKG/init/zsh</tt>
#**<tt>source $LMOD_PKG/init/ksh</tt>

Latest revision as of 22:23, 28 October 2024

Other languages:

Problems that affect many users and are being investigated, such as a cluster-wide malfunction or outage, are on the Alliance Status page. This "Known issues" page describes problems that affect many users but that may take some time to repair, or are not planned for repair at this time. Problems that only affect a specific software package are described on the wiki page for that software package.

Report an issue

Please report issues to the technical support team.

Shared issues

The status page is updated manually, so there may be a delay between when a problem begins and when it is posted to the status page.

Scheduler issues

No known issues.

Quota and filesystem issues

Missing project folder

Upon creation of a new account for a Principal Investigator, the /project storage space might not be allocated until the next business day.

Cluster-specific issues

Béluga

No known issues.

Cedar

No known issues.

Graham

Graham's /scratch is often slow; it will be replaced soon.

Narval

No known issues.

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Known_issues/en&oldid=163505"