Known issues
Jump to navigation
Jump to search
Report an issue
- Please report issues to the technical support team.
- The status page at http://status.computecanada.ca/ is not updated automatically yet, so may lag in showing current status.
- CC Clusters are vulnerable to the recent Meltdown/Spectre vulnerabilities, and will be updated, which involves updating the OS and CPU microcode. Read more at Meltdown and Spectre bugs.
Scheduler issues
- Slurm can report: "Exceeded step memory limit at some point" which may be surprising and can cause a problem with dependent jobs.
- File I/O uses memory and Slurm is correctly reporting this usage. This usage (primarily delayed writes) was not as visible in previous systems. The kernel usually resolves such memory shortages by flushing writes to the filesystem.
- Memory shortage can cause the kernel to kill processes ("OOM kill"), which results in the same message but affects exit status differently.
- A job that reports DerivedExitStatus 0:125 indicates hitting the memory limit, but not being OOM-killed.
- In the absence of any other action, a step with 0:125 will *not* enable a job which has afterok dependency. This is a Slurm bug that will be fixed in 17.11.3, so that Slurm can distinguish between the warning condition versus actual kernel OOM-kill events. Slurm will continue to limit memory usage from cgroups, so I/O memory will still be counted and be reported when exceeds the job's requested memory.
- The CC Slurm configuration encourages whole-node jobs. When appropriate, users should request whole-node rather than per-core resources. See Job Scheduling - Whole Node Scheduling.
- By default, the job receives environment settings from the submitting shell. This can lead to irreproducible results if it's not what you expect. To force the job to run with a fresh-like login environment, you can submit with --export=none or add #SBATCH --export=NONE to your job script.
Quota and filesystem problems
Quota errors on /project filesystem
- This topic has been moved to Frequently Asked Questions.
Nearline
- Nearline capabilities are not yet available; see https://docs.computecanada.ca/wiki/National_Data_Cyberinfrastructure for a brief description of the intended functionality.
- July 17 update: still not working. If you need your nearline RAC2017 quota, contact technical support.
Missing symbolic links to project folders
- Upon login to the new clusters, symbolic links are not always created in the user's account, as described in Project layout. If this is the case, please verify that your access to the cluster is enabled on this page https://ccdb.computecanada.ca/services/resources.
Cedar only
Nothing to report at this time.
Graham only
- We are currently updating the compute and login nodes due to the recent Meltdown/Spectre issue. Nodes will be rebooted in succession so that service will not be interrupted.
- Compute nodes cannot access Internet
- Solution: Contact technical support to request exceptions to be made; describe what you need to access and why.
- Crontab is not offered on Graham.
Other issues
- Modules don't work for shells other than bash(sh) and tcsh.
- Workaround: (this appears to work but not tested extensively)
- source $LMOD_PKG/init/zsh
- source $LMOD_PKG/init/ksh
- Workaround: (this appears to work but not tested extensively)