Visual Studio Code
Visual Studio Code is an integrated development environment (IDE) from Microsoft which can be used for remote development on our clusters over SSH. In this context, users can run the graphical interface of Visual Studio Code locally, on their personal computer, while connecting to a cluster where the source code is located and where the debugging and testing operations are being executed. While Visual Studio Code may be used for development in a variety of programming languages, in this page we will focus on its use with Python.
When using an IDE like Visual Studio Code, it's important to distinguish what sort of action you're performing, even if all of them take place within the context of the IDE. Editing a Python script or any other text file of reasonable size (up to a few thousand lines) is a very light-weight process which requires little processing power or memory but an IDE offers you numerous other possibilities, including the ability to debug your script, for example. In this case, the IDE is now executing your Python code on the remote server and until the script crashes because of a bug, it may well consume an entire CPU core or even several if the script or the libraries which it uses are multithreaded.
Note that if you are using Visual Studio Code with Python, you should avoid installing Conda and its variants as they are poorly adapted to our cluster environment. Alternatives to the use of Conda include Python virtual environments and Apptainer.
Configuring remote access
Configuration of your SSH keys
- If not done already, generate your SSH key. For example, we will name it
~/.ssh/ccdb
. - If not done already, install your public SSH key on CCDB.
- Copy the same public key to
~/.ssh/authorized_keys
on the remote cluster (create this file if it does not exist). That is because compute nodes do not have access to the public keys that are installed on CCDB.
SSH configuration file
VS Code works well with your local SSH configuration file (~/.ssh/config
).
Here are the recommended options :
Host *
ServerAliveInterval 300
Host beluga cedar graham narval
HostName %h.alliancecan.ca
IdentityFile ~/.ssh/ccdb
User your_username
Host bc????? bg????? bl?????
ProxyJump beluga
IdentityFile ~/.ssh/ccdb
User your_username
Host cdr*
ProxyJump cedar
IdentityFile ~/.ssh/ccdb
User your_username
Host gra1* gra2* gra3* gra4* gra5* gra6* gra7* gra8* gra9*
ProxyJump graham
IdentityFile ~/.ssh/ccdb
User your_username
Host nc????? ng????? nl?????
ProxyJump narval
IdentityFile ~/.ssh/ccdb
User your_username
First connection with VS Code
Remote access and code editing
https://carleton.ca/scs/2023/vscode-remote-access-and-code-editing/
Because some clusters do not provide access to the internet from compute nodes, the installation of VS Code Server must be done prior to using a remote connection to any compute node. Therefore, in VS Code, a first connection to any login node is required - select either :
- beluga
- cedar
- graham
- narval
Note : you will be prompted many times for your SSH key passphrase. If not, make sure to copy your SSH public key as described above in point 3 under Configuration of your SSH keys.
That first connection will automatically install VS Code Server in ~/.vscode-server/
.
The installation can take up to 5 minutes.
Once done, close the connection.
Connection with MFA enabled
If the multifactor authentication is enabled, you will need to check the "details" of the connection, which will bring you to the VS Code Terminal in which you may be prompted for your 2nd factor of authentication.
Closing your connection
When closing the local VS Code window, the remote process of VS Code Server may keep running in the background, which tends to accumulate orphan processes on that login node. Not only that, but your next connection may connect to a different login node, which can cause issues and confusion. To cleanly close a connection, click on the bottom-left corner of VS Code and select Close Remote Connection at the top of the window.
Connection to a compute node
Connection procedure:
- Make sure VS Code Server is installed as described in the previous sections.
- In an external SSH client connected to the cluster, start a new interactive job (with
salloc
).- Important: make sure to request enough memory (at least
2000M
). - Take note of the allocated compute node name.
- If you will need to work with
SLURM_*
environment variables in VS Code, save them all in a source file:[name@server ~]$ env | grep SLURM_ | sed -e 's/^\(.*\)=\(.*\)$/export \1="\2"/g' > slurm_var.sh
- Important: make sure to request enough memory (at least
- In VS Code, start a new remote session with the name of the allocated compute node:
- Press
F1
orCtrl+Shift+P
to start the command prompt>
in the Command Palette. - Start typing Remote and select Remote-SSH: Connect to Host... > Remote-SSH: Connect to Host...
- Enter the noted compute node name.
- If you get prompted for the type of operating system, select Linux
- Press
- If you need to work with
SLURM_*
environment variables, navigate to the working directory in a VS Code terminal and source theslurm_var.sh
file:[name@server ~]$ source slurm_var.sh
Troubleshooting
The remote session does not work anymore
The newest versions of VS Code Server no longer support CentOS 7, which affects connections to Graham and Niagara. Users can update their version of VS Code to 1.86.1 or later in order to connect, but will see warnings about the unsupported operating system.
- Note: an older version of VS Code still works through JupyterHub.
Because VS Code is intended for a different use case than developing on our clusters, having several instances of VS Code Server running on different login nodes can lead to issues. To fix them:
- Connect to all login nodes and stop (with
kill <PID>
) all VS Code processes you see in the output ofps aux | grep $USER
ortop -u $USER
. - If the above does not fix the problem, carefully delete the content of the
~/.vscode-server
directory, and then reconnect for a fresh installation of VS Code Server.
- We recently determined that in the last week of August/2024 a bug with the Remote-SSH extension for VSCode was introduced. As far as we could ascertain, scp is not working as expected. That is not our software, this is a Microsoft product, so we offer best effort support.
Here is some more info on the bug:
https://github.com/microsoft/vscode-remote-release/issues/10174#issuecomment-2316629187
Our suggestions are
- downgrade to VSCode 1.92
- use wget (https://github.com/microsoft/vscode-remote-release/issues/10174#issuecomment-2316093550) by adding "remote.SSH.useCurlAndWgetConfigurationFiles: true" to Remote-SSH settings it should replace the defective scp functionality.
- use the "Remote Tunnels" extension by running 'code tunnel' (google vscode cli and download it). Use github or a microsoft account (OAuth) and connect to a remote vscode server, such as a niagara login node. You can have all the "benefits" of not using mfa as long as your server is running. See Developing with Remote Tunnels.
See also
- SHARCNET General Interest Webinar, "Remote Development on Clusters with VSCode", presented by Armin Sobhani:
- Tutorial video created by two users (best viewed in 720p):