SSH security improvements

SSH changes (Summer 2019)

With constant increase in computing power over time, some encryption algorithms and protocols which were reasonably secure ten or fifteen years ago now pose an unacceptable risk of the connection being compromised by a third party. For this reason, Compute Canada is modifying its policies and practices regarding SSH, the principal tool used to provide secure access to its clusters. Some users may have to update their SSH client software, some may have to generate a new public/private key-pair, and everyone will have to update the local copy of the "host key" which is used to identify each Compute Canada cluster.

What is changing?

During the summer of 2019, we will make the following SSH security improvements on Compute Canada clusters:

1. Disable certain encryption algorithms.
2. Disable certain public key types.
3. Regenerate the cluster's host keys.

If you do not understand the significance of "encryption algorithms", "public keys", or "host keys", do not be alarmed. Simply follow the steps outlined below.

Updating your client's known host list

The first time you login to a Compute Canada cluster after the changes, you will probably see the following warning message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:mf1jJ3ndpXhpo0k38xVxjH8Kjtq3o1+ZtTVbeM0xeCk.
Please contact your system administrator.
Add correct host key in /home/username/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/username/.ssh/known_hosts:109
ED25519 host key for graham.computecanada.ca has changed and you have requested strict checking.
Host key verification failed.
Killed by signal 1.

This warning is displayed because the host keys on the cluster (in this case Graham) were changed, and your SSH client software remembers the old host keys. (It does this to prevent "man-in-the-middle" attacks.)

You may also get a warning regarding "DNS spoofing", which is related to the same change.

MobaXterm, PuTTY, or WinSCP

If you are using MobaXterm, PuTTY, or WinSCP as your ssh (or scp) client under Windows, the warning will appear in a pop-up window and will allow you to accept the new host key by clicking "Yes". Only click yes if the fingerprint matches one listed in SSH host key fingerprints. If the fingerprint does not match any on the list, do not accept the connection, and contact Technical support with the details.

macOS, Linux, GitBash or Cygwin

If you are using the command line ssh command on macOS, Linux, GitBash or Cygwin, you should tell your system to "forget" the old host key by running one of the following commands:

ssh-keygen -R graham.computecanada.ca
ssh-keygen -R cedar.computecanada.ca
ssh-keygen -R beluga.computecanada.ca

Afterwards, the next time you ssh to the cluster you'll be asked to confirm the new host keys, e.g.:

$ ssh graham.computecanada.ca
The authenticity of host 'graham.computecanada.ca (142.150.188.70)' can't be established.
ED25519 key fingerprint is SHA256:mf1jJ3ndpXhpo0k38xVxjH8Kjtq3o1+ZtTVbeM0xeCk.
ED25519 key fingerprint is MD5:bc:93:0c:64:f7:e7:cf:d9:db:81:40:be:4d:cd:12:5c.
Are you sure you want to continue connecting (yes/no)? 

Only type yes if the fingerprint matches one listed in the SSH host key fingerprints. If the fingerprint does not match any on the list below, do not accept the connection, and contact Technical support with the details.

Troubleshooting

Can I test my SSH client before the change?

Yes. We have set up a server that you can use to test how your SSH client software works with the new protocols. Connect to it at ssh-test.computecanada.ca using your regular Compute Canada credentials. If you can connect successfully then your client will be able to connect to our clusters after the SSH changes are made in the summer of 2019.

See the list of SSH host key fingerprints.

My SSH key no longer works

If you're being asked for a password, but were using SSH keys previously on the same system, it's likely because 1024-bit DSA & RSA keys have been disabled.

You need to generate a new stronger key. The process for doing this depends on the operating system you use, either Windows or Linux/macOS. Those instructions also describe how to add your client's new public key to the remote host, so that you can authenticate with the key rather than needing to provide a password.

I can't connect!

If you see one of the following error messages:

Unable to negotiate with 142.150.188.70 port 22: no matching cipher found.
Unable to negotiate with 142.150.188.70 port 22: no matching key exchange method found.
Unable to negotiate with 142.150.188.70 port 22: no matching mac found.

you need to upgrade your SSH client to one of the compatible clients shown below.

Which clients are compatible with the new configuration?

The list below is not exhaustive, but we have tested the configuration with the following clients. Earlier versions of these clients may or may not work. We recommend that you update your operating system and SSH client to the latest version compatible with your hardware.

Linux clients

• OpenSSH_7.4p1, OpenSSL 1.0.2k-fips (CentOS 7.5, 7.6)
• OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13, OpenSSL 1.0.1f (Ubuntu 14)

OS X clients

You can determine the exact version of your SSH client on OS X using the command ssh -V.

• OpenSSH 7.4p1, OpenSSL 1.0.2k (Homebrew)
• OpenSSH 7.9p1, LibreSSL 2.7.3 (OS X 10.14.5)

Windows clients

• MobaXterm Personal Edition v11.1

iOS clients

• Termius, 4.3.12

SSH host key fingerprints

Béluga

ED25519

SHA256:lwmU2AS/oQ0Z2M1a31yRAxlKPcMlQuBPFP+ji/HorHQ

MD5:2d:d7:cc:d0:85:f9:33:c1:44:80:38:e7:68:ce:38:ce

RSA

SHA256:7ccDqnMTR1W181U/bSR/Xg7dR4MSiilgzDlgvXStv0o

MD5:7f:11:29:bf:61:45:ae:7a:07:fc:01:1f:eb:8c:cc:a4

Cedar

ED25519

• SHA256:a4n68wLDqJhxtePn04T698+7anVavd0gdpiECLBylAU
• MD5:f8:6a:45:2e:b0:3a:4b:16:0e:64:da:fd:68:74:6a:24

RSA

• SHA256:91eMtc/c2vBrAKM0ID7boyFySo3vg2NEcQcC69VvCg8
• MD5:01:27:45:a0:fd:34:27:9e:77:66:b0:97:55:10:0e:9b

Graham

ED25519

• SHA256:mf1jJ3ndpXhpo0k38xVxjH8Kjtq3o1+ZtTVbeM0xeCk
• MD5:bc:93:0c:64:f7:e7:cf:d9:db:81:40:be:4d:cd:12:5c

RSA

• SHA256:tB0gbgW4PV+xjNysyll6JtDi4aACmSaX4QBm6CGd3RM
• MD5:21:51:ca:99:15:a8:f4:92:3b:8e:37:e5:2f:12:55:d3

Narval

ED25519

• SHA256:pTKCWpDC142truNtohGm10+lB8gVyrp3Daz4iR5tT1M
• MD5:79:d5:b2:8b:c6:2c:b6:3b:79:d2:75:0e:3b:31:46:17

RSA

• SHA256:tC0oPkkY2TeLxqYHgfIVNq376+RfBFFUZaswnUeeOnw
• MD5:bc:63:b5:f9:e6:48:a3:b7:0d:4a:23:26:a6:31:19:ef

Niagara

ED25519

• SHA256:SauX2nL+Yso9KBo2Ca6GH/V9cSFLFXwxOECGWXZ5pxc
• MD5:b4:ae:76:a5:2b:37:8d:57:06:0e:9a:de:62:00:26:be

RSA

• SHA256:k6YEhYsI73M+NJIpZ8yF+wqWeuXS9avNs2s5QS/0VhU
• MD5:98:e7:7a:07:89:ef:3f:d8:68:3d:47:9c:6e:a6:71:5e

ssh-test.computecanada.ca

ED25519 (256b)

SHA256:Tpu6li6aynYkhmB83Q9Sh7x8qdhT8Mbw4QcDxTaHgxY

MD5:33:8f:f8:57:fa:46:f9:7f:aa:73:e2:0b:b1:ce:66:38

RSA (4096b)

SHA256:DMSia4nUKIyUhO5axZ/As4I8uqlaX0jPcJvcK93D2H0

MD5:a7:08:00:7c:eb:81:f2:f7:2f:5a:92:b0:85:e3:e8:5d

Mp2

ED25519 (256b)

SHA256:hVAo6KoqKOEbtOaBh6H6GYHAvsStPsDEcg4LXBQUP50

MD5:44:71:28:23:9b:a1:9a:93:aa:4b:9f:af:8d:9b:07:01

RSA (4096b)

SHA256:XhbK4jWsnoNNjoBudO6zthlgTqyKkFDtxiuNY9md/aQ

MD5:88:ef:b0:37:26:75:a2:93:91:f6:15:1c:b6:a7:a9:37

Siku

ED25519 (256b)

SHA256:F9GcueU8cbB0PXnCG1hc4URmYYy/8JbnZTGo4xKflWU

MD5:44:2b:1d:40:31:60:1a:83:ae:1d:1a:20:eb:12:79:93

RSA (2048b)

SHA256:cpx0+k52NUJOf8ucEGP3QnycnVkUxYeqJQMp9KOIFrQ

MD5:eb:44:dc:42:70:32:f7:61:c5:db:3a:5c:39:04:0e:91