Automation in the context of multifactor authentication/en: Difference between revisions

← Older edit

Automation in the context of multifactor authentication/en (view source)

Revision as of 18:33, 17 June 2024

5,328 bytes added , 4 months ago

Updating to match new version of source page

FuzzyBot

Bots

38,760

edits

@@ Line 1: / Line 1: @@
 <languages />
-Automated workflows which connect to the clusters without human intervention cannot make use of a second authentication factor. In order to execute such workflows after MFA becomes a requirement, you must request access to one of our special nodes. These nodes will not require the use of a second factor, but will be otherwise much more limited than regular login nodes in terms of the type of authentication they accept and the type of action that they can be used to perform.
+An automated workflow which involves some outside machine connecting to a cluster without human intervention cannot make use of a second authentication factor. In order to execute such a workflow now that MFA is a requirement, you must request access to an '''automation node'''. An automation node does not require the use of a second factor, but is much more limited than a regular login node in terms of the type of authentication it accepts and the types of actions that it can be used to perform.
-= Increased security restrictions =
+= Increased security measures =
 == Available only by request ==
-Users who need to make use of automated workflows for their research must first contact our [[technical support]] to be allowed to use these nodes. When contacting us, please explain in detail the type of automation you intend to use as part of your workflow. Tell us what commands will be executed and what tools or libraries you will be using to manage the automation.
+If you need to make use of an automated workflow for your research, contact our [[technical support]] and request access to an automation node. When contacting us, please explain in detail the type of automation you intend to use. Tell us what commands will be executed and what tools or libraries you will be using to manage the automation.
-== Available only through restricted SSH keys ==
+== Available only through constrained SSH keys ==
-The only accepted means of authentication for the automation nodes will be through [[SSH_Keys#Using_CCDB|SSH keys uploaded to the CCDB]]. SSH keys written in your <i>.ssh/authorized_keys</i> file are not accepted. In addition, the SSH keys <b>must</b> obey the following constraints.
+The only accepted means of authentication for the automation nodes is through [[SSH_Keys#Using_CCDB|SSH keys uploaded to the CCDB]]. SSH keys written in your <i>.ssh/authorized_keys</i> file are not accepted. In addition, the SSH keys <b>must</b> obey the following constraints.
 === <code>restrict</code> ===
@@ Line 14: / Line 14: @@
 === <code>from="pattern-list"</code> ===
-This constraint specifies that the key can only be used from IP addresses that match the patterns. This is to ensure that this key is not used from computers other than the ones intended. The patterns list must include only IP addresses that fully specify at least the network class, the network, and the subnet, which are the first 3 sections of an IP address. For example, <code>192.168.*.*</code> would not be accepted, but <code>192.168.1.*</code> would be accepted.
+This constraint specifies that the key can only be used from IP addresses that match the patterns. This is to ensure that this key is not used from computers other than the ones intended. The pattern list must include only IP addresses that fully specify at least the network class, the network, and the subnet, which are the first three elements of an IP address, for example, <code>x.y.*.*</code> would not be accepted, but <code>x.y.z.*</code> would be accepted. Also, the IP address must be a ''public'' IP address; thus anything like <code>10.0.0.0 – 10.255.255.255</code>, <code>172.16.0.0 – 172.31.255.255</code> and <code>192.168.0.0 – 192.168.255.255</code> is incorrect. You can use a site like [https://whatismyipaddress.com/ What Is My IP Address?] or the shell command <code>curl ifconfig.me</code> to learn your public IP address.
 === <code>command="COMMAND"</code> ===
@@ Line 20: / Line 20: @@
 == Convenience wrapper scripts to use for <code>command=</code> ==
-<code>command</code> constraints can specify any command, but they are most useful when using a wrapper script which will accept or reject commands based on which command is being called. You can write your own script, but for convenience, we provide a number of such scripts which will allow common actions. These scripts are defined in [https://github.com/ComputeCanada/software-stack-custom/tree/main/bin/computecanada/allowed_commands this git repository].
+<code>command</code> constraints can specify any command, but they are most useful when using a wrapper script which will accept or reject commands based on which command is being called. You can write your own script, but for convenience, we provide a number of such scripts which allow common actions. These scripts are defined in [https://github.com/ComputeCanada/software-stack-custom/tree/main/bin/computecanada/allowed_commands this git repository].
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/transfer_commands.sh</code> will allow only file transfers, such as <code>scp</code>, <code>sftp</code> or <code>rsync</code>.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/transfer_commands.sh</code> allows only file transfers, such as <code>scp</code>, <code>sftp</code> or <code>rsync</code>.
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/archiving_commands.sh</code> will allow commands to archive files, such as <code>gzip</code>, <code>tar</code> or <code>dar</code>.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/archiving_commands.sh</code> allows commands to archive files, such as <code>gzip</code>, <code>tar</code> or <code>dar</code>.
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/file_commands.sh</code> will allow commands to manipulate files, such as <code>mv</code>, <code>cp</code> or <code>rm</code>.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/file_commands.sh</code> allows commands to manipulate files, such as <code>mv</code>, <code>cp</code> or <code>rm</code>.
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/git_commands.sh</code> will allow the <code>git</code> command.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/git_commands.sh</code> allows the <code>git</code> command.
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/slurm_commands.sh</code> will allow some Slurm commands, such as <code>squeue</code>, <code>sbatch</code>.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/slurm_commands.sh</code> allows some Slurm commands, such as <code>squeue</code>, <code>sbatch</code>.
-* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/allowed_commands.sh</code> will allow all of the above.
+* <code>/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/allowed_commands.sh</code> allows all of the above.
 == Examples of accepted SSH keys ==
@@ Line 39: / Line 39: @@
 restrict,from="216.18.209.*",command="/cvmfs/soft.computecanada.ca/custom/bin/computecanada/allowed_commands/slurm_commands.sh" ssh-ed25519 AAAAC3NzaC1lZDI1NTE6AACAIExK9iTTDGsyqKKzduA46DvIJ9oFKZ/WN5memqG9Invw
 </pre>
+{{Warning|title=Warning|content=
+The constraints must be added directly as text in front of your key, before uploading the complete string in [https://ccdb.alliancecan.ca/ssh_authorized_keys your account].
+}}
 = Automation nodes for each cluster =
 Here is the hostname of the node to be used for unattended connections on each cluster:
 * Cedar: robot.cedar.alliancecan.ca
-* Graham: gra-platform.computecanada.ca port 222 (testing)
+* Graham: robot.graham.alliancecan.ca
-* Béluga: not available yet
+* Béluga: robot.beluga.alliancecan.ca
-* Narval: not available yet
+* Narval: robot.narval.alliancecan.ca
-* Niagara: robot.niagara.alliancecan.ca
+* Niagara: robot.niagara.alliancecan.ca    (you may also use robot1.niagara.alliancecan.ca or robot2.niagara.alliancecan.ca directly, one is the fallback for the other)
 = Using the right key =
@@ Line 57: / Line 61: @@
 With <code>rsync</code>:
 {{Command|rsync -e "ssh -i .ssh/private_key_to_use" ...}}
+It's often much more convenient to put these parameters into your ~/.ssh/config file, so they get picked up by any ssh client invocation.  For instance:
+ host robot
+  hostname robot.cluster.alliancecan.ca
+  user myrobot
+  identityfile ~/.ssh/my-robot-key
+  identitiesonly yes
+  requesttty no
+this means that the following kinds of commands will do what you want:
+{{Command|ssh robot /usr/bin/ls}}
+{{Command|rsync -a datadir/a robot:scratch/testdata}}
+= IPv4 vs IPv6 issue =
+When connecting to a automation node the SSH client on your computer may choose to use the '''IPv6 addressing''' over the older '''IPv4'''.
+This seems to be more probable in a Windows environment.
+If this is the case you have to make sure that the IP address mask you put in the <code>restrict,from=</code> field of the key
+matches the type your computer will be using when connecting to the node.
+You can check your addresses using this web site: https://test-ipv6.com/ .
+* An IPv4 address could look like '''199.241.166.5'''.
+* An IPv6 address could look like '''2620:123:7002:4::5'''.
+The possible problem is that if you put the IPv4 address mask, '''199.241.166.*''' into the CCDB SSH key, and
+your SSH client will be connecting to the automation node using IPv6 address, the source address will not match the mask in the key
+and the key will not be accepted by the automation node.
+=== How to identify the problem ===
+If you are having difficulties to make the SSH connection to an automation node work,
+try this test command:
+ ssh -i ~/.ssh/automation_key -vvv username@robot.graham.alliancecan.ca "ls -l"
+This tries to connect to the automation node at Graham and execute the <code>ls -l</code> command
+using the <code>~/.ssh/automation_key</code> SSH key.
+Then it prints the list of files in your home directory on Graham to screen.
+This command will produce a lot of debug output due to the <code>-vvv</code> option ("Very Very Verbose").
+Look for the '''Connecting to...''' message there.
+If it says something like this:
+ debug1: Connecting to robot.graham.alliancecan.ca [199.241.166.5] port 22.
+it means that IPv4 is being used.
+If the message is similar to
+ debug1: Connecting to robot.graham.alliancecan.ca [2620:123:7002:4::5] port 22.
+then IPv6 is being used to make the connection.
+=== Possible solutions ===
+* You can make the SSH client to '''explicitly use either IPv4 or IPv6''' using the <code>-4</code> and <code>-6</code> options, respectively, to match the format you used for the key in CCDB.
+* You can try using an '''IP address instead of the name''' to point to the automation node. Using Graham example, try using the
+: <code>ssh -i ~/.ssh/automation_key -vvv username@199.241.166.5 "ls -l"</code>
+: instead, to force SSH to use the IPv4 addresses.
+* You can try to '''disable the IPv6 addressing''' for your computer, to make sure that only IPv4 is used.
+: Currently, there should not be any negative impact on your computer. However, Microsoft does not recommend this, and this should be your '''last resort''' method, if nothing else works.
+: How to disable IPv6 will depend on your operating system.
+= Automation using Python and Paramiko =
+If you are using the [https://www.paramiko.org/index.html Paramiko Python module] to automate your workflow, this is how you can make it work with the automation nodes:
+<source lang=python>
+# ====================================================================================================
+#! /usr/bin/env python3
+# ====================================================================================================
+import os
+import paramiko
+# ====================================================================================================
+key = paramiko.Ed25519Key.from_private_key_file("/home/username/.ssh/cc_allowed")
+user = "username"
+host = "robot.graham.alliancecan.ca"
+ssh = paramiko.SSHClient()
+# If the host is not known, it is OK.
+ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+ssh.connect(hostname=host, username=user, pkey=key)
+cmd = "ls -l"
+stdin, stdout, stderr = ssh.exec_command(cmd)
+print("".join(stdout.readlines()))
+ssh.close()
+# ====================================================================================================
+</source>
+This code connects to the automation node on '''Graham''' using an key specified in CCDB and
+executes the <code>ls -l</code> command to get the list of files.
+Then prints the list to the screen.
+Note that it is important to <b>install Paramiko</b> with the
+ $ pip install paramiko[all]
+command. This will make sure that the support for the '''Ed25519''' key type will also be installed.