Sharing data

Having to share some but not all of your data with a colleague or another research group is a common occurrence. Compute Canada systems provide a variety of mechanisms to facilitate this data sharing with colleagues. If the person you want to share the data with is a member of the same research group as you, then the best approach may be to make use of the project space that each research group have in common. At the opposite extreme, if the person you need to share the data with doesn't even have an account on the cluster, you can use use Globus and in particular what is called a shared endpoint to share the data. To handle the scenario of sharing with a colleague who has an account on the cluster but doesn't belong to common research group with you, the simplest approach is to use the permissions available in the filesystem to share the data, the principal topic of this page.

When sharing a file it's important to realize that the individual you want to share it with must have access to the entire chain of directories leading from /scratch or /project to the directory in which the file is located. If we consider the metaphor of a document locked in a safe in the bedroom of your apartment in a large building, giving me the combination to the safe will not allow me to read this document if I do not also have the necessary keys to enter the apartment building, your apartment and finally your bedroom. In the context of a filesystem, this means having the right to read and execute each directory between the root (e.g. /scratch or /project) and the directory containing the file.

Filesystem permissions

Like most modern filesystems, those used on Compute Canada clusters support the idea of permissions to read, write, and execute files and directories. When you attempt to read, modify or delete a file, or access a directory, e.g. with cd, the Linux kernel first verifies that you have the right to do this. If not, you'll see the error message "Permission denied". For each filesystem object (file or directory) there are three categories of users:

1. the object's owner --- normally the user who created the object,
2. members of the object's group --- normally the same as the owner's default group, and
3. everyone else.

Each of these categories of users may have the right to read, write, or execute the object. Three categories of users times three types of permission means there are nine permissions associated with each object.

You can see what the current permissions are for a filesystem object with the command

[name@server ~]$ ls -l name_of_object

which will print out the permissions for the owner, the group, and everyone else. For example, a file with permissions ‑rw‑r‑‑r‑‑ means the owner can read it and write it but not execute it, and the group members and everyone else can only read the file. You'll also see printed out the name of the object's owner and the group.

To change the permissions of a file or directory you can use the command chmod along with the user category, a plus or minus sign indicating that permission is granted or withdrawn, and the nature of the permission: read (r), write (w) or execute (x). For the user category we use the abbreviations u for the owner (user), g for the group and o for others, i.e. everyone else on the cluster. So a command like

[name@server ~]$ chmod g+r file.txt

would grant read permission to all members of the group that file.txt belongs to, while

[name@server ~]$ chmod o-x script.py

would withdraw execute permission for the file script.py to everyone but the owner and the group. We can also use the user category a to denote everyone (all), thus

[name@server ~]$ chmod a+r file.txt

grants everyone on the cluster the right to read file.txt.

It's also common for people to use "octal notation" when referring to Unix filesystem permissions even if this is somewhat less intuitive than the above symbolic notation. In this case, we use three bits to represent the permissions for each category of user, with these three bits then interpreted as a number from 0 to 7 using the formula (read_bit)*4 + (write_bit)*2 + (execute_bit)*1. In the above example the octal representation would be 4+2+0 = 6 for the owner and 4+0+0 = 4 for the group and everyone else, so 644 overall.

Note that to be able to exercise your rights on a file, you also need to be able to access the directory in which it resides. This means having both read and execute permission ("5" or "7" in octal notation) on the directory in question.

You can alter these permissions using the command chmod in conjunction with the octal notation discussed above, so for example

[name@server ~]$ chmod 777 name_of_file

means that everyone on the cluster now has the right to read, write and execute this file. Naturally you can only modify the permissions of a file or directory you own. You can also alter the group by means of the command chgrp.

Access control lists (ACLs)

The file permissions discussed above have been available in Unix-like operating systems for decades now but they are very coarse-grained. The whole set of users is divided into just three categories: the owner, the group, and everyone else. What if I want to allow a single user who isn't in my group to read a file? Do I really need to make the file readable by everyone in that case? No. The Compute Canada's national systems offer "access control lists" (ACLs) to enable permissions to be set on a user-by-user basis if desired. The two commands needed to manipulate these extended permissions are

• getfacl to see the ACL permissions, and
• setfacl to alter them.

If I want to allow a single person with username smithj to have read and execute permission on the file my_script.py I can achieve this with the command

[name@server ~]$ setfacl -m u:smithj:rx my_script.py

Note: setfacl and getfacl do not work on Graham /home. Use the /project or /scratch filesystems instead.