Globus: Difference between revisions

<languages />

[https://www.globus.org/ Globus] is a service for fast, reliable, secure transfer of files. Designed specifically for researchers, Globus has an easy-to-use interface with background monitoring features that automate the management of file transfers between any two resources, whether they are at an Alliance cluster, another supercomputing facility, a campus cluster, lab server, desktop or laptop.

Globus leverages GridFTP for its transfer protocol but shields the end user from complex and time consuming tasks related to GridFTP and other aspects of data movement. It improves transfer performance over GridFTP, rsync, scp, and sftp, by automatically tuning transfer settings, restarting interrupted transfers, and checking file integrity.

Globus can be accessed via the main [https://www.globus.org/ Globus website] or via the Alliance Globus portal at [https://globus.computecanada.ca https://globus.computecanada.ca].

Go to [http://globus.computecanada.ca http://globus.computecanada.ca]. Your "existing organizational login" is your CCDB account. Ensure that "Compute Canada" is selected in the drop-down, then click Continue. Supply your CCDB username (not your e-mail address or other identifier) and password on the Compute Canada MyProxy page which appears. This takes you to the web portal for Globus.

[[File:1st-panel.png|400px|thumb|none| CC Globus Authentication page. (Click for larger image.)]]

Globus transfers happen between "collections" (formerly known as "endpoints" in previous Globus versions).  Most Alliance clusters have some standard collections set up for you to use.  To transfer files to and from your computer, you need to create a collection for it. This requires a bit of setup initially, but once it has been done, transfers via Globus require little more than making sure the Globus Connect Personal software is running on your machine. More on this below under [[#Personal Computers|Personal Computers]].

If the [https://globus.computecanada.ca/file-manager File Manager page in the Globus Portal] is not already showing (see image), select it from the left sidebar.

[[File:Globus-file-manager.png|400px|thumb|none| Globus File Manager. (Click for larger image.)]]

On the top right of the page there are three buttons labelled "Panels". Select the second button (this will allow you to see two collections at the same time).

[[File:Globus-select-collection.png|400px|thumb|none| Selecting a Globus collection. (Click for larger image.)]]

You can start typing a collection name to select it. For example, if you want to transfer data to or from the Béluga cluster, type "beluga", wait two seconds for a list of matching sites to appear, and select <code>computecanada#beluga-dtn</code>.  

All Compute Canada resources have names prefixed with <code>computecanada#</code>. For example, [https://globus.computecanada.ca/file-manager?origin_id=278b9bfe-24da-11e9-9fa2-0a06afd4a22e <code>computecanada#beluga-dtn</code>], [https://globus.computecanada.ca/file-manager?origin_id=c99fd40c-5545-11e7-beb6-22000b9a448b <code>computecanada#cedar-dtn</code>],  [https://globus.computecanada.ca/file-manager?origin_id=07baf15f-d7fd-4b6a-bf8a-5b5ef2e229d3 <code>computecanada#graham-globus</code>] or [https://globus.computecanada.ca/file-manager?origin_id=77506016-4a51-11e8-8f88-0a6d4e044368 <code>computecanada#niagara</code>] (note that 'dtn' stands for 'data transfer node').

You may be prompted to "authenticate" the collection, depending on which site is hosting the collection. For example, if you are activating a collection hosted on Graham, you will be asked for your Alliance username and password.  The authentication of a collection remains valid for some time - typically one week for CC collections while personal collections do not expire.

Now select a second collection, searching for it and authenticating if required.

Once a collection has been activated you should see a list of directories and files. You can navigate these by double-clicking on directories and using the "up one folder" button. Highlight a file or directory that you want to transfer by single-clicking on it. Control-click to highlight multiple things. Then click one of the big blue buttons with white arrowheads to initiate the transfer. The transfer job will be given a unique id and will begin right away. You will receive an email when the transfer is complete. You can also monitor in-progress transfers and view details of completed transfers from the ["Activity" button](https://globus.computecanada.ca/activity) on the left hand side .

[[File:Globus-Initiate-Transfer.png|400px|thumb|none| Initiating a transfer.  Note the highlighted file in the left-hand pane. (Click for larger image.)]]

See also [https://docs.globus.org/how-to/get-started/ How To Log In and Transfer Files with Globus] at the Globus.org site.

Globus provides several other options in the "Transfer & Sync Options" in between the two "Start" buttons in the middle of the screen. Here you can direct Globus to

* sync - only transfer new or changed files

Note that enabling encryption significantly reduces transfer performance, so it should only be used for sensitive data.

Globus provides a desktop client, [https://www.globus.org/globus-connect-personal Globus Connect Personal], to make it easy to transfer files to and from a personal computer running Windows, MacOS X, or Linux.

There are links on the [https://www.globus.org/globus-connect-personal Globus Connect Personal] page which walk you through the setup of globus connect personal on the various operating systems, including setting it up from the commandline on Linux. If you are running Globus Connect Personal from the command line on linux, this [https://docs.globus.org/faq/globus-connect-endpoints/#how_do_i_configure_accessible_directories_on_globus_connect_personal_for_linux FAQ on the Globus site] describes configuring which paths you share and their permissions.

[[File:GetGlobusConnectPersonal.png|400px|thumb|none| Finding the installation button. (Click for larger image.)]]

# Go to the [https://globus.computecanada.ca/endpoints?scope=administered-by-me Compute Canada Globus portal] and log in if you have not already done so.

# From the File Manager screen click on the "Collections" icon on the left hand side.

# Click on the "+ Get Globus Connect Personal" button in the top of the screen

# Click on the download link for your operating system (May need to click on "Show me other supported operating systems if downloading for another computer)

# You should now be able to access the endpoint through Globus. The full endpoint name is [your username]#[name you give setup] Example: smith#WorkPC

<!--T:25-->