Transferring data: Difference between revisions

Jump to navigation Jump to search

Transferring data (view source)

Revision as of 21:28, 1 February 2017

65 bytes added ,  7 years ago
→‎Synchronize or verify files after transfer
Line 18: Line 18:


==Synchronize or verify files after transfer==
==Synchronize or verify files after transfer==
There are several tools that you can use to verify that your files have transferred safely or to synchronize ("sync") a changed dataset to update a location.
Two synchronize or "sync" two files (or two directories) stored in two different locations means to ensure that the two copies are the same. Here are several different ways to do this.


===Globus Transfer===
===Globus Transfer===
For performance and reliabilty reasons we recommend Globus Transfer.
We find Globus Transfer usually gives the greatest performance and reliability.


Normally when a Globus Transfer is initiated it will overwrite all the files on the destination with the files from the source, which potentially means you would transfer all of the files. Instead, if you go to the bottom of the transfer window as shown in the screenshot you can choose to "sync" instead.
Normally when a Globus Transfer is initiated it will overwrite the files on the destination with the files from the source, which means all of the files on the source will be transferred. If some of the files may already exist on the destination, and need not be transferred if they match, you should go to the bottom of the transfer window as shown in the screenshot and choose to "sync" instead.


[[File:Globus_Transfer_Sync_Options.png|280px|thumb|left]]
[[File:Globus_Transfer_Sync_Options.png|280px|thumb|left]]


This gives you the option to only transfer new or changed files if:
You may choose how Globus decides which files to transfer:
{| class="wikitable"
{| class="wikitable"
|-
|-
| Their checksums are different  || This is the slowest option but most accurate. This will catch errors that may have resulted in the same size of file, but with different contents.
| Their checksums are different  || This is the slowest option but most accurate. This will catch changes or errors that result in the same size of file, but with different contents.
|-
|-
| File doesn't exist on destination || This will only transfer new files that have been created since the last transfer / sync which is useful if you are incrementally creating files.
| File doesn't exist on destination || This will only transfer files that have been created since the last sync. Useful if you are incrementally creating files.
|-
|-
| File size is different || A quick process that checks to see if data has been removed / added to a file so that its size changed and therefore needs to be re-transferred
| File size is different || A quick test. If the file size has changed then its contents must have changed, and it will be re-transferred.
|-
|-
| Modification time is newer || It will check the file's recorded modification time and only transfer the file if it is newer on the source than the destination. If you want to depend on this it is important to check the "preserve source file modification times" option when initiating a Globus Transfer
| Modification time is newer || This will check the file's recorded modification time and only transfer the file if it is newer on the source than the destination. If you want to depend on this it is important to check the "preserve source file modification times" option when initiating a Globus Transfer.
|}
|}


For more information about Globus please see our [[Globus]].
For more information about Globus please see [[Globus]].


===Rsync===
===Rsync===
Rdickson
Bureaucrats, cc_docs_admin, cc_staff
2,879

edits

Retrieved from "https://docs.alliancecan.ca/wiki/Special:MobileDiff/8355"

Navigation menu