Bureaucrats, cc_docs_admin, cc_staff
2,879
edits
Using nearline storage: Difference between revisions
clarify advice to create archive directly on nearline
No edit summary |
(clarify advice to create archive directly on nearline) |
||
| Line 20: | Line 20: | ||
<!--T:37--> | <!--T:37--> | ||
Use [[A tutorial on 'tar'|tar]] or [[dar]] to create an archive file | Use [[A tutorial on 'tar'|tar]] or [[dar]] to create an archive file. | ||
* Keep the source files on their original filesystem. Do NOT copy the source files to nearline before creating the archive! | |||
* Write the archive file directly to nearline. While writing it to a different filesystem and then moving it to nearline is not as bad as moving the source files to nearline and then writing the archive, it is unnecessary. | |||
<!--T:38--> | <!--T:38--> | ||
| Line 29: | Line 31: | ||
<!--T:48--> | <!--T:48--> | ||
When you bundle files, such as with tar, it becomes inconvenient to find individual files. | When you bundle files, such as with tar, it becomes inconvenient to find individual files. To avoid having to restore an entire large collection from tape, when you only need a one or a few of the files in it, you should save an index of all archive files you create. Construct an index as soon as you create the collection. For instance, you can save the output of tar with the "verbose" option when you create the archive, like this: | ||
<!--T:49--> | <!--T:49--> | ||
tar cvvf /nearline/def-sponsor/user/mycollection.tar /project/def-sponsor/user/something > /nearline/def-sponsor/user/mycollection.index | {{Command|tar cvvf /nearline/def-sponsor/user/mycollection.tar /project/def-sponsor/user/something > /nearline/def-sponsor/user/mycollection.index}} | ||
<!--T:50--> | <!--T:50--> | ||
If you've just created the archive (again using tar as an example), you can create an index like this: | |||
<!--T:51--> | <!--T:51--> | ||
tar tvvf /nearline/def-sponsor/user/mycollection.tar > /nearline/def-sponsor/user/mycollection.index | {{Command|tar tvvf /nearline/def-sponsor/user/mycollection.tar > /nearline/def-sponsor/user/mycollection.index}} | ||
==== No access from compute nodes ==== <!--T:40--> | ==== No access from compute nodes ==== <!--T:40--> | ||