Diskusage Explorer: Difference between revisions
No edit summary |
(Marked this version for translation) |
||
| Line 2: | Line 2: | ||
<translate> | <translate> | ||
==Content of Folders== | ==Content of Folders== <!--T:1--> | ||
<!--T:2--> | |||
<span style="color:red">Warning, at the moment this feature is only available on [[Béluga]]</span> | <span style="color:red">Warning, at the moment this feature is only available on [[Béluga]]</span> | ||
<!--T:3--> | |||
You can get a break down by folder on how the data is consumed in your project space. That information is currently updated once a day and is stored in a portable sqlite format. | You can get a break down by folder on how the data is consumed in your project space. That information is currently updated once a day and is stored in a portable sqlite format. | ||
<!--T:4--> | |||
Here is a walkthrough of how to look at your data where <code>$GROUP</code> will be the project space folder to investigate. | Here is a walkthrough of how to look at your data where <code>$GROUP</code> will be the project space folder to investigate. | ||
=== Ncurse User Interface === | === Ncurse User Interface === <!--T:5--> | ||
First list all the projects that you have access to: | First list all the projects that you have access to: | ||
<pre> | <pre> | ||
| Line 25: | Line 28: | ||
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | <br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | ||
<!--T:6--> | |||
Type <code>c</code> to toggle between consumed disk space and the number of files. Type <code>q</code> <code><esc></code> to quit, type <code>h</code> for help | Type <code>c</code> to toggle between consumed disk space and the number of files. Type <code>q</code> <code><esc></code> to quit, type <code>h</code> for help | ||
<!--T:7--> | |||
If I am only interested in a <code>/project/${GROUP}</code> subfolder and do not want to navigate the whole tree in the ncurse ui, | If I am only interested in a <code>/project/${GROUP}</code> subfolder and do not want to navigate the whole tree in the ncurse ui, | ||
<pre> | <pre> | ||
| Line 32: | Line 37: | ||
</pre> | </pre> | ||
<!--T:8--> | |||
A complete manual is available with the <code>man duc</code> command. | A complete manual is available with the <code>man duc</code> command. | ||
=== Graphical User Interface === | === Graphical User Interface === <!--T:9--> | ||
<!--T:10--> | |||
Note that on buzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy. For a better experience, read the section below and run duc on you own machine. | Note that on buzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy. For a better experience, read the section below and run duc on you own machine. | ||
<!--T:11--> | |||
So now you are warned, we recommend the use the standard <code>ui</code> mode on our clusters login nodes, but <code>duc</code> also includes a nice graphical user interface (GUI). | So now you are warned, we recommend the use the standard <code>ui</code> mode on our clusters login nodes, but <code>duc</code> also includes a nice graphical user interface (GUI). | ||
<!--T:12--> | |||
First make sure that you are connected to the cluster using ssh's <code>-Y</code> option | First make sure that you are connected to the cluster using ssh's <code>-Y</code> option | ||
<pre> | <pre> | ||
| Line 51: | Line 60: | ||
</pre> | </pre> | ||
<!--T:13--> | |||
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and their numbers | You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and their numbers | ||
<!--T:14--> | |||
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc gui tool]] | [[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc gui tool]] | ||
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | <br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | ||
=== Browse faster on your own machine === | === Browse faster on your own machine === <!--T:15--> | ||
<!--T:16--> | |||
First [http://duc.zevv.nl/#download install the duc software] on your local machine. Then always on your local machine, download the sqlite file from your cluster and run <code>duc</code>. | First [http://duc.zevv.nl/#download install the duc software] on your local machine. Then always on your local machine, download the sqlite file from your cluster and run <code>duc</code>. | ||
<!--T:17--> | |||
<pre> | <pre> | ||
MYUSERNAME=poq | MYUSERNAME=poq | ||
| Line 67: | Line 80: | ||
</pre> | </pre> | ||
<!--T:18--> | |||
Voilà! A smooth and satisfying browsing experience. | Voilà! A smooth and satisfying browsing experience. | ||
</translate> | </translate> | ||
Revision as of 17:06, 18 March 2021
Content of Folders
Warning, at the moment this feature is only available on Béluga
You can get a break down by folder on how the data is consumed in your project space. That information is currently updated once a day and is stored in a portable sqlite format.
Here is a walkthrough of how to look at your data where $GROUP will be the project space folder to investigate.
Ncurse User Interface
First list all the projects that you have access to:
ls ~/projects/
def-bourqueg def-lathrop def-poq-ab rrg-bourqueg-ad rrg-lathrop
We will take the def-poq-ab project as an example,
GROUP=def-poq-ab
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
This load a browser that states the resources consumed by all files under any directory tree
Type c to toggle between consumed disk space and the number of files. Type q <esc> to quit, type h for help
If I am only interested in a /project/${GROUP} subfolder and do not want to navigate the whole tree in the ncurse ui,
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}/some/subfolder/
A complete manual is available with the man duc command.
Graphical User Interface
Note that on buzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy. For a better experience, read the section below and run duc on you own machine.
So now you are warned, we recommend the use the standard ui mode on our clusters login nodes, but duc also includes a nice graphical user interface (GUI).
First make sure that you are connected to the cluster using ssh's -Y option
ssh -Y poq@beluga.calculcanada.ca
Then make the GUI appear
GROUP=def-poq-ab
duc gui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
You can navigate the folders with the mouse and still type c to toggle between the size of the files and their numbers
Browse faster on your own machine
First install the duc software on your local machine. Then always on your local machine, download the sqlite file from your cluster and run duc.
MYUSERNAME=poq
GROUP=def-poq-ab
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite .
duc gui -d ./${GROUP}.sqlite /project/${GROUP}
Voilà! A smooth and satisfying browsing experience.