Diskusage Explorer: Difference between revisions
No edit summary |
No edit summary |
||
| Line 1: | Line 1: | ||
== | ==Content of Folders== | ||
<span style="color:red">Warning, at the moment this feature is only available on [[Béluga]]</span> | <span style="color:red">Warning, at the moment this feature is only available on [[Béluga]]</span> | ||
| Line 5: | Line 5: | ||
You can get a break down by folder on how the data is consumed in your project space. Here is a walkthrough of how to look at the data where GROUP is the project space folder to investigate. | You can get a break down by folder on how the data is consumed in your project space. Here is a walkthrough of how to look at the data where GROUP is the project space folder to investigate. | ||
=== Ncurse User Interface === | |||
First list all the projects that you have access to: | First list all the projects that you have access to: | ||
<pre> | <pre> | ||
| Line 21: | Line 21: | ||
Type <code>q</code> <code><esc></code> to quit, type <code>h</code> for help | Type <code>q</code> <code><esc></code> to quit, type <code>h</code> for help | ||
If I am only interested in a <code>/project/${GROUP}<code> subfolder and do not want to navigate the whole tree in the ncurse ui, | If I am only interested in a <code>/project/${GROUP}<code> subfolder and do not want to navigate the whole tree in the ncurse ui, | ||
| Line 27: | Line 26: | ||
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}/some/subfolder/ | duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}/some/subfolder/ | ||
</pre> | </pre> | ||
Type <code>c</code> to toggle between consumed disk space and the number of files. | |||
=== Graphical User Interface === | |||
Note that on buzzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy, we have a section below on how to run duc on you own machine. | |||
So now you are warned, we recommend to use the standard <code>ui</code> mode on our clusters login nodes, duc also includes a nice graphical user interface (GUI). | |||
First make sure you are connected to the cluster using ssh's <code>-Y</code> option | First make sure you are connected to the cluster using ssh's <code>-Y</code> option | ||
| Line 41: | Line 45: | ||
</pre> | </pre> | ||
You can navigate the folders with the mouse and type <code>c</code> to toggle between the size of the files and their numbers | You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and their numbers | ||
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc gui tool]] | [[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc gui tool]] | ||
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | <br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | ||
=== Look at your project space on your own machine === | |||
First [install the duc software](http://duc.zevv.nl/#download) in your local machine. Then always on your local machine, download the sqlite file from your cluster and run duc. | |||
<pre> | |||
MYUSERNAME=poq | |||
GROUP=def-poq-ab | |||
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite . | |||
duc gui -d ./${GROUP}.sqlite /project/${GROUP} | |||
</pre> | |||
Revision as of 03:23, 15 January 2021
Content of Folders
Warning, at the moment this feature is only available on Béluga
You can get a break down by folder on how the data is consumed in your project space. Here is a walkthrough of how to look at the data where GROUP is the project space folder to investigate.
Ncurse User Interface
First list all the projects that you have access to:
ls ~/projects/
def-bourqueg def-lathrop def-poq-ab rrg-bourqueg-ad rrg-lathrop
Here i will look at the def-poq-ab project,
GROUP=def-poq-ab
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
This load a browser that state the volume under all directory three
Type q <esc> to quit, type h for help
If I am only interested in a /project/${GROUP} subfolder and do not want to navigate the whole tree in the ncurse ui,
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}/some/subfolder/
Type c to toggle between consumed disk space and the number of files.
Graphical User Interface
Note that on buzzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy, we have a section below on how to run duc on you own machine.
So now you are warned, we recommend to use the standard ui mode on our clusters login nodes, duc also includes a nice graphical user interface (GUI).
First make sure you are connected to the cluster using ssh's -Y option
ssh -Y poq@beluga.calculcanada.ca
Then make GUI appear
GROUP=def-poq-ab
duc gui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
You can navigate the folders with the mouse and still type c to toggle between the size of the files and their numbers
Look at your project space on your own machine
First [install the duc software](http://duc.zevv.nl/#download) in your local machine. Then always on your local machine, download the sqlite file from your cluster and run duc.
MYUSERNAME=poq
GROUP=def-poq-ab
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite .
duc gui -d ./${GROUP}.sqlite /project/${GROUP}