Diskusage Explorer: Difference between revisions
No edit summary |
No edit summary |
||
| Line 24: | Line 24: | ||
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP} | duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP} | ||
</pre> | </pre> | ||
This | This command loads a browser that shows the resources consumed by all files under any directory tree | ||
[[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating you project space with duc ncurse tool]] | [[File:Ncurse duc.png|thumb|using|450px|frame|left| Navigating you project space with duc ncurse tool]] | ||
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | <br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | ||
<!--T:6--> | <!--T:6--> | ||
Type <code>c</code> to toggle between consumed disk space and the number of files | Type <code>c</code> to toggle between consumed disk space and the number of files, <code>q</code> or <code><esc></code> to quit and <code>h</code> for help | ||
<!--T:7--> | <!--T:7--> | ||
| Line 39: | Line 39: | ||
<!--T:8--> | <!--T:8--> | ||
A complete manual page is available with the <code>man duc</code> command. | A complete manual page is available with the <code>man duc</code> command. | ||
=== Graphical User Interface === <!--T:9--> | === Graphical User Interface === <!--T:9--> | ||
| Line 47: | Line 46: | ||
<!--T:11--> | <!--T:11--> | ||
Note that Compute Canada recommends the use of the standard text-based <code>ui</code> mode on our cluster login nodes but <code>diskusage_explorer</code> does also include a nice graphical user interface (GUI). | |||
<!--T:12--> | <!--T:12--> | ||
| Line 54: | Line 53: | ||
ssh -Y poq@beluga.calculcanada.ca | ssh -Y poq@beluga.calculcanada.ca | ||
</pre> | </pre> | ||
You can then make a window appear | |||
<pre> | <pre> | ||
GROUP=def-poq-ab | GROUP=def-poq-ab | ||
| Line 61: | Line 60: | ||
<!--T:13--> | <!--T:13--> | ||
You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and | You can navigate the folders with the mouse and still type <code>c</code> to toggle between the size of the files and the number of files. | ||
<!--T:14--> | <!--T:14--> | ||
[[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc | [[File:Duc gui navigation.gif|thumb|using|450px|frame|left|Navigating you project space with duc GUI tool]] | ||
<br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | <br clear=all> <!-- This is to prevent the next section from filling to the right of the image. --> | ||
| Line 70: | Line 69: | ||
<!--T:16--> | <!--T:16--> | ||
First [http://duc.zevv.nl/#download install the | First [http://duc.zevv.nl/#download install the diskusage_explorer software] on your local machine and then, still on your local machine, download the SQLite file from your cluster and run <code>duc</code>. | ||
<!--T:17--> | <!--T:17--> | ||
| Line 81: | Line 80: | ||
<!--T:18--> | <!--T:18--> | ||
This immediately leads to a smoother and more satisfying browsing experience. | |||
</translate> | </translate> | ||
Revision as of 19:16, 29 March 2021
Content of Folders
Warning, at the moment this feature is only available on Béluga
You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in a portable SQLite format for fast access.
Here is a walkthrough of how to explore at your disk consumption, using the example of the project space, where $GROUP will be the particular folder to investigate.
Ncurse User Interface
First list all the projects that you have access to:
ls ~/projects/
def-bourqueg def-lathrop def-poq-ab rrg-bourqueg-ad rrg-lathrop
We will take the def-poq-ab project as an example,
GROUP=def-poq-ab
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
This command loads a browser that shows the resources consumed by all files under any directory tree
Type c to toggle between consumed disk space and the number of files, q or <esc> to quit and h for help
If you are only interested in a /project/${GROUP} subfolder and do not want to navigate the whole tree in the ncurse user interface,
duc ui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}/some/subfolder/
A complete manual page is available with the man duc command.
Graphical User Interface
Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run diskusage_explorer on you own machine.
Note that Compute Canada recommends the use of the standard text-based ui mode on our cluster login nodes but diskusage_explorer does also include a nice graphical user interface (GUI).
First make sure that you are connected to the cluster using ssh's -Y option
ssh -Y poq@beluga.calculcanada.ca
You can then make a window appear
GROUP=def-poq-ab
duc gui -d /project/.duc_databases/${GROUP}.sqlite /project/${GROUP}
You can navigate the folders with the mouse and still type c to toggle between the size of the files and the number of files.
Browse faster on your own machine
First install the diskusage_explorer software on your local machine and then, still on your local machine, download the SQLite file from your cluster and run duc.
MYUSERNAME=poq
GROUP=def-poq-ab
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite .
duc gui -d ./${GROUP}.sqlite /project/${GROUP}
This immediately leads to a smoother and more satisfying browsing experience.