Diskusage Explorer

Content of Folders

Warning, at the moment this feature is only available on Béluga

You can get a break down by folder on how the data is consumed in your project space. Here is a walkthrough of how to look at the data where GROUP is the project space folder to investigate.

Ncurse User Interface

First list all the projects that you have access to:

 ls ~/projects/
    def-bourqueg  def-lathrop  def-poq-ab  rrg-bourqueg-ad  rrg-lathrop

Here i will look at the def-poq-ab project,

GROUP=def-poq-ab
duc ui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}

This load a browser that state the volume under all directory three

Navigating you project space with duc ncurse tool

Type q <esc> to quit, type h for help

If I am only interested in a /project/${GROUP} subfolder and do not want to navigate the whole tree in the ncurse ui,

duc ui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}/some/subfolder/

Type c to toggle between consumed disk space and the number of files.

Graphical User Interface

Note that on buzzy login node days, or if you have an especially large amount of files in you project space, the GUI mode can be slow and choppy. For a better experience, read the section below and run duc on you own machine.

So now you are warned, we recommend to use the standard ui mode on our clusters login nodes, duc also includes a nice graphical user interface (GUI).

First make sure you are connected to the cluster using ssh's -Y option

ssh -Y  poq@beluga.calculcanada.ca

Then make GUI appear

GROUP=def-poq-ab
duc gui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}

You can navigate the folders with the mouse and still type c to toggle between the size of the files and their numbers

Navigating you project space with duc gui tool

Browse faster on your own machine

First install the duc software in your local machine. Then always on your local machine, download the sqlite file from your cluster and run duc.

MYUSERNAME=poq
GROUP=def-poq-ab
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite  .
duc gui -d ./${GROUP}.sqlite  /project/${GROUP}