Diskusage Explorer

Content of Folders

Warning, at the moment this feature is only available on Béluga

You can get a breakdown by folder of how the diskspace is being consumed in your home, scratch and project spaces. That information is currently updated once a day and is stored in a portable SQLite format for fast access.

Here is a walkthrough of how to explore at your disk consumption, using the example of the project space, where $GROUP will be the particular folder to investigate.

Ncurse User Interface

First list all the projects that you have access to:

 ls ~/projects/
    def-bourqueg  def-lathrop  def-poq-ab  rrg-bourqueg-ad  rrg-lathrop

We will take the def-poq-ab project as an example,

GROUP=def-poq-ab
duc ui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}

This load a browser that states the resources consumed by all files under any directory tree

Navigating you project space with duc ncurse tool

Type c to toggle between consumed disk space and the number of files. Type q <esc> to quit, type h for help

If you are only interested in a /project/${GROUP} subfolder and do not want to navigate the whole tree in the ncurse user interface,

duc ui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}/some/subfolder/

A complete manual page is available with the man duc command.

Graphical User Interface

Note that when the login node is especially busy or if you have an especially large amount of files in you project space, the graphical interface mode can be slow and choppy. For a better experience, you can read the section below to run diskusage_explorer on you own machine.

So now you are warned, we recommend the use the standard ui mode on our clusters login nodes, but duc also includes a nice graphical user interface (GUI).

First make sure that you are connected to the cluster using ssh's -Y option

ssh -Y  poq@beluga.calculcanada.ca

Then make the GUI appear

GROUP=def-poq-ab
duc gui -d /project/.duc_databases/${GROUP}.sqlite  /project/${GROUP}

You can navigate the folders with the mouse and still type c to toggle between the size of the files and their numbers

Navigating you project space with duc gui tool

Browse faster on your own machine

First install the duc software on your local machine. Then always on your local machine, download the sqlite file from your cluster and run duc.

MYUSERNAME=poq
GROUP=def-poq-ab
rsync -v --progress ${MYUSERNAME}@beluga.calculcanada.ca:/project/.duc_databases/${GROUP}.sqlite  .
duc gui -d ./${GROUP}.sqlite  /project/${GROUP}

Voilà! A smooth and satisfying browsing experience.