Visualization: Difference between revisions
No edit summary |
(Marked this version for translation) |
||
| Line 7: | Line 7: | ||
[http://www.paraview.org ParaView] is a general-purpose 3D scientific visualization tool. It is open-source and compiles on all popular platforms (Linux, Windows, Mac), understands a large number of input file formats, provides multiple rendering modes, supports Python scripting, and can scale up to tens of thousands of processors for rendering of very large datasets. | [http://www.paraview.org ParaView] is a general-purpose 3D scientific visualization tool. It is open-source and compiles on all popular platforms (Linux, Windows, Mac), understands a large number of input file formats, provides multiple rendering modes, supports Python scripting, and can scale up to tens of thousands of processors for rendering of very large datasets. | ||
<!--T:148--> | |||
* [[ParaView|Using ParaView on Compute Canada systems]] | * [[ParaView|Using ParaView on Compute Canada systems]] | ||
* [http://www.paraview.org/documentation ParaView official documentation] | * [http://www.paraview.org/documentation ParaView official documentation] | ||
| Line 16: | Line 17: | ||
Similar to ParaView, [https://wci.llnl.gov/simulation/computer-codes/visit/ VisIt] is an open-source, general-purpose 3D scientific data analysis and visualization tool that scales from interactive analysis on laptops to very large HPC projects on tens of thousands of processors. | Similar to ParaView, [https://wci.llnl.gov/simulation/computer-codes/visit/ VisIt] is an open-source, general-purpose 3D scientific data analysis and visualization tool that scales from interactive analysis on laptops to very large HPC projects on tens of thousands of processors. | ||
<!--T:149--> | |||
* [[VisIt|Using VisIt on Compute Canada systems]] | * [[VisIt|Using VisIt on Compute Canada systems]] | ||
* [https://wci.llnl.gov/simulation/computer-codes/visit/manuals VisIt manuals] | * [https://wci.llnl.gov/simulation/computer-codes/visit/manuals VisIt manuals] | ||
| Line 25: | Line 27: | ||
[http://www.ks.uiuc.edu/Research/vmd VMD] is an open-source molecular visualization program for displaying, animating, and analyzing large biomolecular systems in 3D. It supports scripting in Tcl and Python and runs on a variety of platforms (MacOS X, Linux, Windows). It reads many molecular data formats using an extensible plugin system and supports a number of different molecular representations. | [http://www.ks.uiuc.edu/Research/vmd VMD] is an open-source molecular visualization program for displaying, animating, and analyzing large biomolecular systems in 3D. It supports scripting in Tcl and Python and runs on a variety of platforms (MacOS X, Linux, Windows). It reads many molecular data formats using an extensible plugin system and supports a number of different molecular representations. | ||
<!--T:150--> | |||
* [[VMD|Using VMD on Compute Canada systems]] | * [[VMD|Using VMD on Compute Canada systems]] | ||
* [http://www.ks.uiuc.edu/Research/vmd/current/ug VMD User's Guide] | * [http://www.ks.uiuc.edu/Research/vmd/current/ug VMD User's Guide] | ||
| Line 31: | Line 34: | ||
The Visualization Toolkit (VTK) is an open-source package for 3D computer graphics, image processing, and visualization. The toolkit includes a C++ class library as well as several interfaces for interpreted languages such as Tcl/Tk, Java, and Python. VTK was the basis for many excellent visualization packages including ParaView and VisIt. | The Visualization Toolkit (VTK) is an open-source package for 3D computer graphics, image processing, and visualization. The toolkit includes a C++ class library as well as several interfaces for interpreted languages such as Tcl/Tk, Java, and Python. VTK was the basis for many excellent visualization packages including ParaView and VisIt. | ||
<!--T:151--> | |||
* [[VTK|Using VTK on Compute Canada systems]] | * [[VTK|Using VTK on Compute Canada systems]] | ||
* [https://itk.org/Wiki/VTK/Tutorials VTK tutorials] | * [https://itk.org/Wiki/VTK/Tutorials VTK tutorials] | ||
=== YT === | === YT === <!--T:152--> | ||
YT is a Python library for analyzing and visualizing volumetric, multi-resolution data. Initially developed for astrophysical simulation data, it can handle any uniform and multiple-resolution data on Cartesian, curvilinear, unstructured meshes and on particles. | YT is a Python library for analyzing and visualizing volumetric, multi-resolution data. Initially developed for astrophysical simulation data, it can handle any uniform and multiple-resolution data on Cartesian, curvilinear, unstructured meshes and on particles. | ||
<!--T:153--> | |||
* [[yt|Using YT on Compute Canada systems]] | * [[yt|Using YT on Compute Canada systems]] | ||
= Visualization on Compute Canada systems = <!--T:6--> | = Visualization on Compute Canada systems = <!--T:6--> | ||
<!--T:154--> | |||
There are many options for remote visualization on Compute Canada systems. In general, whenever possible, for interactive rendering we recommend '''client-server visualization''' on interactive nodes, and for non-interactive visualization we recommend '''off-screen batch jobs''' on regular compute nodes. | There are many options for remote visualization on Compute Canada systems. In general, whenever possible, for interactive rendering we recommend '''client-server visualization''' on interactive nodes, and for non-interactive visualization we recommend '''off-screen batch jobs''' on regular compute nodes. | ||
<!--T:155--> | |||
Other, ''less efficient'' options are X11-forwarding and VNC. For some packages these are the only available remote GUI options. | Other, ''less efficient'' options are X11-forwarding and VNC. For some packages these are the only available remote GUI options. | ||
=== Client-server interactive visualization === | === Client-server interactive visualization === <!--T:156--> | ||
<!--T:157--> | |||
In the client-server mode, supported by both ParaView and VisIt, all data will be processed remotely on the cluster, using either CPU or GPU rendering, while you interact with your visualization through a familiar GUI client on your laptop. You can find the details of setting up client-server visualization in [[ParaView]] and [[VisIt]] pages. | In the client-server mode, supported by both ParaView and VisIt, all data will be processed remotely on the cluster, using either CPU or GPU rendering, while you interact with your visualization through a familiar GUI client on your laptop. You can find the details of setting up client-server visualization in [[ParaView]] and [[VisIt]] pages. | ||
=== Remote windows with X11-forwarding === | === Remote windows with X11-forwarding === <!--T:158--> | ||
<!--T:159--> | |||
In general, X11-forwarding should be avoided for any heavy graphics, as it requires many round trips and is much slower than VNC (below). However, in some cases you can connect via ssh with X11. Below we show how you would do this on our clusters. We assume you have an X-server installed on your laptop. | In general, X11-forwarding should be avoided for any heavy graphics, as it requires many round trips and is much slower than VNC (below). However, in some cases you can connect via ssh with X11. Below we show how you would do this on our clusters. We assume you have an X-server installed on your laptop. | ||
<!--T:160--> | |||
<tabs> | <tabs> | ||
<tab name="Cedar and Graham"> | <tab name="Cedar and Graham"> | ||
<!--T:161--> | |||
Connect to Cedar or Graham with the -X/-Y flag for X11-forwarding. You can start your graphical application on the login node (small visualizations) | Connect to Cedar or Graham with the -X/-Y flag for X11-forwarding. You can start your graphical application on the login node (small visualizations) | ||
module load vmd | <!--T:162--> | ||
module load vmd | |||
vmd | vmd | ||
<!--T:163--> | |||
or you can request interactive resources on a compute node (large visualizations) | or you can request interactive resources on a compute node (large visualizations) | ||
salloc --time=1:00:0 --ntasks=1 --mem=3500 --account=def-someprof --x11 | <!--T:164--> | ||
salloc --time=1:00:0 --ntasks=1 --mem=3500 --account=def-someprof --x11 | |||
<!--T:165--> | |||
: and, once the job is running, start your graphical application inside the job | : and, once the job is running, start your graphical application inside the job | ||
module load vmd | <!--T:166--> | ||
module load vmd | |||
vmd | vmd | ||
<!--T:167--> | |||
</tab> | </tab> | ||
<tab name="Niagara"> | <tab name="Niagara"> | ||
<!--T:168--> | |||
Since runtime is limited on the login nodes, you might want to request a testing job in order to have more time for exploring and visualizing your data. On the plus side, you will have access to 40 cores on each of the nodes requested. For performing an interactive visualization session in this way please follow these steps: | Since runtime is limited on the login nodes, you might want to request a testing job in order to have more time for exploring and visualizing your data. On the plus side, you will have access to 40 cores on each of the nodes requested. For performing an interactive visualization session in this way please follow these steps: | ||
<!--T:169--> | |||
<ol> | <ol> | ||
<li> ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for X11-forwarding | <li> ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for X11-forwarding | ||
| Line 82: | Line 101: | ||
<li> run your visualization program, eg. VMD </li> | <li> run your visualization program, eg. VMD </li> | ||
module load vmd | <!--T:170--> | ||
module load vmd | |||
vmd | vmd | ||
<!--T:171--> | |||
<li> exit the debug session. | <li> exit the debug session. | ||
</ol> | </ol> | ||
<!--T:172--> | |||
</tab> | </tab> | ||
</tabs> | </tabs> | ||
| Line 109: | Line 131: | ||
=== Webinars and other short presentations === <!--T:10--> | === Webinars and other short presentations === <!--T:10--> | ||
<!--T:173--> | |||
[https://westgrid.github.io/trainingMaterials/tools/visualization/ WestGrid's visualization training materials page] has embedded video recordings and slides from the following webinars: | [https://westgrid.github.io/trainingMaterials/tools/visualization/ WestGrid's visualization training materials page] has embedded video recordings and slides from the following webinars: | ||
<!--T:174--> | |||
* YT series: “Using YT for analysis and visualization of volumetric data” (Part 1) and "Working with data objects in YT” (Part 2) | * YT series: “Using YT for analysis and visualization of volumetric data” (Part 1) and "Working with data objects in YT” (Part 2) | ||
* “Scientific visualization with Plotly” | * “Scientific visualization with Plotly” | ||
| Line 121: | Line 145: | ||
* “Graph visualization with Gephi” | * “Graph visualization with Gephi” | ||
<!--T:175--> | |||
Other visualization presentations: | Other visualization presentations: | ||
Revision as of 17:35, 26 February 2019
Popular visualization packages
ParaView
ParaView is a general-purpose 3D scientific visualization tool. It is open-source and compiles on all popular platforms (Linux, Windows, Mac), understands a large number of input file formats, provides multiple rendering modes, supports Python scripting, and can scale up to tens of thousands of processors for rendering of very large datasets.
- Using ParaView on Compute Canada systems
- ParaView official documentation
- ParaView gallery
- ParaView wiki
- ParaView Python scripting
VisIt
Similar to ParaView, VisIt is an open-source, general-purpose 3D scientific data analysis and visualization tool that scales from interactive analysis on laptops to very large HPC projects on tens of thousands of processors.
- Using VisIt on Compute Canada systems
- VisIt manuals
- VisIt gallery
- VisIt user community wiki
- VisIt tutorials along with sample datasets
VMD
VMD is an open-source molecular visualization program for displaying, animating, and analyzing large biomolecular systems in 3D. It supports scripting in Tcl and Python and runs on a variety of platforms (MacOS X, Linux, Windows). It reads many molecular data formats using an extensible plugin system and supports a number of different molecular representations.
VTK
The Visualization Toolkit (VTK) is an open-source package for 3D computer graphics, image processing, and visualization. The toolkit includes a C++ class library as well as several interfaces for interpreted languages such as Tcl/Tk, Java, and Python. VTK was the basis for many excellent visualization packages including ParaView and VisIt.
YT
YT is a Python library for analyzing and visualizing volumetric, multi-resolution data. Initially developed for astrophysical simulation data, it can handle any uniform and multiple-resolution data on Cartesian, curvilinear, unstructured meshes and on particles.
Visualization on Compute Canada systems
There are many options for remote visualization on Compute Canada systems. In general, whenever possible, for interactive rendering we recommend client-server visualization on interactive nodes, and for non-interactive visualization we recommend off-screen batch jobs on regular compute nodes.
Other, less efficient options are X11-forwarding and VNC. For some packages these are the only available remote GUI options.
Client-server interactive visualization
In the client-server mode, supported by both ParaView and VisIt, all data will be processed remotely on the cluster, using either CPU or GPU rendering, while you interact with your visualization through a familiar GUI client on your laptop. You can find the details of setting up client-server visualization in ParaView and VisIt pages.
Remote windows with X11-forwarding
In general, X11-forwarding should be avoided for any heavy graphics, as it requires many round trips and is much slower than VNC (below). However, in some cases you can connect via ssh with X11. Below we show how you would do this on our clusters. We assume you have an X-server installed on your laptop.
Connect to Cedar or Graham with the -X/-Y flag for X11-forwarding. You can start your graphical application on the login node (small visualizations)
module load vmd vmd
or you can request interactive resources on a compute node (large visualizations)
salloc --time=1:00:0 --ntasks=1 --mem=3500 --account=def-someprof --x11
- and, once the job is running, start your graphical application inside the job
module load vmd vmd
Since runtime is limited on the login nodes, you might want to request a testing job in order to have more time for exploring and visualizing your data. On the plus side, you will have access to 40 cores on each of the nodes requested. For performing an interactive visualization session in this way please follow these steps:
- ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for X11-forwarding
- request an interactive job, ie. debugjob this will connect you to a node, let's say for the argument "niaXYZW"
- run your visualization program, eg. VMD module load vmd vmd
- exit the debug session.
Start a remote desktop via VNC
Frequently, it may be useful to start up graphical user interfaces for various software packages like Matlab. Doing so over X11-forwarding can result in a very slow connection to the server. Instead, we recommend using VNC to start and connect to a remote desktop. For more information, please see the article on VNC.
Visualization training
Please let us know if you would like to see a visualization workshop at your institution.
Full- or half-day workshops
- VisIt workshop slides from HPCS'2016 in Edmonton by Marcelo Ponce and Alex Razoumov
- ParaView workshop slides from July 2017 by Alex Razoumov
- Gnuplot, xmgrace, remote visualization tools (X-forwarding and VNC), python's matplotlib slides by Marcelo Ponce (SciNet/UofT) from Ontario HPC Summer School 2016
- Brief overview of ParaView & VisIt slides by Marcelo Ponce (SciNet/UofT) from Ontario HPC Summer School 2016
Webinars and other short presentations
WestGrid's visualization training materials page has embedded video recordings and slides from the following webinars:
- YT series: “Using YT for analysis and visualization of volumetric data” (Part 1) and "Working with data objects in YT” (Part 2)
- “Scientific visualization with Plotly”
- “Novel Visualization Techniques from the 2017 Visualize This Challenge”
- “Data Visualization on Compute Canada’s Supercomputers” contains recipes and demos of running client-server ParaView and batch ParaView scripts on both CPU and GPU partitions of Cedar and Graham
- “Using ParaViewWeb for 3D Visualization and Data Analysis in a Web Browser”
- “Scripting and other advanced topics in VisIt visualization”
- “CPU-based rendering with OSPRay”
- “3D graphs with NetworkX, VTK, and ParaView”
- “Graph visualization with Gephi”
Other visualization presentations:
- Remote Graphics on SciNet's GPC system (Client-Server and VNC) slides by Ramses van Zon (SciNet/UofT) from October 2015 SciNet User Group Meeting
- VisIt Basics, slides by Marcelo Ponce (SciNet/UofT) from February 2016 SciNet User Group Meeting
- Intro to Complex Networks Visualization, with Python, slides by Marcelo Ponce (SciNet/UofT)
- Introduction to GUI Programming with Tkinter, from Sept.2014 by Erik Spence (SciNet/UofT)
Tips and tricks
This section will describe visualization workflows not included into the workshop/webinar slides above. It is meant to be user-editable, so please feel free to add your cool visualization scripts and workflows here so that everyone can benefit from them.
Regional visualization pages
WestGrid
SciNet, HPC at the University of Toronto
SHARCNET
- Overview
- Running pre-/post-processing graphical applications
- Supported software (see visualization section at bottom)
Visualization gallery
You can find a gallery of visualizations based on models run on Compute Canada systems in the visualization gallery. There you can click on individual thumbnails to get more details on each visualization.
How to get visualization help
Please contact Technical support.