JupyterLab: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
No edit summary
(New redirection)
Tag: Replaced
Line 1: Line 1:
<languages />
<languages />
<translate>
<translate>
== Introduction == <!--T:1-->
#REDIRECT [[Jupyter#JupyterLab]]
JupyterLab is now the recommended general-purpose user interface to use on a [[JupyterHub]]. From a JupyterLab server, you can manage your remote files and folders, and you can launch Jupyter applications like a terminal, (Python 3) notebooks, RStudio and a Linux desktop.
 
<!--T:2-->
The subsection [[JupyterHub#JupyterHub_on_clusters|JupyterHub on clusters]] contains the list of available Jupyter hubs at Compute Canada.
 
== Launching a JupyterLab server as a job == <!--T:3-->
[[File:JupyterHub_Server_Options.png|thumb|''Server Options'' form on Béluga's JupyterHub]]
On [[JupyterHub#JupyterHub_on_clusters|Cedar, Béluga, and Hélios]], once the authentication is done on [[JupyterHub#JupyterHub_on_clusters|JupyterHub]], your Web browser is redirected to either a) a previously launched Jupyter server or b) a form that allows you to configure and submit a new interactive session on the cluster. In any case, the JupyterLab server is running on dedicated compute resources.
 
<!--T:4-->
In the ''Server Options'' form, you can:
* Set all [[JupyterHub#Compute_resources| compute resources]] options you need for the session
* Select ''JupyterLab'' as the ''[[JupyterHub#User_Interface|User Interface]]''
* Start the JupyterLab server by clicking the ''Start'' button
 
== The JupyterLab Interface == <!--T:5-->
When JupyterLab is ready to be used, the interface has multiple panels.
[[File:JupyterLab_HomeTab.png|thumb|Default home tab when JupyterLab is loaded]]
 
=== Menu bar on top === <!--T:6-->
* In the ''File'' menu:
** ''Hub Control Panel'': if you want to manually stop the JupyterLab server and the corresponding job on the cluster. This is useful when you want to start a new JupyterLab server with more or less resources
** ''Log Out'': the [[JupyterHub]] session will end, which will also stop the JupyterLab server and the corresponding job on the cluster
* Most other menu items are related to notebooks and Jupyter applications
 
=== Tool selector on left === <!--T:7-->
* ''File Browser'' (folder icon):
** This is where you can browse in your home, project and scratch spaces
** It is also possible to upload files
* ''Running Terminals and Kernels'' (stop icon):
** To stop kernel sessions and terminal sessions
* ''Commands''
* ''Property Inspector''
* ''Open Tabs'':
** To navigate between application tabs
** To close application tabs - the corresponding kernels remain active
[[File:JupyterLab_Softwares.png|thumb|Loaded modules and available modules]]
* ''Softwares'' (blue diamond sign):
** Compute Canada modules can be loaded and unloaded in the JupyterLab session. Depending on the module loaded, an icon directing to the [[#Jupyter_Applications|Jupyter application]] will appear in the ''Launcher'' tab.
** The search box can search for any [[Available software|available module]] and give the result in the ''Available Modules'' sub-panel. Note: some modules are hidden until their dependency is loaded - we recommend that you first look for a specific module with <code>module spider module_name</code> from a terminal.
** The next sub-panel is the list of ''Loaded Modules'' in the whole JupyterLab session. Note: while <code>python</code> and <code>ipython-kernel</code> modules are loaded by default, additional modules must be loaded before launching some other applications or notebooks. For example: <code>scipy-stack</code>.
** The last sub-panel is the list of ''Available modules'', similar to the output of <code>module avail</code>. By clicking on a module's name, detailed information about the module is displayed. By clicking on the ''Load'' link, the module will be loaded and added to the ''Loaded Modules'' list.
 
=== Application area on right === <!--T:8-->
* The ''Launcher'' tab is opened by default
** It contains all available [[#Jupyter_Applications|Jupyter applications and notebooks]], depending on which modules are loaded
 
=== Status bar at the bottom === <!--T:9-->
* By clicking on the icons, this brings you to the ''Running Terminals and Kernels'' tool.
 
== Jupyter Applications == <!--T:10-->
 
<!--T:11-->
JupyterLab offers access to a terminal, an IDE (Desktop), a Python console and different options to create text and Markdown files. This section presents only the main supported Jupyter applications that work with the Compute Canada software stack.
 
=== Command Line Interpreters === <!--T:33-->
 
<!--T:34-->
[[File:JupyterLab_Launcher_Julia.png|thumb|Julia console launcher button]]
[[File:JupyterLab_Launcher_Python.png|thumb|Python console launcher button]]
[[File:JupyterLab_Launcher_Terminal.png|thumb|Terminal launcher button]]
 
==== Julia Console ==== <!--T:35-->
 
<!--T:36-->
To enable the ''Julia 1.x'' console launcher, an <code>ijulia-kernel</code> module needs to be loaded. When launched, a Julia interpreter is presented in a new JupyterLab tab.
 
==== Python Console ==== <!--T:37-->
 
<!--T:38-->
The ''Python 3.x'' console launcher is available by default in a new JupyterLab session. When launched, a Python 3 interpreter is presented in a new JupyterLab tab.
 
==== Terminal ==== <!--T:12-->
 
<!--T:13-->
This application launcher will open a terminal in a new JupyterLab tab:
* The terminal runs a (Bash) shell on the remote compute node without the need of an SSH connection
** Gives access to the remote filesystems (<code>/home</code>, <code>/project</code>, <code>/scratch</code>)
** Allows running compute tasks
* The terminal allows copy-and-paste operations of text:
** Copy operation: select the text, then press Ctrl+C
*** Note: usually, Ctrl+C is used to send a SIGINT signal to a running process, or to cancel the current command. To get this behaviour in JupyterLab's terminal, click on the terminal to deselect any text before pressing Ctrl+C
** Paste operation: press Ctrl+V
 
=== Available Notebook Kernels === <!--T:39-->
 
==== Julia Notebook ==== <!--T:40-->
 
<!--T:41-->
To enable the ''Julia 1.x'' notebook launcher, an <code>ijulia-kernel</code> module needs to be loaded. When launched, a Julia notebook is presented in a new JupyterLab tab.
 
==== Python Notebook ==== <!--T:14-->
 
<!--T:15-->
[[File:JupyterLab_Softwares_ScipyStack.png|thumb|Searching for scipy-stack modules]]
If any of the following scientific Python packages is required by your notebook, before you open this notebook, you must load the <code>scipy-stack</code> module from the JupyterLab ''Softwares'' tool:
* <code>ipython</code>, <code>ipython_genutils</code>, <code>ipykernel</code>, <code>ipyparallel</code>
* <code>matplotlib</code>
* <code>numpy</code>
* <code>pandas</code>
* <code>scipy</code>
* Other notable packages: <code>Cycler</code>, <code>futures</code>, <code>jupyter_client</code>, <code>jupyter_core</code>, <code>mpmath</code>, <code>pathlib2</code>, <code>pexpect</code>, <code>pickleshare</code>, <code>ptyprocess</code>, <code>pyzmq</code>, <code>simplegeneric</code>, <code>sympy</code>, <code>tornado</code>, <code>traitlets</code>
* And many more (click on the <code>scipy-stack</code> module to see all ''Included extensions'')
 
<!--T:32-->
Note: you may also install needed packages by running for example the following command inside of a cell: <code>!pip install --no-index numpy</code>
* For some packages (like <code>plotly</code>, for example), you may need to restart the notebook's kernel before importing the package.
* The installation of packages in the default Python kernel environment is temporary to the lifetime of the JupyterLab session; you will have to reinstall these packages the next time you start a new JupyterLab session. For a persistent Python environment, you may configure a custom [[JupyterLab#Python_Kernel|Python kernel]].
 
<!--T:16-->
To open an existing Python notebook:
* Go back to the ''File Browser''
* Browse to the location of the <code>*.ipynb</code> file
* Double-click on the <code>*.ipynb</code> file:
** This will open the Python notebook in a new JupyterLab tab
** An IPython kernel will start running in background for this notebook
 
<!--T:17-->
To open a new Python notebook in the current ''File Browser'' directory:
* Click on the ''Python 3.x'' launcher under the ''Notebook'' section:
** This will open a new Python 3 notebook in a new JupyterLab tab
** A new IPython kernel will start running in background for this notebook
 
=== Other Applications === <!--T:42-->
 
==== OpenRefine ==== <!--T:18-->
 
<!--T:26-->
[[File:JupyterLab_Launcher_OpenRefine.png|thumb|OpenRefine launcher button]]
To enable the ''OpenRefine'' application launcher, an <code>openrefine</code> module needs to be loaded. Depending on the software environment version, the latest version of OpenRefine should be loaded:
* With <code>StdEnv/2020</code>, load module: <code>openrefine/3.4.1</code>
* With <code>StdEnv/2018.3</code>, load module: <code>openrefine/3.3</code>
 
<!--T:27-->
This ''OpenRefine'' launcher will open or reopen an OpenRefine interface in a new Web browser tab:
* It is possible to reopen an active OpenRefine session after the Web browser tab was closed
* The OpenRefine session will end when the JupyterLab session will end
 
==== RStudio ==== <!--T:19-->
 
<!--T:20-->
[[File:JupyterLab_Launcher_RStudio.png|thumb|RStudio launcher button]]
To enable the ''RStudio'' application launcher, the following three modules need to be loaded:
# <code>gcc</code>
# <code>r</code>
# <code>rstudio-server</code>
 
<!--T:21-->
Depending on the software environment version, you should load the following two modules (<code>r</code> is loaded automatically):
* With <code>StdEnv/2020</code>, it is not yet supported
* With <code>StdEnv/2018.3</code>, load modules: <code>gcc/7.3.0</code>, <code>rstudio-server/1.2.1335</code>
* With <code>StdEnv/2016.4</code>, load modules: <code>gcc/7.3.0</code>, <code>rstudio-server/1.2.1335</code>
 
<!--T:22-->
This ''RStudio'' launcher will open or reopen an RStudio interface in a new Web browser tab:
* It is possible to reopen an active RStudio session after the Web browser tab was closed
* The RStudio session will end when the JupyterLab session will end
 
==== VS Code ==== <!--T:23-->
 
<!--T:28-->
[[File:JupyterLab_Launcher_VSCode.png|thumb|VS Code launcher button]]
To enable the ''VS Code'' (Visual Studio Code) application launcher, a <code>code-server</code> module needs to be loaded. Depending on the software environment version, the latest version of VS Code should be loaded:
* With <code>StdEnv/2020</code>, load module: <code>code-server/3.5.0</code>
* With <code>StdEnv/2018.3</code>, load module: <code>code-server/3.4.1</code>
 
<!--T:29-->
This ''VS Code'' launcher will open or reopen the VS Code interface in a new Web browser tab:
* For a new session, the ''VS Code'' session can take up to 3 minutes to complete its startup.
* It is possible to reopen an active VS Code session after the Web browser tab was closed
* The VS Code session will end when the JupyterLab session will end
 
==== Desktop ==== <!--T:24-->
 
<!--T:25-->
[[File:JupyterLab_Launcher_Desktop.png|thumb|Desktop launcher button]]
This ''Desktop'' launcher will open or reopen a remote Linux desktop interface in a new Web browser tab:
* This is equivalent to running a [[VNC#Compute_Nodes|VNC server on a compute node]], then creating an [[SSH_tunnelling|SSH tunnel]] and finally using a [[VNC#Setup|VNC client]], but you need nothing of all this with JupyterLab!
* It is possible to reopen an active desktop session after the Web browser tab was closed
* The desktop session will end when the JupyterLab session will end
 
== Create your own Jupyter Application Kernel == <!--T:43-->
It is possible to add kernels for other programming languages, for a different Python version or for a persistent virtual environment that has all required packages and libraries for your project. Refer to [http://jupyter-client.readthedocs.io/en/latest/kernels.html Making kernels for Jupyter] to learn more.
 
<!--T:44-->
The installation of a new kernel is done in two steps:
# Installation of the packages that will allow the language interpreter to communicate with the Jupyter Notebook.
# Creation of a file that will indicate to Jupyter Notebook how to initiate a communication channel with the language interpreter. This file is called a ''kernel spec file'', and it will be saved in a sub-folder of <code>~/.local/share/jupyter/kernels</code>.
 
<!--T:45-->
In the following sections, we provide a few examples of the kernel installation procedure.
 
=== Julia Kernel === <!--T:46-->
Prerequisites:
# The configuration of a Julia kernel depends on a Python virtual environment that already has all the '''[[JupyterLab#Python_Kernel|Python Kernel]]''' dependencies. If you do not have such virtual environment, make sure to follow instructions listed in the next section.
# Since the installation of Julia packages requires an access to Internet, the configuration of a Julia kernel must be done in a '''[[SSH|remote shell session on a login node]]'''. For example, from a local terminal or any SSH client: {{Command2|prompt=[my_computer ~] $|ssh username@cluster_address}}
Once you have a Python virtual environment available, you may configure the Julia kernel:
 
<!--T:57-->
<ol>
<li>Activate the Python virtual environment. {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Load the [[Julia]] module. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|module load julia}}
<li>Install IJulia. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|echo -e 'using Pkg\nPkg.add("IJulia")' {{!}} julia}}
<li>Deactivate the virtual environment and close the remote session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>Start or restart a new JupyterLab session.
</ol>
 
<!--T:58-->
For more information, see the [https://github.com/JuliaLang/IJulia.jl IJulia documentation].
 
==== Installing more Julia packages ==== <!--T:59-->
 
<!--T:60-->
The installation of Julia packages cannot be done from Notebooks because there is no access to Internet.
As in the above installation procedure, it is required to install Julia packages from a login node:
 
<!--T:61-->
<ol>
<li>From a local terminal or any SSH client, connect to a login node of the cluster hosting JupyterHub.
<li>Load the same Julia module. {{Command2|module load julia}}
<li>Install any required package. For example <code>Glob</code>. {{Command2|echo -e 'using Pkg\nPkg.add("Glob")' {{!}} julia}}
<li>Close the remote session
<li>The newly installed Julia packages should already be usable in a Notebook executed by the Julia kernel.
</ol>
 
=== Python Kernel === <!--T:47-->
In a terminal with an active session on the remote server,
you may configure a [[Python#Creating_and_using_a_virtual_environment|Python virtual environment]] with all the required [[Available_Python_wheels|Python modules]]
and a custom Python kernel for JupyterLab.
Here are the initial steps for the simplest Jupyter configuration
in a new Python virtual environment:
 
<!--T:48-->
<ol>
<li>Start from a clean Bash environment (this is required if you are using the Jupyter ''Terminal''): {{Command2|env -i HOME{{=}}$HOME bash -l}}
<li>Load a Python module: {{Command2|module load python/3.8}}
<li>Create a new Python virtual environment: {{Command2|virtualenv --no-download $HOME/jupyter_py3.8}}
<li>Activate your newly created Python virtual environment: {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Install the <code>ipykernel</code> library: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|pip install --no-index ipykernel}}
<li>Create the common <code>kernels</code> folder: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|mkdir -p ~/.local/share/jupyter/kernels}}
<li>Generate the kernel spec file. Substitute <code><unique_name></code> by a name that will uniquely identify your kernel: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|python -m ipykernel install --user --name <unique_name> --display-name "Python 3.8 Kernel"}}
<li>Deactivate the virtual environment and exit the temporary Bash session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>Start or restart a new JupyterLab session.
</ol>
 
<!--T:49-->
For more information, see the [http://ipython.readthedocs.io/en/stable/install/kernel_install.html ipykernel documentation].
 
==== Installing more Python libraries ==== <!--T:50-->
Based on the Python virtual environment created in the previous section:
 
<!--T:51-->
<ol>
<li>Start from a clean Bash environment (this is required if you are using the Jupyter ''Terminal''): {{Command2|env -i HOME{{=}}$HOME bash -l}}
<li>Activate the virtual environment: {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Install any required library (for example: <code>numpy</code>): {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|pip install --no-index numpy}}
<li>Deactivate the virtual environment and exit the temporary Bash session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>The newly installed Python libraries can now be imported in any notebook using the <code>Python 3.8 Kernel</code>.
</ol>
 
=== R Kernel === <!--T:52-->
Prerequisites:
# Configuring an R kernel still depends on a Python virtual environment that already has all the '''[[JupyterLab#Python_Kernel|Python Kernel]]''' dependencies. If you do not have such virtual environment, make sure to follow instructions listed in the previous section.
# Since the installation of R packages requires an access to '''[https://cran.r-project.org/ CRAN]''', the configuration of an R kernel must be done in a '''[[SSH|remote shell session on a login node]]'''. For example, from a local terminal or any SSH client: {{Command2|prompt=[my_computer ~] $|ssh username@cluster_address}}
Once you have a Python virtual environment available, you may configure the R kernel:
 
<!--T:53-->
<ol>
<li>Activate the Python virtual environment. {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Load an R module. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|module load r/4.1}}
<li>Install the R kernel dependencies - this will take up to 10 minutes, and packages should be installed in a local directory like <code>~/R/x86_64-pc-linux-gnu-library/4.1</code>. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|R --no-save}} {{Command2|prompt=> |install.packages(c('crayon', 'pbdZMQ', 'devtools'), repos{{=}}'http://cran.us.r-project.org')}}
<li>Install the R kernel. {{Command2|prompt=> |devtools::install_github(paste0('IRkernel/', c('repr', 'IRdisplay', 'IRkernel')))}}
<li>Install the R kernel spec file. {{Command2|prompt=> |IRkernel::installspec()}}
<li>Quit R (with <code>q()</code>), deactivate the virtual environment and close the remote session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>Start or restart a new JupyterLab session.
</ol>
 
<!--T:54-->
For more information, see the [https://irkernel.github.io/docs/ IRKernel documentation].
 
==== Installing more R packages ==== <!--T:55-->
The installation of R packages cannot be done from Notebooks because there is no access to CRAN.
As in the above installation procedure, it is required to install R packages from a login node:
 
<!--T:56-->
<ol>
<li>From a local terminal or any SSH client, connect to a login node of the cluster hosting JupyterHub.
<li>Load the same R module. {{Command2|module load r/4.1}}
<li>Start the R shell and install any required package. For example <code>doParallel</code>. {{Command2|R --no-save}} {{Command2|prompt=> |install.packages('doParallel', repos{{=}}'http://cran.us.r-project.org')}}
<li>Quit R (with <code>q()</code>) and close the remote session
<li>The newly installed R packages should already be usable in a Notebook executed by the R kernel.
</ol>
 
== Possible error messages == <!--T:30-->
 
<!--T:31-->
* A "time-out" error message when starting a JupyterLab session:
** Just like any interactive job on any cluster, the maximum time request should be under three (3) hours in order to avoid issues with the scheduler or a longer than usual wait time.
** There may be no available interactive node at the moment. You should then try at another moment.
 
</translate>
</translate>

Revision as of 15:33, 11 January 2022

Other languages:
  1. REDIRECT Jupyter#JupyterLab
Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=JupyterLab&oldid=109714"