JupyterLab: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Installing more R packages - contents)
(No more translation for this page.)
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<languages />
#REDIRECT [[Jupyter#JupyterLab]]
<translate>
== Introduction == <!--T:1-->
JupyterLab is now the recommended general-purpose user interface to use on a [[JupyterHub]]. From a JupyterLab server, you can manage your remote files and folders, and you can launch Jupyter applications like a terminal, (Python 3) notebooks, RStudio and a Linux desktop.
 
<!--T:2-->
The subsection [[JupyterHub#JupyterHub_on_clusters|JupyterHub on clusters]] contains the list of available Jupyter hubs at Compute Canada.
 
== Launching a JupyterLab server as a job == <!--T:3-->
[[File:JupyterHub_Server_Options.png|thumb|''Server Options'' form on Béluga's JupyterHub]]
On [[JupyterHub#JupyterHub_on_clusters|Béluga and Hélios]], once the authentication is done on [[JupyterHub#JupyterHub_on_clusters|JupyterHub]], your Web browser is redirected to either a) a previously launched Jupyter server or b) a form that allows you to configure and submit a new interactive session on the cluster. In any case, the JupyterLab server is running on dedicated compute resources.
 
<!--T:4-->
In the ''Server Options'' form, you can:
* Set all [[JupyterHub#Compute_resources| compute resources]] options you need for the session
* Select ''JupyterLab'' as the ''[[JupyterHub#User_Interface|User Interface]]''
* Start the JupyterLab server by clicking the ''Start'' button
 
== The JupyterLab Interface == <!--T:5-->
When JupyterLab is ready to be used, the interface has multiple panels.
[[File:JupyterLab_HomeTab.png|thumb|Default home tab when JupyterLab is loaded]]
 
=== Menu bar on top === <!--T:6-->
* In the ''File'' menu:
** ''Hub Control Panel'': if you want to manually stop the JupyterLab server and the corresponding job on the cluster. This is useful when you want to start a new JupyterLab server with more or less resources
** ''Log Out'': the [[JupyterHub]] session will end, which will also stop the JupyterLab server and the corresponding job on the cluster
* Most other menu items are related to notebooks and Jupyter applications
 
=== Tool selector on left === <!--T:7-->
* ''File Browser'' (folder icon):
** This is where you can browse in your home, project and scratch spaces
** It is also possible to upload files
* ''Running Terminals and Kernels'' (stop icon):
** To stop kernel sessions and terminal sessions
* ''Commands''
* ''Property Inspector''
* ''Open Tabs'':
** To navigate between application tabs
** To close application tabs - the corresponding kernels remain active
[[File:JupyterLab_Softwares.png|thumb|Loaded modules and available modules]]
* ''Softwares'' (blue diamond sign):
** Compute Canada modules can be loaded and unloaded in the JupyterLab session. Depending on the module loaded, an icon directing to the [[#Jupyter_Applications|Jupyter application]] will appear in the ''Launcher'' tab.
** The search box can search for any [[Available software|available module]] and give the result in the ''Available Modules'' sub-panel. Note: some modules are hidden until their dependency is loaded - we recommend that you first look for a specific module with <code>module spider module_name</code> from a terminal.
** The next sub-panel is the list of ''Loaded Modules'' in the whole JupyterLab session. Note: while <code>python</code> and <code>ipython-kernel</code> modules are loaded by default, additional modules must be loaded before launching some other applications or notebooks. For example: <code>scipy-stack</code>.
** The last sub-panel is the list of ''Available modules'', similar to the output of <code>module avail</code>. By clicking on a module's name, detailed information about the module is displayed. By clicking on the ''Load'' link, the module will be loaded and added to the ''Loaded Modules'' list.
 
=== Application area on right === <!--T:8-->
* The ''Launcher'' tab is opened by default
** It contains all available [[#Jupyter_Applications|Jupyter applications and notebooks]], depending on which modules are loaded
 
=== Status bar at the bottom === <!--T:9-->
* By clicking on the icons, this brings you to the ''Running Terminals and Kernels'' tool.
 
== Jupyter Applications == <!--T:10-->
 
<!--T:11-->
JupyterLab offers access to a terminal, an IDE (Desktop), a Python console and different options to create text and Markdown files. This section presents only the main supported Jupyter applications that work with the Compute Canada software stack.
 
=== Command Line Interpreters === <!--T:33-->
 
<!--T:34-->
[[File:JupyterLab_Launcher_Julia.png|thumb|Julia console launcher button]]
[[File:JupyterLab_Launcher_Python.png|thumb|Python console launcher button]]
[[File:JupyterLab_Launcher_Terminal.png|thumb|Terminal launcher button]]
 
==== Julia Console ==== <!--T:35-->
 
<!--T:36-->
To enable the ''Julia 1.x'' console launcher, an <code>ijulia-kernel</code> module needs to be loaded. When launched, a Julia interpreter is presented in a new JupyterLab tab.
 
==== Python Console ==== <!--T:37-->
 
<!--T:38-->
The ''Python 3.x'' console launcher is available by default in a new JupyterLab session. When launched, a Python 3 interpreter is presented in a new JupyterLab tab.
 
==== Terminal ==== <!--T:12-->
 
<!--T:13-->
This application launcher will open a terminal in a new JupyterLab tab:
* The terminal runs a (Bash) shell on the remote compute node without the need of an SSH connection
** Gives access to the remote filesystems (<code>/home</code>, <code>/project</code>, <code>/scratch</code>)
** Allows running compute tasks
* The terminal allows copy-and-paste operations of text:
** Copy operation: select the text, then press Ctrl+C
*** Note: usually, Ctrl+C is used to send a SIGINT signal to a running process, or to cancel the current command. To get this behaviour in JupyterLab's terminal, click on the terminal to deselect any text before pressing Ctrl+C
** Paste operation: press Ctrl+V
 
=== Available Notebook Kernels === <!--T:39-->
 
==== Julia Notebook ==== <!--T:40-->
 
<!--T:41-->
To enable the ''Julia 1.x'' notebook launcher, an <code>ijulia-kernel</code> module needs to be loaded. When launched, a Julia notebook is presented in a new JupyterLab tab.
 
==== Python Notebook ==== <!--T:14-->
 
<!--T:15-->
[[File:JupyterLab_Softwares_ScipyStack.png|thumb|Searching for scipy-stack modules]]
If any of the following scientific Python packages is required by your notebook, before you open this notebook, you must load the <code>scipy-stack</code> module from the JupyterLab ''Softwares'' tool:
* <code>ipython</code>, <code>ipython_genutils</code>, <code>ipykernel</code>, <code>ipyparallel</code>
* <code>matplotlib</code>
* <code>numpy</code>
* <code>pandas</code>
* <code>scipy</code>
* Other notable packages: <code>Cycler</code>, <code>futures</code>, <code>jupyter_client</code>, <code>jupyter_core</code>, <code>mpmath</code>, <code>pathlib2</code>, <code>pexpect</code>, <code>pickleshare</code>, <code>ptyprocess</code>, <code>pyzmq</code>, <code>simplegeneric</code>, <code>sympy</code>, <code>tornado</code>, <code>traitlets</code>
* And many more (click on the <code>scipy-stack</code> module to see all ''Included extensions'')
 
<!--T:32-->
Note: you may also install needed packages by running for example the following command inside of a cell: <code>!pip install --no-index numpy</code>
* For some packages (like <code>plotly</code>, for example), you may need to restart the notebook's kernel before importing the package.
* The installation of packages in the default Python kernel environment is temporary to the lifetime of the JupyterLab session; you will have to reinstall these packages the next time you start a new JupyterLab session. For a persistent Python environment, you may configure a custom [[JupyterLab#Python_Kernel|Python kernel]].
 
<!--T:16-->
To open an existing Python notebook:
* Go back to the ''File Browser''
* Browse to the location of the <code>*.ipynb</code> file
* Double-click on the <code>*.ipynb</code> file:
** This will open the Python notebook in a new JupyterLab tab
** An IPython kernel will start running in background for this notebook
 
<!--T:17-->
To open a new Python notebook in the current ''File Browser'' directory:
* Click on the ''Python 3.x'' launcher under the ''Notebook'' section:
** This will open a new Python 3 notebook in a new JupyterLab tab
** A new IPython kernel will start running in background for this notebook
 
=== Other Applications === <!--T:42-->
 
==== OpenRefine ==== <!--T:18-->
 
<!--T:26-->
[[File:JupyterLab_Launcher_OpenRefine.png|thumb|OpenRefine launcher button]]
To enable the ''OpenRefine'' application launcher, an <code>openrefine</code> module needs to be loaded. Depending on the software environment version, the latest version of OpenRefine should be loaded:
* With <code>StdEnv/2020</code>, load module: <code>openrefine/3.4.1</code>
* With <code>StdEnv/2018.3</code>, load module: <code>openrefine/3.3</code>
 
<!--T:27-->
This ''OpenRefine'' launcher will open or reopen an OpenRefine interface in a new Web browser tab:
* It is possible to reopen an active OpenRefine session after the Web browser tab was closed
* The OpenRefine session will end when the JupyterLab session will end
 
==== RStudio ==== <!--T:19-->
 
<!--T:20-->
[[File:JupyterLab_Launcher_RStudio.png|thumb|RStudio launcher button]]
To enable the ''RStudio'' application launcher, the following three modules need to be loaded:
# <code>gcc</code>
# <code>r</code>
# <code>rstudio-server</code>
 
<!--T:21-->
Depending on the software environment version, you should load the following two modules (<code>r</code> is loaded automatically):
* With <code>StdEnv/2020</code>, it is not yet supported
* With <code>StdEnv/2018.3</code>, load modules: <code>gcc/7.3.0</code>, <code>rstudio-server/1.2.1335</code>
* With <code>StdEnv/2016.4</code>, load modules: <code>gcc/7.3.0</code>, <code>rstudio-server/1.2.1335</code>
 
<!--T:22-->
This ''RStudio'' launcher will open or reopen an RStudio interface in a new Web browser tab:
* It is possible to reopen an active RStudio session after the Web browser tab was closed
* The RStudio session will end when the JupyterLab session will end
 
==== VS Code ==== <!--T:23-->
 
<!--T:28-->
[[File:JupyterLab_Launcher_VSCode.png|thumb|VS Code launcher button]]
To enable the ''VS Code'' (Visual Studio Code) application launcher, a <code>code-server</code> module needs to be loaded. Depending on the software environment version, the latest version of VS Code should be loaded:
* With <code>StdEnv/2020</code>, load module: <code>code-server/3.5.0</code>
* With <code>StdEnv/2018.3</code>, load module: <code>code-server/3.4.1</code>
 
<!--T:29-->
This ''VS Code'' launcher will open or reopen the VS Code interface in a new Web browser tab:
* For a new session, the ''VS Code'' session can take up to 3 minutes to complete its startup.
* It is possible to reopen an active VS Code session after the Web browser tab was closed
* The VS Code session will end when the JupyterLab session will end
 
==== Desktop ==== <!--T:24-->
 
<!--T:25-->
[[File:JupyterLab_Launcher_Desktop.png|thumb|Desktop launcher button]]
This ''Desktop'' launcher will open or reopen a remote Linux desktop interface in a new Web browser tab:
* This is equivalent to running a [[VNC#Compute_Nodes|VNC server on a compute node]], then creating an [[SSH_tunnelling|SSH tunnel]] and finally using a [[VNC#Setup|VNC client]], but you need nothing of all this with JupyterLab!
* It is possible to reopen an active desktop session after the Web browser tab was closed
* The desktop session will end when the JupyterLab session will end
 
== Create your own Jupyter Application Kernel ==
It is possible to add kernels for other programming languages, for a different Python version or for a persistent virtual environment that has all required packages and libraries for your project. Refer to [http://jupyter-client.readthedocs.io/en/latest/kernels.html Making kernels for Jupyter] to learn more.
 
The installation of a new kernel is done in two steps:
# Installation of the packages that will allow the language interpreter to communicate with the Jupyter Notebook.
# Creation of a file that will indicate to Jupyter Notebook how to initiate a communication channel with the language interpreter. This file is called a ''kernel spec file'', and it will be saved in a sub-folder of <code>~/.local/share/jupyter/kernels</code>.
 
In the following sections, we provide a few examples of the kernel installation procedure.
 
=== Julia Kernel ===
''(Section under construction. Please see the older [[JupyterNotebook#Julia|configuration procedure here]])''
 
=== Python Kernel ===
In a terminal with an active session on the remote server,
you may configure a [[Python#Creating_and_using_a_virtual_environment|Python virtual environment]] with all the required [[Available_Python_wheels|Python modules]]
and a custom Python kernel for JupyterLab.
Here are the initial steps for the simplest Jupyter configuration
in a new Python virtual environment:
 
<ol>
<li>Start from a clean Bash environment (this is required if you are using the Jupyter ''Terminal''): {{Command2|env -i HOME{{=}}$HOME bash -l}}
<li>Load a Python module: {{Command2|module load python/3.8}}
<li>Create a new Python virtual environment: {{Command2|virtualenv --no-download $HOME/jupyter_py3.8}}
<li>Activate your newly created Python virtual environment: {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Install the <code>ipykernel</code> library: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|pip install --no-index ipykernel}}
<li>Create the common <code>kernels</code> folder: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|mkdir -p ~/.local/share/jupyter/kernels}}
<li>Generate the kernel spec file. Substitute <code><unique_name></code> by a name that will uniquely identify your kernel: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|python -m ipykernel install --user --name <unique_name> --display-name "Python 3.8 Kernel"}}
<li>Deactivate the virtual environment and exit the temporary Bash session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>Start or restart a new JupyterLab session.
</ol>
 
For more information, see the [http://ipython.readthedocs.io/en/stable/install/kernel_install.html ipykernel documentation].
 
==== Installing more Python libraries ====
Based on the Python virtual environment created in the previous section:
 
<ol>
<li>Start from a clean Bash environment (this is required if you are using the Jupyter ''Terminal''): {{Command2|env -i HOME{{=}}$HOME bash -l}}
<li>Activate the virtual environment: {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Install any required library (for example: <code>numpy</code>): {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|pip install --no-index numpy}}
<li>Deactivate the virtual environment and exit the temporary Bash session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>The newly installed Python libraries can now be imported in any notebook using the <code>Python 3.8 Kernel</code>.
</ol>
 
=== R Kernel ===
Prerequisites:
# Configuring an R kernel still depends on a Python virtual environment that already has all the '''[[JupyterLab#Python_Kernel|Python Kernel]]''' dependencies. If you do not have such virtual environment, make sure to follow instructions listed in the previous section.
# Since the installation of R packages requires an access to '''[https://cran.r-project.org/ CRAN]''', the configuration of an R kernel must be done in a '''[[SSH|remote shell session on a login node]]'''. For example, from a local terminal or any SSH client: {{Command2|prompt=[my_computer ~] $|ssh username@cluster_address}}
Once you have a Python virtual environment available, you may configure the R kernel:
 
<ol>
<li>Activate the Python virtual environment. {{Command2|source $HOME/jupyter_py3.8/bin/activate}}
<li>Load an R module. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|module load r/4.1}}
<li>Install the R kernel dependencies - this will take up to 10 minutes, and packages should be installed in a local directory like <code>~/R/x86_64-pc-linux-gnu-library/4.1</code>. {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|R --no-save}} {{Command2|prompt=> |install.packages(c('crayon', 'pbdZMQ', 'devtools'), repos{{=}}'http://cran.us.r-project.org')}}
<li>Install the R kernel. {{Command2|prompt=> |devtools::install_github(paste0('IRkernel/', c('repr', 'IRdisplay', 'IRkernel')))}}
<li>Install the R kernel spec file. {{Command2|prompt=> |IRkernel::installspec()}}
<li>Quit R (with <code>q()</code>), deactivate the virtual environment and close the remote session: {{Command2|prompt=(jupyter_py3.8) [name@server ~]$|deactivate && exit}}
<li>Start or restart a new JupyterLab session.
</ol>
 
For more information, see the [https://irkernel.github.io/docs/ IRKernel documentation].
 
==== Installing more R packages ====
The installation of R packages cannot be done from Notebooks because there is no access to CRAN.
As in the above installation procedure, it is required to install R packages from a login node:
 
<ol>
<li>From a local terminal or any SSH client, connect to a login node of the cluster hosting JupyterHub: {{Command2|prompt=[my_computer ~] $|ssh username@cluster_address}}
<li>Load the same R module. {{Command2|module load r/4.1}}
<li>Start the R shell. {{Command2|R --no-save}}
<li>Install any required package. For example <code>doParallel</code> {{Command2|prompt=> |install.packages('doParallel', repos{{=}}'http://cran.us.r-project.org')}}
<li>Quit R (with <code>q()</code>) and close the remote session
<li>The newly installed R packages should already be usable in a Notebook executed by the R kernel.
</ol>
 
== Possible error messages == <!--T:30-->
 
<!--T:31-->
* A "time-out" error message when starting a JupyterLab session:
** Just like any interactive job on any cluster, the maximum time request should be under three (3) hours in order to avoid issues with the scheduler or a longer than usual wait time.
** There may be no available interactive node at the moment. You should then try at another moment.
 
</translate>

Latest revision as of 16:21, 11 January 2022

Redirect to:

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=JupyterLab&oldid=109719"