Gurobi

From Alliance Doc
Revision as of 19:43, 26 April 2021 by Diane27 (talk | contribs) (Created page with "== Environnements virtuels Python ==")
Jump to navigation Jump to search
Other languages:

Gurobi est une suite logicielle du commerce qui permet de résoudre des problèmes complexes d'optimisation. Nous abordons ici son utilisation pour la recherche sur les grappes de Calcul Canada. Des renseignements additionnels seront fournis dès que disponibles.

Limites de la licence

Calcul Canada dispense le soutien technique pour la licence gratuite disponible sur Graham, Cedar, Béluga et Niagara. Cette licence permet 4096 utilisations simultanées (avec jetons) et l'optimisation distribuée sur un maximum de 100 nœuds. Un utilisateur peut faire exécuter plusieurs tâches en simultané. Vous devez cependant accepter certaines conditions. Faites parvenir un courriel au soutien technique avec l'entente (Academic Usage Agreement) dûment complétée; vous pourrez ensuite utiliser les applications après un délai de quelques jours.

Academic Usage Agreement

Allocations interactives

où 

   * https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
  ** https://www.gurobi.com/documentation/8.1/refman/parameter_descriptions.html
 *** https://www.gurobi.com/documentation/8.1/refman/optimization_status_codes.html
**** https://www.gurobi.com/documentation/8.1/refman/attributes.html

$ module load gurobi
$ gurobi_cl 1> /dev/null && echo Success || echo Fail

Allocations interactives

Ligne de commande

[gra-login2:~] salloc --time=1:00:0 --cpus-per-task=8 --mem=1G --account=def-xyz
[gra800:~] module load gurobi
[gra800:~] gurobi_cl Record=1 Threads=8 Method=2 ResultFile=p0033.sol LogFile=p0033.log $GUROBI_HOME/examples/data/p0033.mps
[gra800:~] gurobi_cl --help
  1. Create environment file in current directory setting the number of threads:

echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

où 

   * https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
  ** https://www.gurobi.com/documentation/8.1/refman/parameter_descriptions.html
 *** https://www.gurobi.com/documentation/8.1/refman/optimization_status_codes.html
**** https://www.gurobi.com/documentation/8.1/refman/attributes.html

Répéter des appels API

Il est possible d'enregistrer des appels API et de rejouer l'enregistrement avec la commande

[gra800:~] gurobi_cl recording000.grbr 

[name@server ~] $ module load gurobi/9.0.1 python/3.7
[name@server ~] $ virtualenv --no-download  ~/env_gurobi
  Using base prefix '/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.7.4'
  New python executable in /home/name/env_gurobi/bin/python
  Installing setuptools, pip, wheel...
  done.
[name@server ~] $ source ~/env_gurobi/bin/activate

Répéter des appels API

Il est possible d'enregistrer des appels API et de rejouer l'enregistrement avec la commande

Nous pouvons maintenant activer Gurobi et l'environnement avec

Remarquez que nous utilisons maintenant python plutôt que gurobi.sh.

Comment citer Gurobi

  1. use a version >= 9.0.3

module load StdEnv/2020 module load gurobi/9.1.0

gurobi_cl ${GUROBI_HOME}/examples/data/coins.lp }}

Job using Gurobi Python

This is an example jobscript for a model using Gurobi-Python.

File : gurobi-py_example.sh

#!/bin/bash
#SBATCH --account=def-group   # some account
#SBATCH --time=0-00:30        # time limit (D-HH:MM)
#SBATCH --cpus-per-task=1     # number of CPUs (threads) to use
#SBATCH --mem-per-cpu=1000M   # memory per CPU (in MB)

# use a version <= 9.0.2 
module load StdEnv/2016.4     # or StdEnv/2018.3
module load gurobi/9.0.2

# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

gurobi.sh  ${GUROBI_HOME}/examples/python/facility.py


Environnements virtuels Python

Gurobi brings it's own version of Python but that one does not contain any 3rd-party Python packages except Gurobi. In order to use Gurobi together with popular Python packages like NumPy, Matplotlib, Pandas and others, we need to create a virtual Python environment in which we can install both gurobipy and e.g. pandas.

Before we start, we need to decide which combination of versions for Gurobi and Python to use. 

[name@server ~] $ module load gurobi/8.1.1
[name@server ~] $ cd $EBROOTGUROBI/lib
[name@server ~] $ ls -dF python*
  python2.7/        python2.7_utf32/  python3.6_utf32/
  python2.7_utf16/  python3.5_utf32/  python3.7_utf32/
[name@server ~] $ module load gurobi/9.0.1
[name@server ~] $ cd $EBROOTGUROBI/lib
[name@server ~] $ ls -dF python*
  python2.7_utf16/  python3.5_utf32/  python3.7/        python3.8_utf32/
  python2.7_utf32/  python3.6_utf32/  python3.7_utf32/
[name@server ~] $ cd


We see that gurobi/8.1.1 brings it's own installation of python2.7/ and Python packages for Python 2.7, 3.5, 3.6 and 3.7 (pythonX.Y_utf32/), while gurobi/9.0.1 by default uses python3.7/ and brings Python packages for Python 2.7, 3.5, 3.6, 3.7 and 3.8 (pythonX.Y_utf32/).

In this example we want to create a Python environment based on python/3.7 in which we want to use gurobi/9.0.1 and install the Pandas package.

Creating a Python virtual environments with Gurobi

These steps need to be done only once per system.

The first step is to load the modules to create the virtual environment and activate it. 

[name@server ~] $ module load gurobi/9.0.1 python/3.7
[name@server ~] $ virtualenv --no-download  ~/env_gurobi
  Using base prefix '/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.7.4'
  New python executable in /home/name/env_gurobi/bin/python
  Installing setuptools, pip, wheel...
  done.
[name@server ~] $ source ~/env_gurobi/bin/activate


Now that the environment has been activated we can install the Python packages we want to use, in this case pandas.


The third step is to install gurobipy into the environment: 

(env_gurobi) [name@server ~] $ cd $EBROOTGUROBI
(env_gurobi) [name@server ~] $ python setup.py build --build-base /tmp/${USER} install
  running build
  running build_py
  creating /tmp/name
  creating /tmp/name/lib
  creating /tmp/name/lib/gurobipy
  copying lib/python3.7_utf32/gurobipy/__init__.py -> /tmp/name/lib/gurobipy
  copying lib/python3.7_utf32/gurobipy/gurobipy.so -> /tmp/name/lib/gurobipy
  running install
  running install_lib
  creating /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  copying /tmp/name/lib/gurobipy/gurobipy.so -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  copying /tmp/name/lib/gurobipy/__init__.py -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  byte-compiling /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy/__init__.py to __init__.cpython-37.pyc
  running install_egg_info
  Writing /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy-9.0.1-py3.7.egg-info
(env_gurobi) [name@server ~] $ cd


Using the Gurobi-enabled virtual environment

Python scripts can now import both Pandas and Gurobi: 

import pandas as pd
import numpy as np
import gurobipy as gurobi
from gurobipy import *
# [...]

Once created we can activate Gurobi and the environment with: 

module load gurobi/9.0.1
source ~/env_gurobi/bin/activate
python  my_gurobi_script.py

Note that we now use python instead of gurobi.sh!

And this is an example job script that we can use:

File : gurobi-py_example.sh

#!/bin/bash
#SBATCH --time=0-00:30        # time limit (D-HH:MM)
#SBATCH --cpus-per-task=1     # number of CPUs (threads) to use
#SBATCH --mem-per-cpu=1000M   # memory per CPU (in MB)

module load StdEnv/2016.4
module load gurobi/9.0.1

source ~/env_gurobi/bin/activate

# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

python  my_gurobi_script.py


Cite Gurobi

Please see How do I cite Gurobi software for an academic publication?

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Gurobi/fr&oldid=99847"