cc_staff
653
edits
Gurobi: Difference between revisions
→Using Gurobi in Python virtual environments
(create new section) |
|||
| Line 96: | Line 96: | ||
== Using Gurobi in Python virtual environments == | == Using Gurobi in Python virtual environments == | ||
Gurobi brings it's own version of Python but that one does not contain any 3rd-party Python packages except Gurobi. In order to use Gurobi together with popular Python | |||
packages like NumPy, Matplotlib, Pandas and others, we need to create a [[Python#Creating_and_using_a_virtual_environment|virtual Python environment]]in which we can install both <code>gurobipy</code> and e.g. <code>pandas</code>. | |||
Before we start, we need to decide which combination of versions for Gurobi and Python to use. | |||
{{Commands|prompt=[name@server ~] $ | |||
|module load gurobi/8.1.1 | |||
|cd $EBROOTGUROBI/lib | |||
|ls -dF python* | |||
python2.7/ python2.7_utf32/ python3.6_utf32/ | |||
python2.7_utf16/ python3.5_utf32/ python3.7_utf32/ | |||
|module load gurobi/9.0.1 | |||
|cd $EBROOTGUROBI/lib | |||
|ls -dF python* | |||
python2.7_utf16/ python3.5_utf32/ python3.7/ python3.8_utf32/ | |||
python2.7_utf32/ python3.6_utf32/ python3.7_utf32/ | |||
|cd | |||
}} | |||
We see that <code>gurobi/8.1.1</code> brings it's own installation of <code>python2.7/</code> and Python packages for Python 2.7, 3.5, 3.6 and 3.7 (<code>pythonX.Y_utf32/</code>), | |||
while <code>gurobi/9.0.1</code> by default uses <code>python3.7/</code> and brings Python packages for Python 2.7, 3.5, 3.6, 3.7 and 3.8 (<code>pythonX.Y_utf32/</code>). | |||
In this example we want to create a Python environment based on <code>python/3.7</code> in which we want to use <code>gurobi/9.0.1</code> and install the Pandas package. | |||
=== Creating a Python virtual environments with Gurobi === | |||
These steps need to be done only once per system. | |||
The first step is to load the modules, [[Python#Creating_and_using_a_virtual_environment|create the virtual environment]] and activate it. | |||
{{Commands|prompt=[name@server ~] $ | |||
| module load gurobi/9.0.1 python/3.7 | |||
| virtualenv --no-download ~/env_gurobi | |||
Using base prefix '/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.7.4' | |||
New python executable in /home/name/env_gurobi/bin/python | |||
Installing setuptools, pip, wheel... | |||
done. | |||
| source ~/env_gurobi/bin/activate | |||
}} | |||
Now that the environment has been activated we can install the Python packages we want to use, in this case <code>pandas</code>. | |||
{{Commands|prompt=(env_gurobi) [name@server ~] $ | |||
| pip install --no-index pandas | |||
Ignoring pip: markers 'python_version < "3"' don't match your environment | |||
Looking in links: /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/nix/avx2, /cvmfs/soft.computecanada.ca/custom/python/wheelhouse/nix/generic, / cvmfs/soft.computecanada.ca/custom/python/wheelhouse/generic | |||
Collecting pandas | |||
Collecting numpy>=1.13.3 (from pandas) | |||
[...] | |||
Successfully installed numpy-1.18.4 pandas-1.0.3 python-dateutil-2.8.1 pytz-2020.1 six-1.15.0 | |||
}} | |||
The third step is to install gurobipy into the environment: | |||
{{Commands|prompt=(env_gurobi) [name@server ~] $ | |||
| cd $EBROOTGUROBI | |||
| python setup.py build --build-base /dev/shm/${USER} install | |||
running build | |||
running build_py | |||
creating /dev/shm/name | |||
creating /dev/shm/name/lib | |||
creating /dev/shm/name/lib/gurobipy | |||
copying lib/python3.7_utf32/gurobipy/__init__.py -> /dev/shm/name/lib/gurobipy | |||
copying lib/python3.7_utf32/gurobipy/gurobipy.so -> /dev/shm/name/lib/gurobipy | |||
running install | |||
running install_lib | |||
creating /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy | |||
copying /dev/shm/name/lib/gurobipy/gurobipy.so -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy | |||
copying /dev/shm/name/lib/gurobipy/__init__.py -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy | |||
byte-compiling /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy/__init__.py to __init__.cpython-37.pyc | |||
running install_egg_info | |||
Writing /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy-9.0.1-py3.7.egg-info | |||
| cd | |||
}} | |||
=== Using the Gurobi-enabled virtual environment === | |||
Python scripts can now import both Pandas and Gurobi: | |||
import pandas as pd | |||
import numpy as np | |||
import gurobipy as gurobi | |||
from gurobipy import * | |||
# [...] | |||
Once created we can activate Gurobi and the environment with: | |||
module load gurobi/8.1.1 | |||
source ~/env_gurobi/bin/activate | |||
python my_gurobi_script.py | |||
Note that we now use <code>python</code> instead of <code>gurobi.sh</code>! | |||
And this is an example job script that we can use: | |||
{{File | |||
|name=gurobi-py_example.sh | |||
|lang="sh" | |||
|contents= | |||
#!/bin/bash | |||
#SBATCH --time=0-00:30 # time limit (D-HH:MM) | |||
#SBATCH --cpus-per-task=1 # number of CPUs (threads) to use | |||
#SBATCH --mem-per-cpu=1000M # memory per CPU (in MB) | |||
module purge | |||
module load gurobi/9.0.1 | |||
source ~/env_gurobi/bin/activate | |||
# Create environment file in current directory setting the number of threads: | |||
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env | |||
python my_gurobi_script.py | |||
}} | |||
== Cite Gurobi == | == Cite Gurobi == | ||
Please see [https://support.gurobi.com/hc/en-us/articles/360013195592-How-do-I-cite-Gurobi-software-for-an-academic-publication- How do I cite Gurobi software for an academic publication?] | Please see [https://support.gurobi.com/hc/en-us/articles/360013195592-How-do-I-cite-Gurobi-software-for-an-academic-publication- How do I cite Gurobi software for an academic publication?] | ||