Gurobi: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(add translate tags)
(Marked this version for translation)
Line 16: Line 16:
  My Compute Canada username is "_______" and I am a member of the academic institution "_____________________".  This message confirms that I will only use the Compute Canada Gurobi license provided on Compute Canada systems for the purpose of non-commercial research project(s) to be published in publicly available article(s).
  My Compute Canada username is "_______" and I am a member of the academic institution "_____________________".  This message confirms that I will only use the Compute Canada Gurobi license provided on Compute Canada systems for the purpose of non-commercial research project(s) to be published in publicly available article(s).


== Interactive Allocations ==
== Interactive Allocations == <!--T:41-->


===Gurobi Command-Line Tools=== <!--T:5-->
===Gurobi Command-Line Tools=== <!--T:5-->


  [gra-login2:~] salloc --time=1:00:0 --cpus-per-task=8 --mem=1G --account=def-xyz
  <!--T:42-->
[gra-login2:~] salloc --time=1:00:0 --cpus-per-task=8 --mem=1G --account=def-xyz
  [gra800:~] module load gurobi
  [gra800:~] module load gurobi
  [gra800:~] gurobi_cl Record=1 Threads=8 Method=2 ResultFile=p0033.sol LogFile=p0033.log $GUROBI_HOME/examples/data/p0033.mps
  [gra800:~] gurobi_cl Record=1 Threads=8 Method=2 ResultFile=p0033.sol LogFile=p0033.log $GUROBI_HOME/examples/data/p0033.mps
Line 42: Line 43:
  gurobi> help()
  gurobi> help()


<!--T:43-->
where
where
     * https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
     * https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
Line 50: Line 52:
===Replay API Call Recording=== <!--T:7-->
===Replay API Call Recording=== <!--T:7-->


  [gra800:~] gurobi_cl recording000.grbr
  <!--T:44-->
[gra800:~] gurobi_cl recording000.grbr


<!--T:45-->
Reference: https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
Reference: https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html


== Batch Jobs ==
== Batch Jobs == <!--T:46-->


=== Model in LP-format ===  
=== Model in LP-format === <!--T:47-->


<!--T:48-->
This is an example job script for an optimization model written in the [https://www.gurobi.com/documentation/8.1/refman/lp_format.html LP format].
This is an example job script for an optimization model written in the [https://www.gurobi.com/documentation/8.1/refman/lp_format.html LP format].
{{File
{{File
Line 69: Line 74:
module load gurobi
module load gurobi


<!--T:49-->
# Create environment file in current directory setting the number of threads:
# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env


<!--T:50-->
gurobi_cl ${GUROBI_HOME}/examples/data/coins.lp
gurobi_cl ${GUROBI_HOME}/examples/data/coins.lp
}}
}}


=== Job using Gurobi Python ===  
=== Job using Gurobi Python === <!--T:51-->


<!--T:52-->
This is an example jobscript for a model using [https://www.gurobi.com/documentation/8.1/refman/py_python_api_overview.html#sec:Python Gurobi-Python].
This is an example jobscript for a model using [https://www.gurobi.com/documentation/8.1/refman/py_python_api_overview.html#sec:Python Gurobi-Python].
{{File
{{File
Line 89: Line 97:
module load gurobi
module load gurobi


<!--T:53-->
# Create environment file in current directory setting the number of threads:
# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env


<!--T:54-->
gurobi.sh  ${GUROBI_HOME}/examples/python/facility.py
gurobi.sh  ${GUROBI_HOME}/examples/python/facility.py
}}
}}


== Using Gurobi in Python virtual environments ==
== Using Gurobi in Python virtual environments == <!--T:55-->


<!--T:56-->
Gurobi brings it's own version of Python but that one does not contain any 3rd-party Python packages except Gurobi.  In order to use Gurobi together with popular Python  
Gurobi brings it's own version of Python but that one does not contain any 3rd-party Python packages except Gurobi.  In order to use Gurobi together with popular Python  
packages like NumPy, Matplotlib, Pandas and others, we need to create a [[Python#Creating_and_using_a_virtual_environment|virtual Python environment]] in which we can install both <code>gurobipy</code> and e.g. <code>pandas</code>.
packages like NumPy, Matplotlib, Pandas and others, we need to create a [[Python#Creating_and_using_a_virtual_environment|virtual Python environment]] in which we can install both <code>gurobipy</code> and e.g. <code>pandas</code>.


<!--T:57-->
Before we start, we need to decide which combination of versions for Gurobi and Python to use.
Before we start, we need to decide which combination of versions for Gurobi and Python to use.


Line 120: Line 132:
<translate>
<translate>


<!--T:58-->
We see that <code>gurobi/8.1.1</code> brings it's own installation of <code>python2.7/</code> and Python packages for Python 2.7, 3.5, 3.6 and 3.7 (<code>pythonX.Y_utf32/</code>),
We see that <code>gurobi/8.1.1</code> brings it's own installation of <code>python2.7/</code> and Python packages for Python 2.7, 3.5, 3.6 and 3.7 (<code>pythonX.Y_utf32/</code>),
while <code>gurobi/9.0.1</code> by default uses <code>python3.7/</code> and brings Python packages for Python 2.7, 3.5, 3.6, 3.7 and 3.8 (<code>pythonX.Y_utf32/</code>).
while <code>gurobi/9.0.1</code> by default uses <code>python3.7/</code> and brings Python packages for Python 2.7, 3.5, 3.6, 3.7 and 3.8 (<code>pythonX.Y_utf32/</code>).


<!--T:59-->
In this example we want to create a Python environment based on <code>python/3.7</code> in which we want to use <code>gurobi/9.0.1</code> and install the Pandas package.
In this example we want to create a Python environment based on <code>python/3.7</code> in which we want to use <code>gurobi/9.0.1</code> and install the Pandas package.


=== Creating a Python virtual environments with Gurobi ===
=== Creating a Python virtual environments with Gurobi === <!--T:60-->


<!--T:61-->
These steps need to be done only once per system.  
These steps need to be done only once per system.  


<!--T:62-->
The first step is to load the modules, [[Python#Creating_and_using_a_virtual_environment|create the virtual environment]] and activate it.
The first step is to load the modules, [[Python#Creating_and_using_a_virtual_environment|create the virtual environment]] and activate it.


<!--T:63-->
{{Commands|prompt=[name@server ~] $
{{Commands|prompt=[name@server ~] $
| module load gurobi/9.0.1 python/3.7
| module load gurobi/9.0.1 python/3.7
Line 141: Line 158:
}}
}}


<!--T:64-->
Now that the environment has been activated we can install the Python packages we want to use, in this case <code>pandas</code>.
Now that the environment has been activated we can install the Python packages we want to use, in this case <code>pandas</code>.


<!--T:65-->
{{Commands|prompt=(env_gurobi) [name@server ~] $
{{Commands|prompt=(env_gurobi) [name@server ~] $
| pip install --no-index  pandas
| pip install --no-index  pandas
Line 153: Line 172:
}}
}}


<!--T:66-->
The third step is to install gurobipy into the environment:
The third step is to install gurobipy into the environment:


<!--T:67-->
{{Commands|prompt=(env_gurobi) [name@server ~] $
{{Commands|prompt=(env_gurobi) [name@server ~] $
| cd $EBROOTGUROBI
| cd $EBROOTGUROBI
Line 176: Line 197:
}}
}}


=== Using the Gurobi-enabled virtual environment ===
=== Using the Gurobi-enabled virtual environment === <!--T:68-->


<!--T:69-->
Python scripts can now import both Pandas and Gurobi:
Python scripts can now import both Pandas and Gurobi:


  import pandas as pd
  <!--T:70-->
import pandas as pd
  import numpy as np
  import numpy as np
  import gurobipy as gurobi
  import gurobipy as gurobi
Line 186: Line 209:
  # [...]
  # [...]


<!--T:71-->
Once created we can activate Gurobi and the environment with:
Once created we can activate Gurobi and the environment with:


  module load gurobi/9.0.1
  <!--T:72-->
module load gurobi/9.0.1
  source ~/env_gurobi/bin/activate
  source ~/env_gurobi/bin/activate
  python  my_gurobi_script.py
  python  my_gurobi_script.py


<!--T:73-->
Note that we now use <code>python</code> instead of <code>gurobi.sh</code>!
Note that we now use <code>python</code> instead of <code>gurobi.sh</code>!


<!--T:74-->
And this is an example job script that we can use:
And this is an example job script that we can use:


Line 216: Line 243:
<translate>
<translate>


== Cite Gurobi ==
== Cite Gurobi == <!--T:75-->


<!--T:76-->
Please see [https://support.gurobi.com/hc/en-us/articles/360013195592-How-do-I-cite-Gurobi-software-for-an-academic-publication- How do I cite Gurobi software for an academic publication?]
Please see [https://support.gurobi.com/hc/en-us/articles/360013195592-How-do-I-cite-Gurobi-software-for-an-academic-publication- How do I cite Gurobi software for an academic publication?]
</translate>
</translate>

Revision as of 16:57, 21 July 2020

Other languages:

Gurobi is a commercial software suite for solving complex optimization problems. This page wiki page describes the non-commercial use of Gurobi software on Compute Canada clusters and is currently a work in progress.

License limitations

Compute Canada supports and provides a free license to use Gurobi on the Graham, Cedar, Beluga and Niagara clusters. The license provides a total number of 4096 simultaneous uses (tokens in use) and permits distributed optimization with up to 100 nodes. A single user can run multiple simultaneous jobs. In order to use Gurobi you must agree to certain conditions. Please contact support and include a copy of the following completed Academic Usage Agreement. You will then be added into the Compute Canada license file as a permitted user within a few days:

Academic Usage Agreement

My Compute Canada username is "_______" and I am a member of the academic institution "_____________________".  This message confirms that I will only use the Compute Canada Gurobi license provided on Compute Canada systems for the purpose of non-commercial research project(s) to be published in publicly available article(s).

Interactive Allocations

Gurobi Command-Line Tools

[gra-login2:~] salloc --time=1:00:0 --cpus-per-task=8 --mem=1G --account=def-xyz
[gra800:~] module load gurobi
[gra800:~] gurobi_cl Record=1 Threads=8 Method=2 ResultFile=p0033.sol LogFile=p0033.log $GUROBI_HOME/examples/data/p0033.mps
[gra800:~] gurobi_cl --help

Gurobi Interactive Shell

[gra-login2:~] salloc --time=1:00:0 --cpus-per-task=8 --mem=1G --account=def-xyz
[gra800:~] module load gurobi
[gra800:~] echo "Record 1" > gurobi.env    see *
[gra800:~] gurobi.sh
gurobi> m = read('/cvmfs/restricted.computecanada.ca/easybuild/software/2017/Core/gurobi/8.1.1/examples/data/glass4.mps') 
gurobi> m.Params.Threads = 8               see **
gurobi> m.Params.Method = 2
gurobi> m.Params.ResultFile = "glass4.sol"
gurobi> m.Params.LogFile = "glass4.log"
gurobi> m.optimize()
gurobi> m.write('glass4.lp')
gurobi> m.status                           see ***
gurobi> m.runtime                          see ****
gurobi> help()

where 

   * https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html
  ** https://www.gurobi.com/documentation/8.1/refman/parameter_descriptions.html
 *** https://www.gurobi.com/documentation/8.1/refman/optimization_status_codes.html
**** https://www.gurobi.com/documentation/8.1/refman/attributes.html

Replay API Call Recording

[gra800:~] gurobi_cl recording000.grbr

Reference: https://www.gurobi.com/documentation/8.1/refman/recording_api_calls.html

Batch Jobs

Model in LP-format

This is an example job script for an optimization model written in the LP format.

File : gurobi_example.sh

#!/bin/bash
#SBATCH --time=0:30           # time limit (D-HH:MM)
#SBATCH --mem-per-cpu=1000M   # memory per CPU (in MB)
module purge  
module load gurobi

# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

gurobi_cl ${GUROBI_HOME}/examples/data/coins.lp


Job using Gurobi Python

This is an example jobscript for a model using Gurobi-Python.

File : gurobi-py_example.sh

#!/bin/bash
#SBATCH --time=0-00:30        # time limit (D-HH:MM)
#SBATCH --cpus-per-task=1     # number of CPUs (threads) to use
#SBATCH --mem-per-cpu=1000M   # memory per CPU (in MB)
module purge  
module load gurobi

# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

gurobi.sh  ${GUROBI_HOME}/examples/python/facility.py


Using Gurobi in Python virtual environments

Gurobi brings it's own version of Python but that one does not contain any 3rd-party Python packages except Gurobi. In order to use Gurobi together with popular Python packages like NumPy, Matplotlib, Pandas and others, we need to create a virtual Python environment in which we can install both gurobipy and e.g. pandas.

Before we start, we need to decide which combination of versions for Gurobi and Python to use. 

[name@server ~] $ module load gurobi/8.1.1
[name@server ~] $ cd $EBROOTGUROBI/lib
[name@server ~] $ ls -dF python*
  python2.7/        python2.7_utf32/  python3.6_utf32/
  python2.7_utf16/  python3.5_utf32/  python3.7_utf32/
[name@server ~] $ module load gurobi/9.0.1
[name@server ~] $ cd $EBROOTGUROBI/lib
[name@server ~] $ ls -dF python*
  python2.7_utf16/  python3.5_utf32/  python3.7/        python3.8_utf32/
  python2.7_utf32/  python3.6_utf32/  python3.7_utf32/
[name@server ~] $ cd


We see that gurobi/8.1.1 brings it's own installation of python2.7/ and Python packages for Python 2.7, 3.5, 3.6 and 3.7 (pythonX.Y_utf32/), while gurobi/9.0.1 by default uses python3.7/ and brings Python packages for Python 2.7, 3.5, 3.6, 3.7 and 3.8 (pythonX.Y_utf32/).

In this example we want to create a Python environment based on python/3.7 in which we want to use gurobi/9.0.1 and install the Pandas package.

Creating a Python virtual environments with Gurobi

These steps need to be done only once per system.

The first step is to load the modules, create the virtual environment and activate it. 

[name@server ~] $ module load gurobi/9.0.1 python/3.7
[name@server ~] $ virtualenv --no-download  ~/env_gurobi
  Using base prefix '/cvmfs/soft.computecanada.ca/easybuild/software/2017/Core/python/3.7.4'
  New python executable in /home/name/env_gurobi/bin/python
  Installing setuptools, pip, wheel...
  done.
[name@server ~] $ source ~/env_gurobi/bin/activate


Now that the environment has been activated we can install the Python packages we want to use, in this case pandas.


The third step is to install gurobipy into the environment: 

(env_gurobi) [name@server ~] $ cd $EBROOTGUROBI
(env_gurobi) [name@server ~] $ python setup.py build --build-base /tmp/${USER} install
  running build
  running build_py
  creating /tmp/name
  creating /tmp/name/lib
  creating /tmp/name/lib/gurobipy
  copying lib/python3.7_utf32/gurobipy/__init__.py -> /tmp/name/lib/gurobipy
  copying lib/python3.7_utf32/gurobipy/gurobipy.so -> /tmp/name/lib/gurobipy
  running install
  running install_lib
  creating /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  copying /tmp/name/lib/gurobipy/gurobipy.so -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  copying /tmp/name/lib/gurobipy/__init__.py -> /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy
  byte-compiling /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy/__init__.py to __init__.cpython-37.pyc
  running install_egg_info
  Writing /home/name/env_gurobi/lib/python3.7/site-packages/gurobipy-9.0.1-py3.7.egg-info
(env_gurobi) [name@server ~] $ cd


Using the Gurobi-enabled virtual environment

Python scripts can now import both Pandas and Gurobi: 

import pandas as pd
import numpy as np
import gurobipy as gurobi
from gurobipy import *
# [...]

Once created we can activate Gurobi and the environment with: 

module load gurobi/9.0.1
source ~/env_gurobi/bin/activate
python  my_gurobi_script.py

Note that we now use python instead of gurobi.sh!

And this is an example job script that we can use:


File : gurobi-py_example.sh

#!/bin/bash
#SBATCH --time=0-00:30        # time limit (D-HH:MM)
#SBATCH --cpus-per-task=1     # number of CPUs (threads) to use
#SBATCH --mem-per-cpu=1000M   # memory per CPU (in MB)
module purge
module load gurobi/9.0.1
source ~/env_gurobi/bin/activate

# Create environment file in current directory setting the number of threads:
echo "Threads ${SLURM_CPUS_ON_NODE:-1}" > gurobi.env

python  my_gurobi_script.py


Cite Gurobi

Please see How do I cite Gurobi software for an academic publication?

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Gurobi&oldid=86456"