This guide Tensorflow is installed by default in several python modules available on Artemis. Simply executing, e.g. module load python/3.8.2 will give you a python environment with Tensorflow installed.
| Python module | TensorFlow Version |
|---|---|
| python/3.8.2 | 2.1.0 |
| python/3.7.7 | 2.3.0 |
| python/3.7.2 | 1.13.1 |
| python/3.6.5 | 1.14.1 |
| python/3.5.1 | 1.12.0 |
Custom TensorFlow versions
If you need a specific version you can install following this guide, which is an adaption of the README file provided by Bernie Kirby in the directory /usr/local/tensorflow/v100/cuda10.2 on Artemis. For the most up-to-date information, have a look for newer README files in /usr/local/tensorflow/v100 directories.
| Warning |
|---|
GPU versions of TensorFlow only work on GPU nodes. You cannot run GPU versions of TensorFlow on any other Artemis node. |
| Warning |
Only Keep in mind: only the Tensorflow packages provided by us in /usr/local/tensorflow/v100 will work on Artemis. Tensorflow downloaded from the internet will not work. If you would like to use a new version of Tensorflow, submit a High Performance Computing request for it to be installed. |
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
<unikey>@login$ qsub -I -P <project> qsub: waiting for job 466227.pbstraining to start qsub: job 466227.pbstraining ready <unikey>@hpc213 ~$ cd /project/<project> <unikey>@hpc213 <project>$ module load python/3.8.2 cuda/10.2.89 openmpi-gcc/3.1.5 <unikey>@hpc213 <project>$ virtualenv --system-site-packages /project/<project>/tf <unikey>@hpc213 <project>$ source tf/bin/activate (tf) <unikey>@hpc213 <project>$ pip install -U /usr/local/tensorflow/v100/cuda10.2/tensorflow_gpu-2.97.0-cp38-cp38-linux_x86_64.whl |
...
You can optionally install other packages. For example, to install/upgrade Keras, run:
| Code Block |
|---|
(tf) $ pip install -U keras |
| Warning |
|---|
GPU versions of TensorFlow only work on GPU nodes. You cannot run GPU versions of TensorFlow on any other Artemis node. |
Example job script
An example job script for Tensorflow:
| Code Block | ||||
|---|---|---|---|---|
| ||||
#!/bin/bash #PBS -P <project> #PBS -l select=1:ncpus=1:mem=4gb:ngpus=1:mem=4gb #PBS -l walltime=1:00:00 module load python/3.8.2 cuda/10.2.89 openmpi-gcc/3.1.5 source /project/<project>/tf/bin/activate cd "$PBS_O_WORKDIR" python myscript.py |
...