TensorFlow

Tensorflow is installed by default in several python modules available on Artemis. Simply executing, e.g. module load python/3.8.2 will give you a python environment with Tensorflow installed.

Python module	TensorFlow Version
python/3.8.2	2.1.0
python/3.7.7	2.3.0
python/3.7.2	1.13.1
python/3.6.5	1.14.1
python/3.5.1	1.12.0

Custom TensorFlow versions

If you need a specific version you can install following this guide, which is an adaption of the README file provided by Bernie Kirby in the directory /usr/local/tensorflow/v100/cuda10.2 on Artemis. For the most up-to-date information, have a look for newer README files in /usr/local/tensorflow/v100 directories.

Keep in mind: only the Tensorflow packages provided by us in /usr/local/tensorflow/v100 will work on Artemis. Tensorflow downloaded from the internet will not work. If you would like to use a new version of Tensorflow, submit a High Performance Computing request for it to be installed.

Tensorflow needs to be installed in a Python virtual environment in a directory you have access to on Artemis. To create your own Python virtual environment and install Tensorflow, enter these commands from a login node, to start an interactive job and set up your virtual environment. Note that here the virtual environment name is "tf". You can replace this with any name you like. And replace <project> with your actual project shortcode:

<unikey>@login$ qsub -I -P <project>
qsub: waiting for job 466227.pbstraining to start
qsub: job 466227.pbstraining ready
<unikey>@hpc213 ~$ cd /project/<project>
<unikey>@hpc213 <project>$ module load python/3.7.7 cuda/10.2.89 openmpi-gcc/4.1.1
<unikey>@hpc213 <project>$ virtualenv --system-site-packages /project/<project>/tf
<unikey>@hpc213 <project>$ source tf/bin/activate
(tf) <unikey>@hpc213 <project>$ pip install -U /usr/local/tensorflow/v100/cuda10.2/tensorflow-2.7.0-cp37-cp37-linux_x86_64.whl

There are severl combinations of python and tensorflow version you can install. See available versions of Tensorflow by executing:

ls /usr/local/tensorflow/v100/cuda10.2

You can optionally install most other packages as normal (an interactive job is recommended though). For example, to install/upgrade Keras, run:

(tf) $ pip install -U keras

GPU versions of TensorFlow only work on GPU nodes. You cannot run GPU versions of TensorFlow on any other Artemis node.

Example job script

An example job script for Tensorflow:

#!/bin/bash
#PBS -P <project>
#PBS -l select=1:ncpus=1:mem=4gb:ngpus=1
#PBS -l walltime=1:00:00

module load python/3.8.2 cuda/10.2.89
source /project/<project>/tf/bin/activate

cd "$PBS_O_WORKDIR"
python myscript.py