TensorFlow

Tensorflow is installed by default in several python modules available on Artemis. Simply executing, e.g. module load python/3.8.2 will give you a python environment with Tensorflow installed. 

Custom TensorFlow versions

If you need a specific version you can install following this guide, which is an adaption of the README file provided by Bernie Kirby in the directory /usr/local/tensorflow/v100/cuda10.2 on Artemis. For the most up-to-date information, have a look for newer README files in /usr/local/tensorflow/v100 directories.

Tensorflow needs to be installed in a Python virtual environment in a directory you have access to on Artemis. To create your own Python virtual environment and install Tensorflow, enter these commands from a login node, to start an interactive job and set up your virtual environment. Note that here the virtual environment name is "tf". You can replace this with any name you like. And replace <project> with your actual project shortcode:

<unikey>@login$ qsub -I -P <project>
qsub: waiting for job 466227.pbstraining to start
qsub: job 466227.pbstraining ready
<unikey>@hpc213 ~$ cd /project/<project>
<unikey>@hpc213 <project>$ module load python/3.7.7 cuda/10.2.89 openmpi-gcc/4.1.1
<unikey>@hpc213 <project>$ virtualenv --system-site-packages /project/<project>/tf
<unikey>@hpc213 <project>$ source tf/bin/activate
(tf) <unikey>@hpc213 <project>$ pip install -U /usr/local/tensorflow/v100/cuda10.2/tensorflow-2.7.0-cp37-cp37-linux_x86_64.whl

There are severl combinations of python and tensorflow version you can install. See available versions of Tensorflow by executing:

ls /usr/local/tensorflow/v100/cuda10.2

You can optionally install most other packages as normal (an interactive job is recommended though). For example, to install/upgrade Keras, run:

(tf) $ pip install -U keras

Example job script

An example job script for Tensorflow:

#!/bin/bash
#PBS -P <project>
#PBS -l select=1:ncpus=1:mem=4gb:ngpus=1
#PBS -l walltime=1:00:00

module load python/3.8.2 cuda/10.2.89
source /project/<project>/tf/bin/activate

cd "$PBS_O_WORKDIR"
python myscript.py