Jupyter Notebooks

Generally, Artemis is not well designed for notebook style interaction (as it is usually inefficient).

But, you CAN do it! Either in a No Machine for simple tasks https://sydneyuni.atlassian.net/wiki/spaces/RC/pages/190349512

Or if you need more significant resources you can follow the below steps, keeping in mind if you are not actively sitting at your computer waiting for the job to start, you may miss the runtime and waste the resources! Nevertheless…

Submit a job with your specs, something like this:

#!/bin/bash
#PBS -P <PROJECT>
#PBS -l select=1:ncpus=4:ngpus=1:mem=8gb
#PBS -l walltime=01:00:00
#PBS -m b
#PBS -M abcd1234@sydney.edu.au

# Load the libraries and software needed
module load python/3.8.2

# Change directories to wherever you want your notebook to serve from
cd /project/<PROJECT>/mydata

# Sanity check the hostname of the current compute node
echo $HOSTNAME > jupyter_output.txt

#Launch the notebook (and redirect the output/error to a file)
jupyter notebook --no-browser --ip=0.0.0.0 --port=11111 >> jupyter_output.txt 2>&1

This will email you when your jobs starts, and it will output the Artemis host and jupyter token will need to connect.

The host will be something like "hpc213". This line in the output file will give you everything you need:

...
[I 10:47:59.718 NotebookApp] The Jupyter Notebook is running at:
[I 10:47:59.718 NotebookApp] http://hpc213:11111/?token=2476a9ba8927939c0ed2033baac7f7400fe610a95e251dad
...

Now you have to set up "port forwarding" on your local machine to connect to the Artemis host and notebook port, something like this from another terminal:

ssh -N -L 8888:hpc213:11111 abcd1234@hpc.sydney.edu.au

Now you should be able to go to http://localhost:8888/ in your browser copy the jupyter notebook key and use a notebook! Easy?
Note: you must specify the ip as 0.0.0.0, the choice of port is arbitrary and can be left as default 8888, but should be something not in use on Artemis or something reserved for another service.