Attention: : Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au

Skip to end of banner
Go to start of banner

Jupyter Notebooks

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Current »

Generally, Artemis is not well designed for notebook style interaction (as it is usually inefficient).

But, you CAN do it! Either in a No Machine for simple tasks Interactive access to a compute node

Or if you need more significant resources you can follow the below steps, keeping in mind if you are not actively sitting at your computer waiting for the job to start, you may miss the runtime and waste the resources! Nevertheless…

Submit a job with your specs, something like this:

#!/bin/bash
#PBS -P <PROJECT>
#PBS -l select=1:ncpus=4:ngpus=1:mem=8gb
#PBS -l walltime=01:00:00
#PBS -m b
#PBS -M abcd1234@sydney.edu.au

# Load the libraries and software needed
module load python/3.8.2

# Change directories to wherever you want your notebook to serve from
cd /project/<PROJECT>/mydata

# Sanity check the hostname of the current compute node
echo $HOSTNAME > jupyter_output.txt

#Launch the notebook (and redirect the output/error to a file)
jupyter notebook --no-browser >> jupyter_output.txt 2>&1

This will email you when your jobs starts, and it will output the Artemis host and jupyter token will need to connect.

The host will be something like "hpc213". This line in the output file will give you everything you need: 

...

[I 10:47:59.718 NotebookApp] The Jupyter Notebook is running at:                      

[I 10:47:59.718 NotebookApp] http://hpc213:11111/?token=2476a9ba8927939c0ed2033baac7f7400fe610a95e251dad

...

Now you have to set up "port forwarding" on your local machine to connect to the Artemis host and notebook port, something like this from another terminal:

ssh -N -L 8888:hpc213:11111 abcd1234@hpc.sydney.edu.au

​Now you should be able to go to http://localhost:8888/ in your browser copy the jupyter notebook key and use a notebook! Easy?

  • No labels