Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au
Job Queues
The qstat -Q
shows a summary of all Artemis queues. Not all of these can be submitted to directly; some are reserved for users with strategic allocations and many are only accessible via Job Queues#defaultQ. Your job will usually be routed to other “internal” queues after submission depending on the size of your job.
Queues are requested using the following PBS directive:
#PBS -q Queue
Where Queue
is the name of the queue you wish to submit your job to. For example, if you want to run your job in “defaultQ”, you would use:
#PBS -q defaultQ
Queues available to all users
The queues available to all Artemis users are listed below:
defaultQ
If you don’t specify a queue, your job will be placed in this queue. Once placed in this queue, your job will be routed to one of the following internal queues based on the resources requested in your PBS script:
- small
- normal
- large
- highmem (High Memory)
- GPU
For details about how jobs are routed to these internal queues, see the queue resource limits table.
small-express
An “express” queue for small jobs. These jobs will typically start quickly, but will increase your fair share weight 5 times faster than small, normal or large jobs in Job Queues#defaultQ. This queue is useful for quickly testing your jobs before submitting a longer job to Job Queues#defaultQ.
scavenger
A low priority queue that runs jobs on idle resources in strategic allocations. If your job completes it will be “free”, in that it will not increase your Project’s fair share value. The downside is there is no assurance that your job will finish. If a strategic allocation user submits jobs to their allocation and requires resources that a scavenger job is using, the scavenger job will be suspended. If the scavenger job cannot be resumed within half the requested walltime of the scavenger job, it will be terminated and all unsaved progress will be lost.
dtq
This queue grants access to nodes dedicated to input/output (I/O) jobs and can be used to copy data between Artemis, the Research Data Store and other external locations. It can also be used for other I/O intensive tasks, like compressing and archiving data, or simply for moving or copying data around Artemis. Jobs run in this queue will not increase your fair share weight, but we reserve the right to terminate compute jobs running in this queue. For more information about using this queue, see the Data Transfer Queue section.
interactive
This queue is for running interactive (command line) sessions on a compute node. Interactive sessions can only be requested using the qsub –I
command from the command line. You cannot request an interactive session from a job script. This queue attracts a fair share weighting 10 times higher than standard memory jobs in Job Queues#defaultQ. An example command to submit a job to the interactive queue is:
qsub -IP PANDORA -l select=1:ncpus=1:mem=4GB,walltime=1:00:00
remembering to replace PANDORA with your abbreviated project name.
gpu
This queue is for performing computations with programs that use GPUs (Graphical Processing Units). To use GPUs, request gpus in the select
directive, and submit your job to Job Queues#defaultQ:
#PBS -l select=1:ncpus=1:ngpus=1
ngpus
should be 1 to 4 per chunk as GPU nodes have a maximum of 4 GPUs.
The fair share weighting of this queue is 5 times higher than Job Queues#defaultQ.
Condominiums
Some researchers have access to condominium compute resources. Only projects nominated by the relevant condominium owner may submit jobs to these queues.
Queue | Lead Chief Investigator |
---|---|
condo-civil | Associate Professor Luming Shen |
alloc-op | Professor Olivier Piguet |
alloc-dt | Professor Dacheng Tao |
Users with access to condominiums are also free to request their jobs be run in all other publicly accessible queues in addition to their condominium.