Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au
Artemis Storage
- Stephen Kolmann
Artemis uses a Lustre file system that is accessible from any compute node and provides excellent I/O performance (especially for large files). The Lustre file system contains the following storage spaces:
- Home (
/home/abcd1234, 10 GB per person) - Project (
/project/PANDORA, 1 TB per project quota) - Scratch (
/scratch/PANDORA, no quotas applied)
Remember to replace abcd1234 with your UniKey and PANDORA with your abbreviated project name.
Data is not backed up at all, so we cannot offer any ability to retrieve lost data. For long-term, backed-up storage, use the Research Data Store.
/home
Each researcher has their own home directory within Artemis. This can be used to store program code, batch scripts and other files. Please note that the home directory has limited space and is intended for storing code and configuration information. Data on /home is backed up in case of system failure (that is, if the Lustre filesystem fails), but files cannot be recovered if they are accidentally deleted. Important data should be stored in the Research Data Store.
To check your /home usage, type pquota.
/project
The default allocation is 1 TB of project storage, shared between all members of the project. Please note that no additional space will be provided. If you require more disk space for job output, use /scratch instead. Since /project (or /scratch or /home) is not backed up, please back up all important data to the Research Data Store regularly.
To check your /project usage, type pquota.
/scratch
/scratch is a 490 TB pool of shared storage for data that needs to be saved during a job, but can be deleted once the job completes. As this directory is shared by all users, please do not leave your data in this area for longer than absolutely necessary. Transfer important files to /project if you need them after your job finishes. /scratch is not backed up, so please back up important data to the Research Data Store.
To see how much /scratch space is available, type:
df -h /scratch
To see how much /scratch space your project is using, type:
lfs quota -hg RDS-ICT-PANDORA-RW /scratch
Remember to replace RDS-ICT-PANDORA-RW with your project name, as described in the Projects and UniKeys section.
For details about disk usage by specific sub-directories, type:
du -sh /scratch/PANDORA/directory
where the last argument is the directory you want to know the size of.
Storage management procedure
We delete data older than 3 months from /scratch. Please ensure you regularly copy important data off /scratch, and clean up unneeded data in /scratch often.