XNAT Utils (https://pypi.org/project/xnatutils/) is a collection of command-line tools/Python scripts for conveniently downloading, uploading and listing data to/from/on XNAT. The available tools are:
xnat-get - download scans and resources
xnat-put - upload scans and resources (requires write privileges for project you would like to upload to)
xnat-ls - list projects/subjects/sessions/scans
xnat-rename - renames an XNAT session
xnat-varget - set a metadata field (including "custom variables")
xnat-varput - retrieve a metadata field (including "custom variables")
For detailed help information on each tool pass the --help or -h option.
Loading environment module on Artemis
Before you can use xnat-utils on Artemis you will need to load the xnatutils environment module
(artemis)$ module load xnatutils
xnat-get
The xnat-get command is used to download resources from an XNAT instance from the command line (e.g. on Artemis). It is a flexible tool that can used to download resources specified in XML files generated by the XNAT UI or match sessions and scans using wildcard patterns.
Authentication
The first time you use xnat-get you will be prompted to enter the URL of the XNAT server you would like to download from (e.g. https://xnat.sydney.edu.au) and your username and password. If you have a University of Sydney “unikey” that is linked to your XNAT account use this for your username and password. xnat-get will generate a user token and store it in a hidden file in your user directory, ~/.netrc in the following format (with permissions set to 600 on the file)
machine xnat.sydney.edu.au user <your-alias-token> password <your-alias-secret>
If you don't want these credentials stored (e.g. if you are using an insecure account), then pass the --no_netrc (or -n) option to xnat-get. The token will last for two days, after which you will be prompted to reenter your username and password.
If you don’t have a “unikey” account and instead use the Australian Authentication Federation to login to XNAT, you will need to generate this ~/.netrc file yourself after generating a token in the XNAT UI
NB: make sure that the permissions on the ~/.netrc file are 600 otherwise it won’t work.
Using XML downloaded from UI
If you have downloaded an XML file from the “Download Images” page with the list of resources you would need to download, e.g.
then you first need to copy the XML file to Artemis
(your-workstation)$ scp Downloads/my-downloaded-xml-file.xml hpc.sydney.edu.au:~
Then you can use xnat-get to download all the resources listed within it
(artemis)$ xnat-get ~/my-downloaded-xml-file.xml --target ~/my-downloads
Specifying sessions and scans to download from command line
Alternatively you can specifiy the resources you want to download without using the UI. For example, to download all scans in the sessions 556342B DSAD_2010 in the project with ID “001”
(artemis)$ xnat-get 556342B DSAD_2010 --project 001
If you would like to download data from a range of sessions you can use a (regular expression) search patterns. For example the following command
(artemis)$ xnat-get '400.*DX_1' --project BCD4
will download the first imaging session of subjects 40000 - 40099 in the project “BCD4”. Note the single quotes around the pattern string, as these stop the '' being interpreted as a filename glob. Please refer to https://docs.python.org/3/library/re.html for the complete regular expression syntax you can use. However, most of the time just need the '.' wildcard to match any string of characters or perhaps '|'
to specify a list of options. For example the following command
(artemis)$ xnat-get '(1|2|3|9)_MR_(1|3)' --project ABC
will download sessions 1 & 3 for subjects 1, 2, 3 & 9 in project ABC.
By default all scans in the provided session(s) are downloaded to the current working directory unless they are filtered by the provided '--scan' option(s). Both the session name and scan filters can be regular expressions, e.g.
(artemis)$ xnat-get '1_MR_.*' --scan 'ep2d_diff.*'
The destination directory can be specified by the --directory option. Each session will be downloaded to its own folder under the destination directory unless the --subject-dir option is provided in which case the sessions will be grouped under separate subject directories.
If there are multiple resources for a dataset on an XNAT instance (unlikely) the one to download can be specified using the --format option, otherwise the only recognised neuroimaging format (e.g. DICOM, NIfTI, MRtrix format).
DICOM files (ONLY DICOM file) name can be stripped using the option --strip_name or -sn. If specified, the final name will be in the format 000*.dcm.
The downloaded images can be automatically converted to NIfTI or MRtrix formats using dcm2niix or mrconvert (if the tools are installed and on the system path) by providing the '--convert_to' option and specifying the desired format.
(artemis)$ xnat-get TEST001_001_MR01 --scan 'ep2d_diff.*' --convert_to nifti_gz
User credentials can be stored in a ~/.netrc file so that they don't need to be entered each time a command is run. If a new user provided or netrc doesn't exist the tool will ask whether to create a ~/.netrc file with the given credentials.