Command line client XNAT-Utils (Artemis)

XNAT Utils (https://pypi.org/project/xnatutils/) is a collection of command-line tools/Python scripts for conveniently downloading, uploading and listing data to/from/on XNAT. The available tools are:

xnat-get - download scans and resources
xnat-put - upload scans and resources (requires write privileges for project you would like to upload to)
xnat-ls - list projects/subjects/sessions/scans
xnat-rename - renames an XNAT session
xnat-varget - set a metadata field (including "custom variables")
xnat-varput - retrieve a metadata field (including "custom variables")

For detailed help information on each tool pass the --help or -h option.

Loading environment module on Artemis

Before you can use xnat-utils on Artemis you will need to load the xnatutils environment module

(artemis)$ module load xnatutils

NB: This module also loads the Python, Dcm2niix and MRtrix modules

xnat-get

The xnat-get command is used to download resources from an XNAT instance from the command line (e.g. on Artemis). It is a flexible tool that can used to download resources specified in XML files generated by the XNAT UI or match sessions and scans using wildcard patterns.

For in-depth help pass the --help or -h flag to xnat-get

(artemis)$ xnat-get --help

Authentication

The first time you use xnat-get you will be prompted to enter the URL of the XNAT server you would like to download from (e.g. https://xnat.sydney.edu.au) and your username and password. If you have a University of Sydney “unikey” that is linked to your XNAT account use this for your username and password. xnat-get will generate a user token and store it in a hidden file in your user directory, ~/.netrc in the following format (with permissions set to 600 on the file)

machine xnat.sydney.edu.au
user <your-alias-token>
password <your-alias-secret>

If you don't want these credentials stored (e.g. if you are using an insecure account), then pass the --no_netrc (or -n) option to xnat-get. The token will last for two days, after which you will be prompted to reenter your username and password.

If you don’t have a “unikey” account and instead use the Australian Authentication Federation to login to XNAT, you will need to generate this ~/.netrc file yourself after generating a token in the XNAT UI

NB: make sure that the permissions on the ~/.netrc file are 600 otherwise it won’t work.

Using XML downloaded from UI

If you have downloaded an XML file from the “Download Images” page with the list of resources you would need to download, e.g.

then you first need to copy the XML file to Artemis

Then you can use xnat-get to download all the resources listed within it

Specifying sessions and scans to download from command line

Alternatively you can specifiy the resources you want to download without using the UI. For example, to download all scans in the sessions 556342B DSAD_2010 in the project with ID “001”

If you would like to download data from a range of sessions you can use a (regular expression) search patterns. For example the following command

will download the first imaging session of subjects 40000 - 40099 in the project “BCD4”. Note the single quotes around the pattern string, as these stop the '' being interpreted as a filename glob. Please refer to https://docs.python.org/3/library/re.html for the complete regular expression syntax you can use. However, most of the time just need the '.' wildcard to match any string of characters or perhaps '|'
to specify a list of options. For example the following command

will download sessions 1 & 3 for subjects 1, 2, 3 & 9 in project ABC.

By default all scans in the provided session(s) are downloaded to the current working directory unless they are filtered by the provided '--scan' option(s). Both the session name and scan filters can be regular expressions, e.g.

The destination directory can be specified by the --directory option. Each session will be downloaded to its own folder under the destination directory unless the --subject-dir option is provided in which case the sessions will be grouped under separate subject directories.

If there are multiple resources for a dataset on an XNAT instance (unlikely) the one to download can be specified using the --format option, otherwise the only recognised neuroimaging format (e.g. DICOM, NIfTI, MRtrix format).

DICOM files (ONLY DICOM file) name can be stripped using the option --strip_name or -sn. If specified, the final name will be in the format 000*.dcm.

Conversion to NIfTI format on the fly

The downloaded images can be automatically converted to NIfTI or MRtrix Image Format (MIF) formats using dcm2niix or mrconvert (if the tools are installed and on the system path) by providing the '--convert_to' option and specifying the desired format.

By default DICOM → NIfTI conversion is performed by Dcm2niix, but mrconvert can be used instead by passing the --converter flag