1. RESTAPI Basic Operation Manual
List projects (will return only the projects the user has access to)
curl -u username "https://xnat.sydney.edu.au/data/archive/projects"
^data is returned in JSON format by default. Modify format using the format parameter with html, xml, json, csv.
^username is the XNAT logon username, password will be prompt.
List projects (in XML format and save to a file)
curl -u username "https://xnat.sydney.edu.au/data/archive/projects/?format=xml" -o projects.xml
List files for an experiment (in XML format and save to a file)
curl -u username "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/files?format=xml" -o filelist.xml
Download all DICOM files for a scan (in zip format)
curl -u username "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/DICOM/files?format=zip" -o dicomfiles.zip
Note: In this above example, PVT000 is the project name, Haofei01 is the subject name, "Haofei01_MRI" is the session/experiment name, "4" is the scan name.
Upload all files into the scan
curl -u username -X PUT "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/ABC/files" -F "file=@filelist"
Note: In this above example, PVT000 is the project name, Haofei01 is the subject name, "Haofei01_MRI" is the session/experiment name, "4" is the scan name, "ABC" is the subfolder name, and "files" are necessary to be added.
Create folder
curl -u username -X PUT "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/test"
Creates a subfolder under scans/4 named test. Note that no further subdirectories are allowed.
2. How can we determine the project, subject, session/experiments, scan name via the XNAT Web Interface ?
It is troublesome in using the RESTAPI command to list project, as they are in HTML/XML/JSON format. Another straightforward way to retrieve the information is via the XNAT web interface.
From the web interface, we can see the project name is : PVT100, Subject name is: "Haofei01", and session/experiment name is "Haofei01_MRI", Scan name is "4" .
Then we would like to know the directory structure below scan name, so we click on the "Manage Files" on the right side of the "Actions" Page.
So we know that the "DICOM" files have all the DICOM images we need. So it is easy for us to figure out the URL path mentioned above in the RESTAPI manuals.
2.1 What is "scans" vs "Resources" ?
We noticed in the File Manager screenshot above, there are "scans" vs "Resources".
Scans are the folder created by XNAT for the DICOM data. Therefore, all the DICOM related files (NIFTI, snapshot, description, report) can be placed into the folder.
Resources are the folder to keep all the non-DICOM data. By default it is not created. It has to be manually created.
We can click "Add Folder" in the FileManager to create new folder
Click "Create"
3. Example on how to run a DICOM to NIFTI convertion on Artemis (University of Sydney High Performance Computing).
In order to run on the Artemis, you need to have an Artemis account. For more information, please refer to below guide to logon to Artemis.
https://sydney.edu.au/research_support/hpc/access/index.shtml#newacct
For Artemis user guide, please refer to the University of Sydney Artemis User Guide
Step 1. Logon to Artemis
ddd-mbp:Deployment$ ssh unikey@hpc.sydney.edu.au
unikey@hpc.sydney.edu.au's password:
Last login: Wed May 9 13:53:09 2018 from vlan-2689-10-17-108-122.staff.wireless.sydney.edu.au
------------------------------------------------------------------------------
***************************** ARTEMIS 3.0 ******************************
------------------------------------------------------------------------------
*** !!! LOGIN NODES ARE FOR JOB SETUP & SUBMISSION ONLY !!! ***
------------------------------------------------------------------------------
Use Interactive flag (-I) or defaultQ or small-express queue for job testing
NO ACTUAL COMPUTE WORK IS TO BE CONDUCTED ON LOGIN NODES
Any sustained compute or multicore/memory intensive process on login nodes
will be subject to being killed without notice.
------------------------------------------------------------------------------
!!! MEMORY MUST BE SPECIFIED FOR ALL JOBS - use the mem= PBS directive !!!
------------------------------------------------------------------------------
Please refer to the Quick Start User Guide to get started with Artemis 2.0:
https://informatics.sydney.edu.au/services/artemis/
-------------------------------------------------------------------------------
Step 2. Download the XNAT data into Artemus directory into the folder. For data in big size, it is recommended to put in to the Artemis /scratch directory.
As the example is only a small set of data (32MB), we are putting it into the home folder on the Artemis Logging node.
[unikey@login2 ~]$ curl -u username "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/DICOM/files?format=zip" -o dicomfiles.zip
Enter host password for user 'unikey':
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 22.8M 0 22.8M 0 0 5827k 0 --:--:-- 0:00:04 --:--:-- 5994k
[unikey@login2 ~]$ ls -ltr
-rw-r--r-- 1 unikey linuxusers 23963682 May 11 09:22 dicomfiles.zip
[unikey@login2 ~]$ unzip dicomfiles.zip
Archive: dicomfiles.zip
extracting: Haofei01_MR1/scans/4-Ax_3D_T1/resources/DICOM/files/1.3.6.1.4.1.14301.53.4.1079983.1-4-134-4qu0ab.dcm
extracting: Haofei01_MR1/scans/4-Ax_3D_T1/resources/DICOM/files/1.3.6.1.4.1.14301.53.4.1079983.1-4-34-4qtzjm.dcm
...
[unikey@login2 ~]$ ls -ltr
total 23480
-rw-r--r-- 1 unikey linuxusers 23963682 May 11 09:22 dicomfiles.zip
drwxr-xr-x 3 unikey linuxusers 4096 May 11 09:24 Haofei01_MR1
Step 3. Create the Artemis Scripts
For information about Artemis Job Submission, monitoring and management, please refer to the information page about Artemis.
https://sydneyuni.atlassian.net/wiki/spaces/RC/pages/190284213/Job+Submission
https://sydneyuni.atlassian.net/wiki/spaces/RC/pages/213090348/Job+Monitoring+and+Management
[unikey@login2 ~]$ vi run.pbs
#!/bin/bash
#PBS -P SASTEST
#PBS -l select=1:ncpus=1:mem=1gb
#PBS -l walltime=1:00:00
#PBS -M emailemail@sydney.edu.au
#PBS -m abe
#PBS -j oe
module load dcm2niix
mkdir -p /home/unikey/Haofei01_MR1/scans/4-Ax_3D_T1/resources/NIFTI
dcm2niix -o /home/unikey/Haofei01_MR1/scans/4-Ax_3D_T1/resources/NIFTI/ /home/hfen0483/Haofei01_MR1/scans/4-Ax_3D_T1/resources/DICOM/files
Step 4. Run the Artemis Script
[unikey@login2 ~]$ qsub run.pbs
2238022.pbsserver
Step 5. Upload the result back to XNAT
[unikey@login2 ~]$ cd /home/unikey/Haofei01_MR1/scans/4-Ax_3D_T1/resources/NIFTI/
[unikey@login2 NIFTI]$ ls -ltr
total 21000
-rw-r--r-- 1 hfen0483 RDS-ICT-SASTEST-RW 21496160 May 11 09:29 files_Ax_3D_T1_20140604082236_4.nii
-rw-r--r-- 1 hfen0483 RDS-ICT-SASTEST-RW 751 May 11 09:29 files_Ax_3D_T1_20140604082236_4.json
Note, below command create the "unikey" folder and upload the file in one command in XNAT
[unikey@login2 NIFTI]$ curl -u username -X PUT "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/unikey/files" -F "file=@files_Ax_3D_T1_20140604082236_4.nii"
Enter host password for user 'unikey':
[unikey@login2 NIFTI]$ curl -u username -X PUT "https://xnat.sydney.edu.au/data/archive/projects/PTV000/subjects/Haofei01/experiments/Haofei01_MR1/scans/4/resources/unikey/files" -F "file=@files_Ax_3D_T1_20140604082236_4.json"
Enter host password for user 'unikey':
Step 5. Check from XNAT
Reference
https://wiki.xnat.org/docs16/4-developer-documentation/using-the-xnat-rest-api
https://sydney.edu.au/research_support/hpc/access/index.shtml#newacct