The AWS Simple Queue Services (SQS) and pipeline cluster supports XNAT scale-out compute capability. This provides a more robust environment for handling multi-user peak periods.
Instruction
XNAT uses AWS SQS (Simple Queue System) to manage and queue the jobs. At the moment, a maximum of 6 concurrent jobs can be executed in XNAT pipeline cluster. All the submitted pipeline jobs will be queued in XNAT system and also the AWS SQS.
The pipeline cluster caters to running small pipeline jobs (single core, requires less than 4-6GB RAM for each job), as it is consisted of up to 3 VMs, each VM (2vCPU and 8GB RAM) can run maximum 2 concurrent jobs. Jobs requiring more than 4-6GB RAM or multiple CPUs should consider running on Artemis. Please refer to the User guide to run jobs in Artemis.
| Drawio | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
There is no change on how to run pipeline in XNAT. Please refer to the below link for how to run a pipeline in XNAT
https://wiki.xnat.org/documentation/how-to-use-xnat/running-pipelines-in-xnat
...
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
| hidden | true |
|---|
...
Pipelines in XNAT are powerful mini-applications that can be run on your project data, to aid in complex processing or leverage the power of large computing clusters. Some pipeline-enabled workflows are carried out automatically without any human intervention, like auto-QC'ing images as they are added to your project archive. Others require a person to do manual steps, such as drawing a region of interest.
When pipelines are integrated into your project, you can:
- set up project-based workflows with project specific and experiment specific parameters,
- track a pipeline and send email notifications, and
- capture provenance information as the pipeline executes.
See the following Links: