Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au
Data transfer
Working with research compute systems will require you to move your data between your personal computer, the systems you work on, and various storage systems. There are lots of ways to move data, Globus and FileSender as the preferred ways to move your research data securely.
Recommended approaches
The University supports multiple data movement services, the right one for you will depend on your dataset and the locations you’re moving it between. There are multiple protocols and tools available for transferring data. Each has its strengths and limitations.
Method | Use case | Benefits | Limitations |
|---|---|---|---|
scp/sftp | small, ad-hoc file transfers | simple, widely available | fragile for large data, no automatic resume |
rsync | synchronising directories | efficient for incremental updates | manual retries, complex flags, not fault-tolerant |
browser (HTTP) downloads | small public datasets | easy for one-off use | not suitable for large or private datasets |
cloud sync clients | desktop-focused workflows | convenient for local files | poor fit for shared storage and HPC |
managed transfer services (Globus and FileSender) | research-scale workflows | reliable, scalable, auditable | requires some setup and understanding |
Please be aware that we are rolling out access to Globus for early adopters in Q1 with a broader rollout in Q2 2026. If you would like to be an early adopter, please get in touch with SIH (https://sih.tools/request).
What is required for data movement?
At a minimum, moving data between systems successfully requires:
A source and a destination
A network connection between them
A protocol that controls how data is transferred
Appropriate authentications and permissions
A way to handle errors, interruptions, and retries
Supported services
The University of Sydney supports data movement between a range of research computing services, including:
Research computing platforms
University-managed storage services (RDS)
Selected cloud platforms
Researcher-managed systems
For most supported services, Globus and FileSender are the recommended mechanisms for data transfer.
Source and destination
All data movement involves a source (where data is coming from) and a destination (where data is moved to). In research computing these are commonly:
A workstation or laptop
A shared research storage system like RDS
An HPC platform
A cloud storage system
An external institution’s system
Understanding both ends of a transfer is important because access permissions can differ between systems, performance characteristics vary, and not all protocols are supported everywhere. Managed services like Globus are designed to handle transfers between heterogeneous systems without requiring users to manage these differences manually.
Network connections
Data transfers rely on network connectivity between the source and destination. In practice network connections can vary due to network reliability, firewalls and security controls, long-distance connections, and institutional network policies.
Beware that tools that depend on continuous interactive connections are more likely to fail when networks are unstable. Managed transfer services like Globus and FileSender are better suited to research workflows because they can tolerate interruptions and resume automatically.
Managed services
Globus
See the Globus pages for details on how it can be accessed and used:
Filesender
See the FileSender pages for details on how it can be accessed and used:
CLI based methods
You may prefer to use familiar command-line tools like rsync. For more information on these methods see: