Attention: Confluence is not suitable for the storage of highly confidential data. Please ensure that any data classified as Highly Protected is stored using a more secure platform.
If you have any questions, please refer to the University's data classification guide or contact ict.askcyber@sydney.edu.au

SFTP access to the Research Data Store

Owned by Stephen Kolmann
Last updated: Jul 29, 2021
5 min read

RDS is accessible via SFTP using the following servers:

Use hpc.sydney.edu.au for data transfers between Artemis HPC and RDS only. Use research-data-ext.sydney.edu.au or research-data-int.sydney.edu.au for transfers between RDS and any other location.

The path to your RDS directory on all three servers is /rds/PRJ-<Project>, where <Project> is your abbreviated project name as specified in the corresponding Researcher Dashboard (DashR) project (do not include the angle brackets).

Your RDS directory is not visible in /rds. You must type cd /rds/PRJ-<Project> to access your directory. You cannot reach it by browsing in a GUI SFTP window or by typing "ls" in /rds.

There are many ways of establishing a SFTP session. You can use any of the below methods to access RDS via SFTP. If you need to access to an internal service (research-data-int-int or hpc) from off-campus, first connect to the University VPN, then connect to the relevant internal service.


External (research-data-ext)

Internal (research-data-int)

FileZilla

Cyberduck

SFTP client

Interactive SSH

scp

rsync

In all examples below, whenever you see angle brackets < >, replace whatever is inside the angle brackets AND the angle brackets with the content relevant to you. For example, if you see <UniKey>, replace it with your UniKey. If your UniKey was abcd1234, replace <UniKey> with abcd1234.

Connecting to RDS using FileZilla

FileZilla is a freely available SFTP client. You can download a copy from their website: http://filezilla-project.org. Recently, the FileZilla installer comes bundled with affiliate software. Ensure you select "no" when asked if you wish to install affiliate software. Only say "yes" to installing FileZilla.

To transfer data from your local computer to RDS using FileZilla, follow these steps:

  1. Go to File -> Site Manager in the menu bar at the top of the program.
  2. Click New Site in the lower left-hand corner and give this connection a name. The name is for your reference only.
  3. Click the General tab in Site Manager.
  4. Type research-data-ext.sydney.edu.au in the Host field.
  5. Leave the port blank.
  6. Select the SFTP protocol in the protocol field.
  7. Select Normal in the logon type field.
  8. Type your UniKey in the User field.
  9. Type your UniKey password in the Password field.
  10. Click Connect.
  11. An Unknown host key box will pop up. Click Always trust this host, add this key to the cache box and then click OK.

If you successfully logged on, you will see a message saying “Directory listing of “/home/<unikey>” successful”. The final step is to change directories to your RDS directory:

    12. Type /rds/PRJ-<Project> in the remote site field, remembering to replace <Project> with your short project name, as specified in Researcher Dashboard (DashR).

If you do not complete step 12, your data will be saved in /home/<unikey>, which is not accessible to Artemis HPC or your group members.

Your RDS directory is not visible in /rds. You must type /rds/PRJ-<Project>, as specified in step 12, to access your directory. You cannot reach it by navigating the directory tree.

Do not store research data in your home directories on research-data-int.sydney.edu.au or research-data-ext.sydney.edu.au. These directories have limited storage and are provided to store environment configuration files and SSH keys only.

To transfer data between RDS and your local computer, drag and drop files between the left-hand side (local files) and the right-hand side (RDS files).

Connecting to RDS using CyberDuck

Download Cyberduck from https://cyberduck.io, then open Cyberduck and connect to RDS:

  1. Click Open Connection.
  2. Select SFTP (SSH File Transfer Protocol) from the drop-down menu at the top of the Open Connection box.
  3. In the Server field, type research-data-ext.sydney.edu.au.
  4. In the Username field, type your UniKey.
  5. In the Password field, type your UniKey password.
  6. Click Connect.
  7. When the Unknown fingerprint box appears, click the Always check box in the lower-left hand corner, then click Allow.

If you successfully logged in, you will see a directory listing of /home/<UniKey>.

      8. Press Ctrl + g, then type /rds/PRJ-<Project>, remembering to replace <Project> with your short project name, as specified in Researcher Dashboard (DashR).

You can then transfer data to and from your local computer and RDS by dragging and dropping files between your computer’s file explorer and the Cyberduck window.

Connecting to RDS using command-line SFTP

If you have access to a Linux terminal, you can use the command line program sftp to transfer files between your local computer and RDS. Open a Linux Terminal, such as Cygwin on Windows, or the built-in Terminal apps on MacOS or Linux, and type the following to connect to RDS:

sftp <UniKey>@research-data-ext.sydney.edu.au

remembering to replace <UniKey> with your UniKey. This will start a command-line SFTP session. Your terminal will change to look like this:

sftp>

You can use SFTP commands to navigate the filesystem on your local and remote computers. A summary of SFTP commands is shown in the table below. If you connected using the SFTP command above, your local computer is your computer and the remote computer is the remote server that mounts RDS.

SFTP Command

Description

cd <directory>

Change directory on the remote computer

ls

List files on the remote computer

lcd

Change directory on your local computer

lls

List files on your local computer

pwd

Present working directory on the remote computer

lpwd

Present working directory on your local computer

mkdir <directory>

Make a directory on the remote computer

lmkdir <directory>

Make a directory on your local computer

put <file(s)>

Copy files from your local computer to the remote computer

get <file(s)>

Copy files from the remote computer to your local computer

Using scp or rsync

scp and rsync can be used on research-data-int.sydney.edu and hpc.sydney.edu.au only. research-data-ext.sydney.edu.au only allows SFTP connections. An example rsync command to transfer data from your local computer to RDS is:

rsync -tvxPr /path/to/my/files <UniKey>@research-data-int.sydney.edu.au:/rds/PRJ-<Project>/