transferring files to/from the turing cluster files to/from the turing cluster - version.1...

Transferring Files To/From the Turing Cluster - Version.1

Transferring Files To/From the Turing ClusterUsers with datasets (be they large or small) require methods to transfer data files on and back off of the Turing cluster. The transfers require connections between the cluster and the computer/workstation with access to the data. [Note: All data is subject to ODU IT Policies and Standards including but not limited to Data Classification Standards. The policies can be found at http://occs.odu.edu/policies/index.php]

The Turing FilesystemAs a user, there are several locations on the cluster that are of particular importance regarding storage. The following listed paths are available cluster wide (on both the login nodes and compute nodes.)

● User Home Directory - /home/username

● User Scratch Space - /scratch/username○ {Note: This area is considered temporary storage, and should be considered

volatile.}

● Installed Applications - /cm/shared/apps/applicationname/version

Microsoft WindowsAs with the secure shell, Microsoft Windows does not include native support to connect to the Linux based cluster in order to transfer files. To do this in Microsoft Windows, a SCP (secure copy) or SFTP (secure file transport protocol) client is needed. Though several exist, this document will focus on one named WinSCP. {Note: If the user is on the ODU network with reduced user privileges (i.e. a user is unable to install programs), a request to install the software can be made to the ITS group through the email address [email protected].} WinSCP can either be downloaded and installed like any other application, or downloaded as a self contained, portable application which, can also run on computers without additional permissions.

WinSCP Installation and Connection to Turing

1. Download and install/extract WinSCP from http://winscp.net/2. Open WinSCP by double-clicking on the application icon

Information Technology Services - High Performance Computing Group 1 of 12

http://occs.odu.edu/policies/index.php

mailto:[email protected]

http://winscp.net/


3. If asked, click “Run” in the “Open File - Security Warning” window



4. In the “WinSCP Login” window select “SCP” as the “File Protocol”

5. In the “Host name:” field type “turing.hpc.odu.edu”.6. Enter your username (i.e. the part of your ODU email address before the @ symbol)

and password (i.e. the ODU email account password) in the corresponding fields and then click [Login].



7. If asked “Continue connecting to an unknown server and add its host key to a cache?” select “Yes”

8. You are now connected to the Turing cluster filesystem



WinSCP Uploading/Downloading FilesFiles can both be clicked and dragged onto (uploaded) or off from (downloaded) the cluster filesystem. The left window pane is the user local computer/workstation. The right window pane is the remote connection to the Turing cluster.



Apple Macintosh OSX (GUI)To visually connect to the Turing filesystem, a client with SCP/SFTP protocols is needed. Though several are available in the Apple OSX App Store and Web, this document will focus on the use of one named Cyberduck. The Cyberduck.app can be downloaded for free ($0.00) from http://cyberduck.io/ or purchased from the Apple OSX App Store for $23.99.

Cyberduck.app Installation and Connection to Turing 1. Download and extract Cyberduck from http://cyberduck.io/


http://cyberduck.io/




2. Once extracted double-click the Cyberduck.app icon

3. If asked “...Are you sure you want to open it?” click “Open”

4. In the “Cyberduck” window select “Open Connection”



5. Select “SFTP (SSH File Transfer Protocol)” from the drop-down menu6. In the “Server:” field type turing.hpc.odu.edu7. Enter your username (i.e. the part of your ODU email address before the @ symbol)

and password (i.e. the ODU email account password) in the corresponding fields and then click [Connect].

8. If a window says “Unknown host key for turing.hpc.odu.edu” click “Allow”



9. You are now connected to the Turing filesystem

Cyberduck Uploading/Downloading FilesFiles can both be clicked and dragged onto (uploaded) or off from (downloaded) the cluster filesystem. The window displays the files that are located over the remote connection, on the Turing cluster.



Linux/Unix/OSX TerminalThe commands scp can be accesses to securely copy data to or from the turing cluster. Both commands can be accessed from a command window.

scp Syntax for Uploading/Downloading FilesThe command scp is built on the same principles in which the command cp utilize. scp however, adds the ability to define remotely accessed computers. The local file (in this example) is located on the computer/workstation in which you the user is working. The remote computer/workstation is the Turing cluster.

Uploading Datascp local_filename [email protected]:/remote_file_location/remote_filename{Note: You will be asked for your password to the cluster}

Downloading Datascp [email protected]:/remote_file_location/remote_filename local_filename



Download from a Webpage directly into the clusterData files posted on websites that have direct links can be directly downloaded to a working directory on the cluster using the following command:

wget http://www.someplace.org/datafile

{Note: To use this command, the user must have first logged into the Turing cluster, have a working directory with write access (i.e. /home/username)}


transferring files to/from the turing cluster files to/from the turing cluster - version.1...

Documents