https file transfer guide

HTTPS File Transfer

Specification

Version 1.1 30 30-Jun-2011

Copyright 2009-2010, Westpac Banking Corporation, ABN 33 007 457 141. All rights reserved.

HTTPS File Transfer Specification

Date Version Description

30-Aug-2010 1.0 Original Version

30-Jun-2011 1.1 Added FAQ



Table of Contents

1 Introduction ..................................................................................................... 5

2 Integration Options .......................................................................................... 5

3 Using the Command Line Scripts ...................................................................... 6

3.1 Downloading cURL ............................................................................................. 6

3.2 Configuration .................................................................................................... 6

3.3 Uploading Files .................................................................................................. 6

3.3.1 Example ...................................................................................................... 7

3.3.2 File Names .................................................................................................. 7

3.3.3 Handling Errors ............................................................................................ 7

3.4 Downloading Files .............................................................................................. 7

3.4.1 Example ...................................................................................................... 7

3.4.2 Handling Errors ............................................................................................ 7

4 Developing Custom HTTPS Connectivity ........................................................... 9

4.1 Recommendations ............................................................................................. 9

4.2 Handling Errors.................................................................................................. 9

4.3 Installing the Client Certificate ............................................................................. 9

4.4 Protecting Your Certificate ................................................................................. 10

4.5 Upgrading Your Certificate ................................................................................ 10

5 Appendix: File Transfer Protocol ..................................................................... 11

5.1 Uploading Files ................................................................................................ 11

5.1.1 HTTP Request ............................................................................................ 11

5.1.2 HTTP Response .......................................................................................... 11

5.1.3 Handling Errors .......................................................................................... 12

5.2 Listing Files for Download ................................................................................. 12

5.2.1 HTTP Request ............................................................................................ 12

5.2.2 HTTP Response .......................................................................................... 12

5.2.3 Handling Errors .......................................................................................... 13

5.3 Downloading a File ........................................................................................... 14

5.3.1 HTTP Request ............................................................................................ 14

5.3.2 HTTP Response .......................................................................................... 14



5.3.3 Handling Errors .......................................................................................... 14

5.4 Deleting a File ................................................................................................. 15

5.4.1 HTTP Request ............................................................................................ 15

5.4.2 HTTP Response .......................................................................................... 15

5.4.3 Handling Errors .......................................................................................... 15

6 Appendix: Westpac HTTPS addresses ............................................................. 16

6.1 Test Environment ............................................................................................. 16

6.2 Production Environment .................................................................................... 16

Appendix D FAQ ................................................................................................ 17



1 Introduction

This document defines Westpacs WIBS HTTPS file transfer protocol.

The HTTPS file transfer protocol allows partners to transfer files securely and reliably

over the internet. It provides a standard mechanism for transport using existing

infrastructure (such as web servers, proxy servers and firewalls). Communication uses

standard TCP/IP ports to prevent the need for network firewall changes. SSL is used to

provide encryption of data between partners, and authorisation assures the identity of

each partner.

The intended audience of this document is:

Server administrators who wish to use the provided command line scripts, and

Software developers who wish to implement this messaging protocol in their

software.

Knowledge of HTTP and SSL is assumed.

2 Integration Options

The HTTPS file transfer service is provided to allow your system to easily upload files to

Westpac and download files from Westpac. You have two integration options:

1. Use the provided command-line scripts. These scripts use the cURL command-

line program and standard in and out streams to transfer files. Minimal

configuration work is required.

2. Write code to generate HTTPS requests sent to Westpac and interpret the

responses you receive. A moderate amount of development work is required, and

you will need previous expertise with HTTPS.

These options are further detailed in the following sections.



3 Using the Command Line Scripts

Command line scripts have been provided for Unix and Windows that demonstrate the

use of cURL to upload and download files.

The scripts have been designed to exit with an error code if the file could not be

successfully uploaded or download. Error details will be written to the standard error

stream.

3.1 Downloading cURL

You will need to download the cURL client for your platform from

http://curl.haxx.se/download.html. Be sure to download the correct distribution for your

servers operating system. You must download a version that includes SSL support.

If you are using Windows, we recommend that you download the binary distribution

listed under the heading Win32 - Generic labelled Win32 2000/XP, binary and SSL

and maintained by Gnter Knauf. This document does not provide a direct link because

you should always download the latest version.

3.2 Configuration

You need to edit the configuration script before you can use the upload or download

scripts. The configuration script is named config.cmd on Windows and config.sh on

Unix.

Open the file in a text editor and make the following changes:

1. Set the CERT_FILE variable to the file path of your .pem certificate file, e.g.

c:\fileTransfer\certificate.pem

2. Set the CERT_PASSWORD variable to the password for your .pem certificate file.

3. If you require a proxy to connect to the internet:

a. Uncomment the first PROXY_OPTIONS line and change proxy_address to the host name of your proxy server, and proxy_port to the port number your proxy server listens on.

b. If your proxy server requires authentication, uncomment the second

PROXY_OPTIONS line and change username to your proxy username and password to your proxy password.

4. If you are uploading and downloading files in the test environment, leave the

BASE_URL variable set to https://ws.support.qvalent.com/. If you are uploading

and downloading files in the production environment, set the BASE_URL variable

to https://ws.qvalent.com/.

5. Set the CURL_EXE variable to the file path of the cURL executable file, e.g.

C:\curl-7.21.0\curl.exe

3.3 Uploading Files

The upload script is named uploadFile.cmd on Windows and uploadFile.sh on Unix.

It takes one argument, which is the file name to send to Westpac. The file contents are

read from standard in.



3.3.1 Example

The following example shows how to send a file from the file system:

Windows: uploadFile.cmd mts20100830_01.txt < c:\mts20100830.txt

Unix: uploadFile.sh mts20100830_01.txt < c:\mts20100830.txt

Note that this allows the file name sent to Westpac to be different from the file name on

disk.

Important Security Note: If the file contains credit card numbers, PCI DSS regulations

require that you do not store the file unencrypted on disk. Instead, you must decrypt

the file in memory and stream that data to the input stream of the upload script.

3.3.2 File Names

The file name sent to Westpac must be unique. If you send a different file with the same

name as a previous file, you will receive a HTTP 400 Bad Request error response. To

ensure uniqueness of file names, include the file type, date and a daily sequence number

in the file name.

3.3.3 Handling Errors

If you receive a network error while transmitting your file, you can simply run the same

upload command again (re-using the same file name). This will never result in a

duplicate file because Westpac will silently ignore duplicate files with the same name.

If you receive any other error, it is likely a configuration issue you need to investigate

based on the error message provided.

3.4 Downloading Files

The download files script is named downloadFiles.cmd on Windows and downloadFiles.sh

on Unix. It downloads all your available files and removes them from the download list

(preventing duplicate downloads).

By default, the file contents are echoed to standard out. You will need to modify this

script to save the files to an appropriate location and trigger any required file processing.

Important Security Note: If the file contains credit card numbers, PCI DSS regulations

require that you do not store the file unencrypted on disk. Instead, you must read the

standard out stream from the script and encrypt the file in memory before saving it to

disk.

3.4.1 Example

To download all available files, run the following command:

Windows: downloadFiles.cmd

Unix: downloadFiles.sh


To download the available files, the script performs the following operations:

1. Get a list of all the available files.

2. For each file in that list:

a. Download the file.



b. If the file has not already been processed, run your custom file processing.

c. Delete the file (removing it from the available file list).

Operations 1, 2a and 2c can fail due to network errors. In that event, simply run the

download script again and the file will be downloaded again. The file is not deleted until

it is downloaded and processed, which allows you to retry the download operation in the

event of a failure.

If you receive an error not related to network connectivity or your processing, it is likely

a configuration issue you need to investigate based on the error message provided.



4 Developing Custom HTTPS Connectivity

This section gives an overview of developing your own connectivity code to use the

HTTPS file transfer service. It details the requirements of your system and gives

instruction on how to integrate the file transfer protocol into your system. This section

does not contain details of the file transfer protocol itself. That information is in section

5.

Note that this section only gives a general overview of what needs to be done. Specific

details are not provided since they vary depending on the actual technology you use.

For this reason, Westpac can provide only limited assistance to customers using custom

integration. We recommend that you only undertake a custom integration if you have

previous experience with HTTPS and client certificates.

To use the Westpac HTTPS file transfer service, you will need a HTTP client that supports

the following standards:

HTTP 1.1 http://www.ietf.org/rfc/rfc2616.txt

TLS 1.0 or higher http://www.ietf.org/rfc/rfc2246.txt

4.1 Recommendations

Log detailed information about each file you upload and download. This

information will help you diagnose any potential problems you may encounter

later. You should not log your certificate details or any full credit card numbers

contained in the file.

Protect your certificate file. If necessary, restrict the permissions on this file so

that only your application can access the certificate.

4.2 Handling Errors

If you encounter a network error or HTTP 500 response code, your system should

automatically retry the request. You should wait at least 5 minutes before retrying the

request, and you should only retry for a maximum of 4 hours before you investigate the

cause of the issue manually.

4.3 Installing the Client Certificate

You must use an SSL client certificate to communicate with the Westpac server.

This certificate will be provided to you by Westpac. You may not use a certificate

from another source (e.g. Verisign).

Your certificate is generally provided in PKCS12 (or PFX) format. This file format

requires a password for the certificate. You will be provided with this certificate

password. Do not confuse this password with the password you use to login to

any Westpac websites. Your certificate password will always start with a C.

A few HTTP libraries require the client certificate file to be in PEM format. If this

is the case, please ensure that you get the certificate in the correct format.

Once you have obtained your certificate in the correct format, you must reference

it from your HTTP library. The details of this will vary between HTTP libraries, so

consult the documentation for your HTTP library.



Be sure to document the procedure to install the certificate, since the certificate

must be replaced every 2 years.

4.4 Protecting Your Certificate

Your certificate file is used to authorise your requests to the Westpac server. Do not

give your certificate file to anyone. Never email your certificate in clear text. Always

use a secure method to copy your certificate file to your server.

4.5 Upgrading Your Certificate

Your certificate is valid for a maximum of 2 years. When your certificate is near expiry,

you will be contacted by Westpac with details on how to get a new certificate file. The

steps to install your new certificate will vary depending on your technology, so it is

important to document this procedure for later use.



5 Appendix: File Transfer Protocol

This section documents the HTTPS file transfer protocol for uploading and downloading

files. Authentication is based on your client certificate and the IP address that your

request comes from.

5.1 Uploading Files

5.1.1 HTTP Request

To upload a file, send a HTTPS POST to the following URL:

https://ws.support.qvalent.com/services/messaging/rest/uploadedFiles/{fileName}

You must choose a unique file name for each file you upload. To ensure uniqueness of

file names, include the file type, date and a daily sequence number in the file name.

An example upload HTTP request is shown below. Note that represents an ASCII

carriage-return line-feed sequence.

POST /services/messaging/rest/uploadedFiles/de20100831_01.txt HTTP/1.1

Content-Type: application/octet-stream

Host: ws.support.qvalent.com:443

Content-Length: 19

This is a test file

Note that no special HTTP headers are required. You simply include the file name in the

URL and the file contents in the HTTP request body.

5.1.2 HTTP Response

The table below lists the possible HTTP status codes that can be returned.

HTTP Status Code Meaning

202 Accepted The file has been accepted for processing. The file can still be

rejected at a later time if it is malformed, but the file transfer has

been successful.

400 Bad Request You have reused an old file name for a new file. You must use a

unique file name for each file you upload.

403 Forbidden Either:

You have not specified your client certificate correctly.

Check your configuration.

Or your request has come from an IP address that has not

been registered in Westpacs system.

404 Not Found The file name you have specified is not acceptable. The file name

may contain letters, numbers underscores, dashes and full stops.

Retry your upload request with a different file name.

500 Internal Server

Error

The Westpac server has encountered an internal error. Retry the

same upload request again later.

503 Service

Unavailable

You have tried to upload too many files at the same time. You

should upload only one file at a time.



An example upload HTTP response is shown below. Note that represents an ASCII


HTTP/1.1 202 Accepted

Content-Length: 0

Date: Mon, 30 Aug 2010 23:24:20 GMT


If you receive a network error or HTTP 500 status when transmitting your file, you can

simply run the same upload command again (re-using the same file name). This will

never result in a duplicate file because Westpac will silently ignore duplicate files with

the same name.



5.2 Listing Files for Download

5.2.1 HTTP Request

To get the list of files for download, send a HTTP GET to the following URL:

https://ws.support.qvalent.com/services/messaging/rest/filesForDownload/

You can choose the format of the file list using the HTTP Accept header as shown in the

following table:

HTTP Accept Header Response Format

text/plain The list of files is in plain text with each file name on a

different line. If no files are available for download, the HTTP

status code will be 204. If files are available for download, the

HTTP status code will be 200.

text/xml or

application/xml

The list of files is in XML format as shown in the example in

the next section.

An example HTTP request is shown below where the file list is requested in plain text

format:

GET /services/messaging/rest/filesForDownload/ HTTP/1.1

Accept: text/plain


An example HTTP request is shown below where the file list is requested in XML format:

GET /services/messaging/rest/filesForDownload/ HTTP/1.1

Accept: application/xml


5.2.2 HTTP Response



200 OK The list of files for download is returned in the response body.

204 No Content No files are available for download (plain text only).








been registered in our system.

404 Not Found The URL in the request is incorrect. It should be /services/messaging/rest/filesForDownload/

500 Internal Server

Error


same request again later.

An example plain text HTTP response is shown below. Note that represents an ASCII


HTTP/1.1 200 OK

Expires: 0

Pragma: no-cache

Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate

Content-Type: text/plain

Date: Mon, 30 Aug 2010 23:24:20 GMT

Transfer-Encoding: chunked

16

File1.txt

File2.txt

0

An example XML HTTP response is shown below. Note that represents an ASCII


HTTP/1.1 200 OK

Expires: 0

Pragma: no-cache


Content-Type: text/xml

Date: Mon, 30 Aug 2010 23:24:20 GMT


e8

File1.txt

File2.txt

0


Requesting the list of files for download does not change the server state in any way, so

you can safely request the list as many times as you need to. If you receive a network

error or HTTP 500 status when requesting the list, you can simply repeat the same

request again.





5.3 Downloading a File

5.3.1 HTTP Request

To download a file, select the file name from the list of files available for download and

send a HTTP GET to the following URL:

https://ws.support.qvalent.com/services/messaging/rest/filesForDownload/{fileName}

An example download HTTP request is shown below:

GET /services/messaging/rest/filesForDownload/File1.txt HTTP/1.1


5.3.2 HTTP Response



200 OK The file contents are returned in the response body.






404 Not Found The file name you specified no longer exists in the list of files

available for download.

500 Internal Server

Error


same download request again later.

An example plain text HTTP response is shown below. Note that represents an ASCII


HTTP/1.1 200 OK

Expires: 0

Pragma: no-cache


Content-Type: application/octet-stream

Date: Mon, 30 Aug 2010 23:24:20 GMT


13

This is a test file

0


Downloading a file does not change the server state in any way, so you can safely

download the file as many times as you need to. If you receive a network error or HTTP

500 status when downloading the file, you can simply repeat the same download request

again.





5.4 Deleting a File

5.4.1 HTTP Request

To delete a file from the list of files available for download, send a HTTP DELETE to the

following URL:


An example download HTTP request is shown below:

DELETE /services/messaging/rest/filesForDownload/File1.txt HTTP/1.1


5.4.2 HTTP Response



204 No Content The file was deleted from the list successfully.






404 Not Found The file name you specified is not valid. Ensure that the file name

you are trying to delete was copied exactly from the list of files for

download.

500 Internal Server

Error


same delete request again later.

An example delete HTTP response is shown below. Note that represents an ASCII


HTTP/1.1 204 No Content

Date: Mon, 30 Aug 2010 23:24:20 GMT


0


If you receive a network error or HTTP 500 status, you can simply repeat the same

delete request again. If the file has already been deleted, the server will ignore the

delete request and return a HTTP 204 status.





6 Appendix: Westpac HTTPS addresses

6.1 Test Environment

Upload File

https://ws.support.qvalent.com/services/messaging/rest/uploadedFiles/{fileName}

Files for Download List

https://ws.support.qvalent.com/services/messaging/rest/filesForDownload/

Download/Delete File


6.2 Production Environment

Upload File

https://ws.qvalent.com/services/messaging/rest/uploadedFiles/{fileName}

Files for Download List

https://ws.qvalent.com/services/messaging/rest/filesForDownload/

Download/Delete File

https://ws.qvalent.com/services/messaging/rest/filesForDownload/{fileName}



Appendix D FAQ

Q) I execute the downloadFile.cmd and receive the following error:

Curl: (22) The requested URL returned error: 502

A) The file you are trying to fetch may have a space in its name. Contact Westpac and

ask them to check that there is no space in the filenames being fetched.

https file transfer guide

Documents