corporate data warehouse file extract specification chad ... · nursing data mart. the signed off...

23
NSS IT: File Extract Specification Version 1.2 Page 1 of 23 © Common Services Agency, NHS Scotland National Services Scotland Architecture & Solutions Service NSS Information Technology Corporate Data Warehouse File Extract Specification CHAD Project District Nursing Data Mart

Upload: others

Post on 09-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 1 of 23 © Common Services Agency, NHS Scotland

National Services Scotland Architecture & Solutions Service

NSS Information Technology

Corporate Data Warehouse

File Extract Specification

CHAD Project

District Nursing Data Mart

Page 2: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 2 of 23 © Common Services Agency, NHS Scotland

Document Control

Version Date Author Description

0.1 02/06/2015 Angela Forbes Initial draft

0.2 09/06/2015 Angela Forbes

Other Aim of Care (4) and Other Intervention (4) added and

validation updated accordingly.

0.3 07/07/2015 Angela Forbes Contact Start Date changed to Contact Date.

0.4 10/07/2015 Angela Forbes Denise added to Business Contacts

1.0 16/07/2015 Angela Forbes For sign off

1.0 28/07/2015 Angela Forbes Signed off by Margaret P in Lee’s absence

1.1 15/12/2015 Angela Forbes

Note added that free text fields will not include confidential

information

1.2 13/01/2016 Angela Forbes

Travel Time in Contacts file format changed to varchar.

Validation rules VE15 and VE16 updated.

Supporting Documentation

Document Owner

Business User Requirements Business

Reference Information NSS IT

Functional Requirements NSS IT

Additional Information NSS IT

Help Text Document NSS IT

Corporate Data Warehouse Security NSS IT

Business Contacts

Name Role Email Telephone

Elaine McNish Data Manager [email protected] 0141 300 1926

Margaret Parsons Information Manager [email protected] 0131 275 6528

Margaret Quinn Principal Information Development

Manager

[email protected] 0131 275 6695

Lee Davies Service Manager [email protected] 0131 275 6193

Denise Hastie Senior Information Analyst [email protected] 0131 275 6368

NSS IT Contacts

Name Role Email Telephone

Irene Gow Project Manager [email protected] 0131 275 6209

Angela Forbes Analyst [email protected] 0131 275 6275

Neil Sinclair Technical Lead [email protected] 0131 275 6952

Jill Smith Head of Business Intelligence & [email protected] 0131 275 6788

Page 3: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 3 of 23 © Common Services Agency, NHS Scotland

Corporate Applications

Page 4: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 4 of 23 © Common Services Agency, NHS Scotland

Glossary

Acronym/Word Description

BI Business Intelligence.

BOXI Business Objects XI is a web-based reporting tool that allows users to create

reports which can be made available to other users who have the appropriate

access permissions.

BOXI Universe The BOXI universe is the reporting environment which is built on and allows

access to the associated data mart.

Case A case is used to identify a patient’s single episode with the nursing service.

Within a single case a patient may have multiple contacts with health care

professionals.

CDD Common Dimensions Database – the database which holds all of the common

dimension tables used by the data marts within the Corporate Data Warehouse.

CDW The Corporate Data Warehouse is a central data store maintained by NSS IT. It

is made up of a collection of independent data marts which share a common

database for reference information.

CHI Community Health Index number – a unique patient identifier which is issued to

residents in Scotland when they register with NHS Scotland.

Contact An interaction between the patient and the District Nurse during an episode of

care.

Data mart A subset of a data warehouse which has been designed to satisfy a particular

function or reporting area within an organisation. A data mart can be designed

and developed independently of other data marts. A data warehouse will contain

multiple data marts.

Episode The period of care whilst the patient is part of the District Nurse’s caseload.

ETL Extract, Transform and Load - The process of extracting data from external

sources, transforming the data if necessary to meet operational needs and

loading it into the data warehouse.

Fact A fact is a table in a data mart which contains data, usually measures, which can

be reported on. The measures can usually (but not always) be aggregated at a

dimensional level, e.g. a count of patients can be aggregated to give the counts

per location, organisation etc.

SWIFT Submission with internet file transfer

Page 5: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 5 of 23 © Common Services Agency, NHS Scotland

Contents

1. Introduction ............................................................................................................................................................ 6

1.1. Purpose ........................................................................................................................................................ 6

1.2. Overview....................................................................................................................................................... 6

1.3. General Data Assumptions .......................................................................................................................... 6

2. File Extracts ........................................................................................................................................................... 7

3. File Submission ..................................................................................................................................................... 7

4. File Processing ...................................................................................................................................................... 7

4.1. File Processing Schedule ............................................................................................................................. 8

5. Validation ............................................................................................................................................................... 8

5.1. Accessing Validation Results ....................................................................................................................... 8

6. Automated Emails .................................................................................................................................................. 8

7. File Format ............................................................................................................................................................. 9

7.1. File Name ..................................................................................................................................................... 9

7.2. Header Record ............................................................................................................................................. 9

7.3. General File Rules ...................................................................................................................................... 10

7.4. Data Element Rules ................................................................................................................................... 11

7.5. Episode File Content .................................................................................................................................. 11

7.6. Contact File Content ................................................................................................................................... 13

8. Appendix A – Data Validation Specification ........................................................................................................ 16

9. Appendix A – Sending Locations ......................................................................................................................... 23

Page 6: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 6 of 23 © Common Services Agency, NHS Scotland

1. Introduction

1.1. Purpose

The purpose of this document is to detail the file schemas and processing rules for files submitted to a District

Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the

BI team to develop the data mart as specified.

This document is intended to give details of the file specifications and how the files will be processed into the data

mart. It is not intended to provide recording guidance.

1.2. Overview

District nursing data will be submitted by the NHS Boards and will be held in the Corporate Data Warehouse in a

District Nursing data mart. This will be delivered as part of the first phase of the Community Health Activity Data

(CHAD) project.

1.3. General Data Assumptions

The following general assumptions have been made about the data being submitted

All files will be sent from a single location within a NHS Board, i.e. multiple sites will not be responsible for

submitting information for the NHS Board.

The NHS Board code used in the header record to identify which NHS Board has submitted the data will be

included as a data item in the data mart to identify the NHS Board of Treatment.

A patient can have more than one episode or contact.

An episode can have zero or many contacts.

A contact does not have to be associated with an episode.

There is no interdependency between the files, e.g. a submission does not need to contain both an episode

and a contact record for the patient.

Free text fields will not include confidential information.

Unique Record ID

It must be possible to uniquely identify an episode and contact record to avoid duplicate records being

processed and to allow records to be updated if required.

All NHS Boards will be able to provide a system generated Unique Record ID

o The Episode ID will uniquely identify an episode within the NHS Board. The Episode ID will be fixed

for a given episode and will not differ between the original episode and any update to that episode.

This will allow Episode information to be updated.

o The Contact ID will uniquely identify a contact within the NHS Board. The Contact ID will be fixed

for a given contact and will not differ between the original contact and any update to that contact.

This will allow Contact information to be updated.

Page 7: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 7 of 23 © Common Services Agency, NHS Scotland

2. File Extracts

Extract Type Description Number of

Records per file

EPISODE The file will contain episode records which have been added or

updated during the previous quarter.

An episode starts when a patient is added to a District Nurse’s

caseload and ends when the patient is discharged from the

caseload.

50,000 per quarter

CONTACT The file will contain contact records which have been added or

updated during the previous quarter.

A contact is an interaction between the patient and District Nurse

during an episode of care. An episode can have multiple contacts.

200,000 per quarter

NOTE

1. There is no requirement for a separate historic data load. Any historic data submissions will conform to this

specification.

3. File Submission

The files will be submitted using SWIFT.

SWIFT is currently being used to submit test files. The destination location of the SWIFT account will be modified so

that files are transferred to the source directory for the data mart.

4. File Processing

The files will be processed using an insert and update process.

The following logic will be applied

If a record is submitted and the Unique Record ID does not exist in the data table the record will be

inserted/added to the data table.

If a record is submitted and the Unique Record ID already exists in the data table it will overwrite the record

already held in full - it is not a selective update so the update record must contain all of the relevant

information.

It is important that the latest record submitted for a Unique Record ID contains all of the relevant information.

Records can be submitted or resubmitted from any historic timeframe if required.

Records cannot be automatically deleted. If a record needs to be deleted a fast track CR will be raised to

have the record removed.

Page 8: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 8 of 23 © Common Services Agency, NHS Scotland

4.1. File Processing Schedule

The automated file load process will run at 6pm on weekdays. The file polling will increase 6 weeks after the end of

a Quarter to run three times daily at 11am, 3pm and 6pm for 2 weeks only.

5. Validation

The validation rules are specified in Appendix A.

If any records fail a validation rule the file will fail and the error must be corrected and the file resubmitted.

The file will go through two stages of validation as detailed in the table below. For the data validation to begin the

file must pass the initial file validation stage.

Validation

Stage

Error Type Description Outcome of Validation Rule Failure

File File Name

Header

Schema

The format of the file is validated, e.g.

file name, header record, number of

columns.

The file will fail. The error must be

corrected by the NHS Board and the file

resubmitted.

Data Data The content of the file is validated.

The record will fail. The error must be

corrected by the NHS Board and the

record resubmitted.

5.1. Accessing Validation Results

Validation results will be accessed using Business Objects.

6. Automated Emails

The following emails will be generated. The emails can be sent to multiple recipients.

Email Description Recipient

Passed Contains details of the file submitted and that it

passed validation.

NHS Board (email address(s) to be

provided).

CHAD support mailbox

[email protected]

ISD Data Management mailbox

[email protected]

Failed Contains details of the file submitted.

The number of records processed.

The number of records which passed validation.

The number of records which failed validation.

If the file has failed totally it will include an

NHS Board (email address(s) to be

provided).

CHAD support mailbox

[email protected]

Page 9: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 9 of 23 © Common Services Agency, NHS Scotland

Email Description Recipient

indication of why the file could not be processed.

ISD Data Management mailbox

[email protected]

Loaded The data will not be available immediately after

validation because of the processing required.

This email will advise when the data are

available for analysis.

NHS Board (email address(s) to be

provided).

CHAD support mailbox

[email protected]

ISD Data Management mailbox

[email protected]

7. File Format

The data extract(s) will follow the general standards.

7.1. File Name

The file will fail validation if the file name does not conform to the following format.

DATAMART_EXTRACT TYPE_SENDING LOCATION_EXTRACT DATE. EXTENSION

e.g. DNURSE_EPISODE_N_20150114.csv

Where:

Item Description Example

Data Mart Identifies the data mart which the files are submitted to DNURSE

Extract Type The type of file extract: Episode or Contact EPISODE

Sending Location The cipher for the sending location e.g. N = Grampian

See Appendix A for valid sending locations.

N

Extract Date The date the file was compiled. Date format: YYYYMMDD 20150114

File extension Identifies the type of file, must be a .csv .csv

7.2. Header Record

The first row in the file must contain a header record which contains details of the file. All data items in the header

record are mandatory and must be in the specified format. The information in the header record should be

consistent with the information supplied in the file name.

The file will fail validation if the header format does not conform to the following comma delimited format.

DATA MART,EXTRACT TYPE,SENDING LOCATION,EXTRACT DATE,SUBMISSION REFERENCE

NUMBER,RECORD COUNT

e.g. DNURSE,EPISODE,N,20150114,1,12100

Page 10: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 10 of 23 © Common Services Agency, NHS Scotland

Where:

Item Format Description Example

Data Mart Varchar (6) Identifies the data mart which the files are

submitted to

DNURSE

Extract Type Varchar (15) The type of file extract: Episode or Contact EPISODE

Sending Location Varchar (1) The cipher for the sending NHS Board e.g. N =

Grampian.

See Appendix B for valid sending locations.

N

Extract Date Varchar (8) The date the file was compiled. Date format:

YYYYMMDD

20150114

Submission

Reference Number

Integer (6) A unique number identifying the number of the

submission for the sending location.

1

Record Count Integer A count of the number of records in the file

(excluding the header record and column names).

12100

NOTE

1. The header record is not case sensitive.

2. The Submission Reference Number is required to allow multiple files for the same location to be processed in the

correct order, and to flag if there are any gaps in the received files. The Submission Reference Number should

increment per successful validation for each sending location. Files which are invalid and rejected for any reason

must be corrected and resubmitted with the same Submission Reference Number as the original.

3. There should be no characters (including commas) after the last item.

7.3. General File Rules

The following table include the general rules which apply to the file format

Category Description

Columns The columns are comma delimited to separate fields and must be in the order specified

in the file specification. If a field is empty the column must still be included in the

extract.

Column Names The file will contain column name headings. The headings are for local use only and

will be ignored when the file is loaded. The name can only contain a comma when

surrounded by double quotes.

Column Width The column width is the minimum width from a development perspective. The data

mart will be developed to process columns with a width wider than documented to

ensure that the field is not truncated to a valid value.

Blank Rows Blank data rows must not be included in the data file.

Null/Missing Values Empty fields will indicate where a value is null (missing). Zero must not be used.

Spaces (non-

printable values)

Spaces should not be present between fields. The comma alone should delimit fields.

Page 11: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 11 of 23 © Common Services Agency, NHS Scotland

7.4. Data Element Rules

The following table includes the data element rules which apply to the data in the file

Category Description

Number Type Numeric data items will appear in the text file as a string of continuous numeric digits.

For large numbers the ‘thousand’ display format characters such as the comma will not

be included. Where the number contains decimals, the period character ‘.’ will be used

as the decimal point.

Text/Character Type The text string will contain any alphanumeric character including the space character

other than double quotes. The format of the text field will be defined as ‘Varchar (n)’

where Varchar (n) is a variable length string of maximum length ‘n’.

Free text fields should be qualified/surrounded by double quotes.

Codes and descriptions will be converted to upper case for all data items.

Date Type A column containing date strings will be formatted as DDMMCCYY (e.g. 14012015).

If a 7 digit date is submitted it will be left zero padded to 8 digits.

Time Type A column containing date time strings will be formatted as HH:MM using the 24 hour

clock (e.g. 17:35).

Leading Zeros Leading zeros must not be stripped off codes which have been identified as having

leading zeros.

If a 9 digit CHI number is submitted it will be left zero padded to 10 digits.

Data Item Priorities The priority categories for including data items are as follows.

M = Mandatory the data item must be populated

S = Should be populated if available

O = Optional

7.5. Episode File Content

The table below gives details of each data item in order it will appear in the Episode file.

Data Item Description Format Priority

Episode ID Identifier used within the NHS Board to uniquely identify an

episode.

Varchar

(100)

M

CHI Number The identifier used by NHS Scotland to uniquely identify a patient.

9 digit codes will be left zero padded to 10 digits.

Varchar (10) S

Surname The surname of the patient.

Required for CHI seeding.

Varchar (35) S

Page 12: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 12 of 23 © Common Services Agency, NHS Scotland

Must be provided if a valid CHI Number is not provided.

Forename The first forename of the patient.

Required for CHI seeding.

Must be provided if a valid CHI Number is not provided.

Varchar (35) S

Date of Birth The patient’s full date of birth.

Required for CHI seeding and will be used to derive age where a

valid CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Date S

Postcode The patient’s postcode of residence.

Required for CHI seeding and will be used to derive the area of

residence where a valid CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Varchar (8) S

Gender The gender of the patient, e.g. 1 = Male, 2 = Female (see the

reference information document for valid codes).

Required for CHI seeding and to populate gender where a valid

CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Varchar (1) S

Ethnicity The client’s ethnic group (see the reference information document

for valid codes).

Varchar (2) S

Date Referral

Received

The date the healthcare service receives a referral (not the date in

the referral letter).

The date could be after the ‘Contact Date’.

Date

S

Source of Referral The source of referral code. Varchar (1) S

Main Aim of Care

A desired achievement of a programme of care e.g. the primary

reason for a referral to a Community Health Care Professional.

Varchar (1) S

Other Aim of Care

(1)

Other desired achievement of a programme of care, but not

deemed to be the primary desired achievement.

Varchar (1) O

Other Aim of Care

(2)

Other desired achievement of a programme of care, but not

deemed to be the primary desired achievement.

Varchar (1) O

Other Aim of Care

(3)

Other desired achievement of a programme of care, but not

deemed to be the primary desired achievement.

Varchar (1) O

Other Aim of Care

(4)

Other desired achievement of a programme of care, but not

deemed to be the primary desired achievement.

Varchar (1) O

Episode Start The date care began for the episode. Date S

Page 13: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 13 of 23 © Common Services Agency, NHS Scotland

Date

Episode End Date The date care finished for the episode.

Date S

Outcome of

Episode

The reason the patient was discharged from the episode of care.

Varchar (1) S

NOTE

1. The recording guidance states that CHI number is mandatory however if it cannot be provided the demographic

information must be submitted and this will be used for CHI seeding.

2. 7 digit dates will be left zero padded to 8 digits.

3. A combination of the Sending Location and Episode ID will uniquely identify a record in the data mart.

4. Episode Start Date is not mandatory because it is likely that initially the episode data will be less complete than

the contact data. As the episode data completeness improves a change request may be raised to make the Episode

Start Date mandatory.

7.6. Contact File Content

The table below gives details of each data item in order it will appear in the Contact file.

Data Item Description Format Priority

Contact ID Identifier used within the NHS Board to uniquely identify a contact. Varchar

(100)

M

Episode ID Identifier used within the NHS Board to uniquely identify an

episode.

Integer (100) S

CHI Number The identifier used by NHS Scotland to uniquely identify a patient.

9 digit codes will be left zero padded to 10 digits.

Varchar (10) S

Surname The surname of the patient.

Required for CHI seeding.

Must be provided if a valid CHI Number is not provided.

Varchar (35) S

Forename The first forename of the patient.

Required for CHI seeding.

Must be provided if a valid CHI Number is not provided.

Varchar (35) S

Date of Birth The patient’s full date of birth.

Required for CHI seeding and will be used to derive age where a

valid CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Date S

Page 14: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 14 of 23 © Common Services Agency, NHS Scotland

Postcode The patient’s postcode of residence.

Required for CHI seeding and will be used to derive the area of

residence where a valid CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Varchar (8) S

Gender The gender of the patient, e.g. 1 = Male, 2 = Female (see the

reference information document for valid codes).

Required for CHI seeding and to populate gender where a valid

CHI number is not available.

Must be provided if a valid CHI Number is not provided.

Varchar (1) S

Ethnicity The client’s ethnic group (see the reference information document

for valid codes).

Varchar (2) S

Contact Date The date the contact between the Healthcare Professional and the

patient began within an Episode of Care.

Date M

Contact Start

Time

The time the contact between the Healthcare Professional and the

patient began.

Time S

Contact End Time The time the contact between the Healthcare Professional and the

patient ended.

Time S

Duration of

Contact

The length of time contact occurred between the Healthcare

Professional and the patient in minutes.

Must be completed if Contact Start Time and Contact End Time

are not recorded.

Varchar (5) S

Location of

Contact

The type of location where the contact occurred, e.g. Health

Centre.

Varchar (1) S

Patient Contact

Category

The category of patient contact, e.g. direct, indirect or other.

Varchar (1) S

Planned/Unplann

ed

Indicates if the contact with the patient was planned or unplanned.

Varchar (1) S

Joint Contact Indicates if two or more staff members were required for the

patient contact.

The value can be 0 or 1 where 1 indicates that joint contact was

required.

Varchar (1) S

Number of staff

present in joint

contact

The number of staff members in attendance during a joint visit. Varchar (1) S

Visit Status Indicates if the patient attended/was seen. Varchar (1) S

Patient Related

Activity Type

Other patient related activities undertaken by the Healthcare

Professional as a result of the contact.

Varchar (1) S

Page 15: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 15 of 23 © Common Services Agency, NHS Scotland

Duration of

Patient Related

Activity

The number of minutes taken by staff to carry out any patient

related activities which serve to enhance the delivery of care.

Varchar (5)

S

Travel Time The number of minutes taken to travel by the staff member to the

location of contact.

Varchar (5) S

Staff Pay Band The current staff band as featured on staff payslip.

Varchar (2) S

Service Team The name of the Service Team.

Free text

Varchar

(100)

S

Primary

Intervention

An activity which is undertaken to maintain or potentially improve

the patient’s state of health and well-being, relieve distress or

reduce risk.

Varchar (8) S

Other Intervention

(1)

Additional activity other than the main activity s undertaken to

maintain or potentially improve the patient's state of health and

well-being, relieve distress or reduce risk.

Varchar (8) O

Other Intervention

(2)

Additional activity other than the main activity s undertaken to

maintain or potentially improve the patient's state of health and

well-being, relieve distress or reduce risk.

Varchar (8) O

Other Intervention

(3)

Additional activity other than the main activity s undertaken to

maintain or potentially improve the patient's state of health and

well-being, relieve distress or reduce risk.

Varchar (8) O

Other Intervention

(4)

Additional activity other than the main activity s undertaken to

maintain or potentially improve the patient's state of health and

well-being, relieve distress or reduce risk.

Varchar (8) O

NOTE

1. Demographic information and episode specific information is included in the Contact file because not all NHS

Boards will be able to provide the Episode file initially.

2. Contact End Date is not included and it will be assumed that the contact will start and end on the same date even

though some visits could straddle midnight.

3. 7 digit dates will be left zero padded to 8 digits.

4. A combination of the Sending Location and Contact ID will uniquely identify a record in the data mart.

Page 16: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 16 of 23 © Common Services Agency, NHS Scotland

8. Appendix A – Data Validation Specification

The table below includes details of the File Validation and Data Validation Errors.

The File Extract Type ALL refers to both the Episode and Contact File Type.

If a validation rules needs to be turned off a Change Request will be raised.

File Extract

Type

Error

Code

Error Type Error

Severity

Error Message Additional Information

ALL FE1 File Name Error Invalid File Name: Data mart is

missing or not recognised

Must be DNURSE

ALL FE2 File Name Error Invalid File Name: File extract

type is missing or not recognised

Must be EPISODE or

CONTACT

ALL FE3 File Name Error Invalid File Name: Sending

location is missing or not

recognised

Must be one of A, B, F, G, H, L,

N, R, S, T, V, W, Y and Z.

ALL FE4 File Name Error Invalid File Name: Extract Date is

missing or invalid. Should be a

date in the format YYYYMMDD

ALL HE1 Header Error Invalid Header Record: Could not

be read – check format

ALL HE2 Header Error Invalid Header Record: Data mart

is missing, not recognised or does

not match the file name

ALL HE3 Header Error Invalid Header Record: File

extract type is missing, not

recognised or does not match the

file name

ALL HE4 Header Error Invalid Header Record: Sending

Location is missing, not

recognised or does not match the

file name

ALL HE5 Header Error Invalid Header Record: Extract

Date is missing, invalid or does

not match the file name. Should

be a date in the format

YYYYMMDD.

ALL HE6 Header Error Invalid Header Record: The

Submission Reference supplied is

out of sequence for the supplied

Sending Location

ALL HE7 Header Error Invalid Header Record: The

Page 17: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 17 of 23 © Common Services Agency, NHS Scotland

number of records is inconsistent

with the number of records

submitted

ALL SE1 Schema Error Data record could not be read –

check file format

EPISODE SE2 Schema Error Episode ID must be provided and

must be unique within the file

CONTACT SE3 Schema Error Contact ID must be provided and

must be unique within the file

ALL VE1 Data Error CHI number must be valid Conditional check only applied if

CHI number has been provided.

9-digit codes will be left zero

padded to 10 digits.

Must exist on the relevant

common dimension.

ALL VE2 Data Error Surname must be provided and in

the correct format

Conditional check only applied if

CHI number has not been

provided.

The following criteria must be

met

The first character must be

alphabetic.

Can contain alphabetic

characters, a hyphen, a

space or an apostrophe.

Must be a minimum of 2

characters.

Maximum field length 35

ALL VE3 Data Error Forename must be provided and

in the correct format

Conditional check only applied if

CHI number has not been

provided.

The following criteria must be

met

The first character must be

alphabetic.

Can contain alphabetic

characters, a hyphen, a

space or an apostrophe.

Must be a minimum of 2

characters.

Page 18: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 18 of 23 © Common Services Agency, NHS Scotland

Maximum field length 35

ALL VE4 Data Error Date of Birth must be provided

and must be a valid date in the

format DDMMCCYY

Conditional check only applied if

CHI number has not been

provided.

Must exist on the relevant

common dimension.

ALL VE5 Data Error Postcode must be provided and

must be valid

Conditional check only applied if

CHI number has not been

provided.

Must exist on the relevant

common dimension.

ALL VE6 Data Error Gender must be provided and

must be valid

Conditional check only applied if

CHI number has not been

provided.

Must exist on the relevant

common dimension.

ALL VE7 Data Error Ethnicity must be valid Conditional check only applied if

Ethnicity has been provided.

Must exist on the relevant

common dimension.

EPISODE VE8 Data Error Date Referral Received must be a

valid date in the format

DDMMCCYY

Conditional check only applied if

Date Referral Received has

been provided.

Must exist on the relevant

common dimension.

EPISODE VE9 Data Error Source of Referral must be valid Conditional check only applied if

Source of Referral has been

provided.

Must exist on the relevant

common dimension.

Must not be 0 (Community

Health Service) – 0 is not valid

for this dataset.

EPISODE VE10 Data Error Main Aim of Care must be valid Conditional check only applied if

Main Aim of Care has been

provided.

Page 19: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 19 of 23 © Common Services Agency, NHS Scotland

Must exist on the relevant

common dimension.

EPISODE VE11 Data Error Other Aim of Care (1) must be

valid

Conditional check only applied if

Other Aim of Care (1) has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE12 Data Error Other Aim of Care (2) must be

valid

Conditional check only applied if

Other Aim of Care (2) has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE13 Data Error Other Aim of Care (3) must be

valid

Conditional check only applied if

Other Aim of Care (3) has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE14 Data Error Other Aim of Care (4) must be

valid

Conditional check only applied if

Other Aim of Care (4) has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE15 Data Error Episode Start Date must be a

valid date in the format

DDMMCCYY

Conditional check only applied if

Episode Start Date has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE16 Data Error Episode End Date must be a valid

date in the format DDMMCCYY

Conditional check only applied if

Episode End Date has been

provided.

Must exist on the relevant

common dimension.

EPISODE VE17 Data Error Outcome of Episode must be

valid

Conditional check only applied if

Outcome of Episode has been

provided.

Must exist on the relevant

common dimension.

Page 20: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 20 of 23 © Common Services Agency, NHS Scotland

CONTACT VE18 Data Error Contact Date must be provided

and must be a valid date in the

format DDMMCCYY

Contact Date is mandatory.

Must exist on the relevant

common dimension.

CONTACT VE19 Data Error Contact Start Time must be a

valid time in the format HH:MM

Conditional check only applied if

Contact Start Time has been

provided.

Must exist on the relevant

common dimension.

CONTACT VE20 Data Error Contact End Time must be a valid

time in the format HH:MM

Conditional check only applied if

Contact End Time has been

provided.

Must exist on the relevant

common dimension.

CONTACT VE21 Data Error Duration of Contact must be

numeric

Conditional check only applied if

Duration of Contact has been

provided.

CONTACT VE22 Data Error Location of Contact must be valid Conditional check only applied if

Location of Contact has been

provided.

Must exist on the relevant

common dimension.

CONTACT VE23 Data Error Patient Contact Category must be

valid

Conditional check only applied if

Patient Contact Category has

been provided.

Must exist on the relevant

common dimension.

CONTACT VE24 Data Error Planned/Unplanned must be valid Conditional check only applied if

Planned/Unplanned has been

provided.

Must exist on the relevant

common dimension.

CONTACT VE25 Data Error Joint Contact must be 0 or 1 Conditional check only applied if

Joint Contact has been

provided.

CONTACT VE26 Data Error Number of Staff Present in Joint

Contact must be numeric

Conditional check only applied if

Number of Staff Present in Joint

Contact has been provided.

Page 21: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 21 of 23 © Common Services Agency, NHS Scotland

Must be a whole number.

CONTACT VE27 Data Error Visit Status must be valid Conditional check only applied if

Visit Status has been provided.

Must exist on the relevant

common dimension.

CONTACT VE28 Data Error Patient Related Activity Type

must be valid

Conditional check only applied if

Patient Related Activity Type

has been provided.

Must exist on the relevant

common dimension.

CONTACT VE29 Data Error Duration of Patient Related

Activity must be numeric

Conditional check only applied if

Duration of Patient Related

Activity has been provided.

CONTACT VE30 Data Error Travel Time must be numeric Conditional check only applied if

Travel Time has been provided.

CONTACT VE31 Data Error Staff Pay Band must be valid Conditional check only applied if

Staff Pay Band has been

provided.

The code supplied will be

mapped to the relevant common

dimension therefore must exist

in the mapping specified.

CONTACT VE32 Data Error Primary Intervention must be valid Conditional check only applied if

Primary Intervention has been

provided.

Can be Category or

Subcategory code.

Must exist on the relevant

common dimension.

CONTACT VE33 Data Error Other Intervention (1) must be

valid

Conditional check only applied if

Other Intervention (1) has been

provided.

Can be Category or

Subcategory code.

Must exist on the relevant

common dimension.

CONTACT VE34 Data Error Other Intervention (2) must be Conditional check only applied if

Page 22: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 22 of 23 © Common Services Agency, NHS Scotland

valid Other Intervention (2) has been

provided.

Can be Category or

Subcategory code.

Must exist on the relevant

common dimension.

CONTACT VE35 Data Error Other Intervention (3) must be

valid

Conditional check only applied if

Other Intervention (3) has been

provided.

Can be Category or

Subcategory code.

Must exist on the relevant

common dimension.

CONTACT VE36 Data Error Other Intervention (4) must be

valid

Conditional check only applied if

Other Intervention (4) has been

provided.

Can be Category or

Subcategory code.

Must exist on the relevant

common dimension.

Page 23: Corporate Data Warehouse File Extract Specification CHAD ... · Nursing data mart. The signed off document will be used in conjunction with other supporting documentation by the BI

NSS IT: File Extract Specification

Version 1.2 Page 23 of 23 © Common Services Agency, NHS Scotland

9. Appendix A – Sending Locations

NHS Board Sending Location Code

Ayrshire & Arran A

Borders B

Dumfries & Galloway Y

Fife F

Forth Valley V

Grampian N

Greater Glasgow and Clyde G

Highland H

Lanarkshire L

Lothian S

Orkney R

Shetland Z

Tayside T

Western Isles W