test strategy for the november 09 bsc systems release

TEST STRATEGY for the November 09 BSC Systems Release v.1.0 of 26 © ELEXON Limited 2009

TEST STRATEGY for the November 09 BSC Systems Release

Prepared by Change Implementation Team

For Use Date of Issue Version Number 1.0

For Attention Of BSC Parties and other interested parties

Overview or Purpose of Document:

Purpose

The purpose of this document is to define the testing to be undertaken for the changes included in the November 09 Release. These changes are defined in the relevant Business Requirements Solution documents and Change Proposals (References 3, 4, 5 and 6).

This document is produced in accordance with the Change Delivery Guidelines for Testing Changes to BSC Systems (Reference 1).

Contact Shamaila Jawaid * [email protected] ( 0207 380 4318


Intellectual Property Rights, Copyright and DisclaimerThe copyright and other intellectual property rights in this document are vested in ELEXON or appear with the consent of the copyright owner. These materials are made available for you for the purposes of your participation in the electricity industry. If you have an interest in the electricity industry, you may view, download, copy, distribute, modify, transmit, publish, sell or create derivative works (in whatever format) from this document or in other cases use for personal academic or other non-commercial purposes. All copyright and other proprietary notices contained in the document must be retained on any copy you make.

All other rights of the copyright owner not expressly dealt with above are reserved.

No representation, warranty or guarantee is made that the information in this document is accurate or complete. While care is taken in the collection and provision of this information, ELEXON Limited shall not be liable for any errors, omissions, misstatements or mistakes in any information or damages resulting from the use of this information or action taken in reliance on it.

Contents

Purpose.............................................................................................................................1

1 Introduction..........................................................................................................31.1 Objective...............................................................................................................31.2 Scope.....................................................................................................................31.3 Testing Required...................................................................................................4

2 Testing Requirements...........................................................................................52.1 Test Activity Involvement.....................................................................................62.2 Test Responsibilities .............................................................................................72.3 Relating Test Types to Test Phases ......................................................................82.4 Entry/Exit Criteria...............................................................................................10

3 P217 Impacted Flows .........................................................................................13

4 Phases of Testing Details....................................................................................144.1 Document Review...............................................................................................144.2 Walkthrough/Page-Turning................................................................................144.3 Preliminary P217 Participant Interface Validation.............................................144.4 ELEXON Witness Testing.....................................................................................154.5 Unit, Module and System Testing .......................................................................154.6 Factory Acceptance Testing................................................................................164.7 Change Specific Testing......................................................................................164.8 Regression Testing..............................................................................................164.9 End-to-End and Participant Testing....................................................................174.10 Parallel Run Testing............................................................................................214.11 Operational Acceptance Testing .........................................................................214.12 Deployment and Backout Testing.......................................................................224.13 Volume and Performance Testing.......................................................................224.14 User Acceptance Testing.....................................................................................22

5 Test Management ...............................................................................................23

6 Assumptions, Risks and Issues...........................................................................24

7 Document Control...............................................................................................24

A Appendix A – Terms used in this Document .......................................................26


1 Introduction

1.1 Objective

The objectives of this test strategy for the November 09 Release (hereafter ‘Nov 09 Release’) are as follows:

• to describe the overall testing framework;

• to define the testing to be undertaken;

• to define the responsibilities of the various parties involved in testing;

• to ensure that all areas of testing requirements are addressed; and

• to ensure that the coverage of the testing is adequate to meet those acceptance criteria that are subject to testing.

This test strategy covers the testing required to provide BSCCo with the assurance that:

• the changes made to the trading arrangements support the requirements and solutions defined in the P217 and P223 Release Business Requirements Solution documents (References 3 and 4)and in the relevant Change Proposals;

• the changes made have not adversely impacted unchanged software systems and documents that support the trading arrangements; and

• the changes made conform to the BSC and Code Subsidiary Documents.

1.2 Scope

The scope of this document is the testing of the changes to (documentation, processes and software systems) deliver the business requirement solutions for the Nov 09 Release, to ensure that all new functionality has been implemented correctly and that unchanged functionality is not adversely impacted.

The Nov 09 Release currently comprises 2 Approved Modifications, 4 Approved Change Proposalsand a further housekeeping Change Proposal, details of which are listed below.

Change Reference

Title & Description Key Impacts

P217 Revised Tagging Process and Calculation of Cash Out Prices

BMRA, SAA, TOMAS, BSC Website

P223 Improvements to the Profile Administrator Service (PrA)

PrA Service Description, BSCP510, internal system for the manipulation of Sample requirements, PrA’s P223 Management Database.

P231 Black Start and Fuel Security BSCP201 (A new BSCP)


Change Reference

Title & Description Key Impacts

Code Procedures under the Balancing and Settlement Code (BSC)

P232 Black Start and Fuel Security Compensation and Single Imbalance Price Derivation

BSCP201 (A new BSCP)

CP1248 Early Release of Meter Technical Details by Non Half Hourly Meter Operator Agent

PARMS

CP1269 Publication of Additional Non Half Hourly Combination Data in Market Domain Data

MDD, MDD File Loader, LLF system

CP1281 Revenue Protection: requiring the NHHDC to send EAC/AA data to the Supplier via the DTC

BSCP504

CP1286 BSCP18 Operational Review: Additional flag in Transmission Company’s BOAL file to indicate an amended Bid-Offer Acceptance

BMRA, SAA

CP1287 Correction of inconsistencies in BSCP536 ‘Supplier Charges’

PARMS

TAI2349 Minor Changes to the Electricity Summary Page on the BMRS

BMRS

CP1294 (Pending Approval)

Housekeeping Change to SVA Data Catalogue Volume 2

SVA Data Catalogue

The detailed requirements and solutions for each change can be found in the P217 and P223Business Requirements Solution documents (Reference 3 and 4) and in the relevant Change Proposals (see Reference section).

1.3 Testing Required

The full details of the scope, approach and deliverables, including timescales and dates for the individual test phases are defined in the PID and Release Plan for November 09 BSC SystemsRelease (Reference 2) and the Exception Plan for November 09 BSC Systems Release (Reference 10).

Changes to BSC Party and Party Agent systems that are impacted by the Nov 09 Release are outside the scope of this test strategy. However, ELEXON will communicate with BSC Parties and Party Agents to ensure they understand the changes required and will offer them the opportunity to take part in testing. The BSC Party and Party Agent systems impacted by the Nov 09 Release


are specified in the Business Requirements Solution documents (References 3 and 4) and relevant CPs.

2 Testing Requirements

The ELEXON Testing Guidelines (Reference 1) have been used to identify the testing that needs to be carried out for the Nov 09 Release. The testing requirements are summarised in the tables below.


2.1 Test Activity Involvement

Change Delivery’s Testing Guidelines (Reference 1) have been used to identify the testing that needs to be carried out for the Release and this is summarised as follows:

Ch

ange

Ref

eren

ce

Bu

sin

ess

Ris

k

Scop

e

Doc

um

enta

tion

R

evie

w

Wal

k-th

rou

gh/P

age-

Turn

ing

Un

it, M

odu

le,

Syst

em T

est

Fact

ory

Acc

epta

nce

(FA

T)

Ch

ange

Spe

cifi

c

Reg

ress

ion

Acc

epta

nce

Tes

t

ELEX

ON

Wit

nes

s Te

stin

g

Par

tici

pan

t /

Serv

ice

Inte

grat

ion

(SI

T)

Par

alle

l Ru

n

Dep

loym

ent

Ope

rati

onal

A

ccep

tan

ce

(OA

T)

Vol

um

e

Per

form

ance

P217 (Central System Changes) High High E/A/B A A A A E N/I/B B B B B B

P217 (TOMAS Changes) High High E/En En En En En E En En En En EnCP1286 (Central System

Changes) Low Low E/A/B/En A A A E N/B B B B B B

TAI2349 (BMRS Change) Low Low E/A/B A A A E B B BP223 (PrA’s P223

Management Database) Low High E/P E/P P P P E I/P P

P223 (ELEXON P223 System) Low High E E E E E E

CP1248 (PARMS Changes) Low Medium E/A/B A A A A E B E

CP1287 (PARMS Changes) Low Low E/A/B A A A A E B E

CP1269 (Central System Changes) Low Low E/A/B A A A A E B B B B

CP1269 (ELEXON System– for MDD Change) Low Low E/En En En En En E En En

P231 (New BSCP) Low Low E/N/I E/N/IP232 (New BSCP) Low Low E/N/I E/N/I


Key to abbreviations: E ELEXON B Logica as Business Process Operator Host I Industry ParticipantsEn Engage P PrAA Logica as Application Management and Developer N National Grid Company

2.2 Test Responsibilities

Test Type

Def

ined

Man

aged

Exec

ute

d

Evid

ence

R

evie

w

Wit

nes

sed

Agr

eed

Documentation Review E E E n/a n/a EWalkthrough/Page-turn E E E/N/I n/a n/a E/N/IUnit/Module/System Testing A/En/P

/EA/En/P/E

A/En/ P/E

E/P n/a E

Factory Acceptance Testing A/EnP/E

A/En/E/P

A/En/E/P E/P n/a E

Change Specific Testing A/En/P A/En/P A/En/P E n/a EAcceptance Testing E/P E/P E/P E n/a EELEXON Witness Testing E E E E E EDeployment Testing E/P E/P E/B/En/P E n/a EPerformance Testing B/En B/En B/En E n/a EVolume Testing B/En B/En B/En E n/a EParallel Run Testing B/En B/En B/En E n/a EParticipant Testing B/En/E B/En B/En/P E n/a E


2.3 Relating Test Types to Test Phases

P217 Test Phase P217 Test Types Participant

Unit and Module Testing Unit, Module and System Testing Logica (AM/Dev)

Formal System Testing Factory Acceptance Testing (consisting of

Change Specific Testing

Regression TestingDeployment and Backout Testing)

Logica (AM/Dev)

BPO Testing Operational Acceptance TestingE2E TestingParallel Run TestingPerformance TestingVolume Testing

Logica (BPO)

Participant Testing Participant Testing Logica (BPO), ELEXON, Industry, National Grid

Witness Testing Witness Testing ELEXON

P223 (internal system)Test Phase P223 Test Types Participant

Unit and Module Testing Unit, Module and System Testing ELEXON

Formal System Testing Factory Acceptance Testing ELEXON

User Acceptance Testing Acceptance Testing of internal P223 system

ELEXON

Deployment Testing Deployment of internal P223 system ELEXON


P223 (P223 Management Database)

Test PhaseP223 Test Types Participant

Unit and Module Testing Unit, Module and System Testing PrA

Formal System Testing Change Specific Testing PrA

User Acceptance Testing Acceptance Testing of P223 Management Database

PrA

Deployment Testing Deployment of P223 Management Database

PrA

Participant Testing Participant Testing of information exchanged between ELEXON, PrA and Suppliers.

ELEXON, PrA, Suppliers

Witness Testing Witness Testing of PrA’s new processes and Testing.

ELEXON

CP1248 Test Phase CP1248 Test Types Participant


Formal System Testing Factory Acceptance Testing Logica (AM/Dev)

User Acceptance Testing Acceptance Testing ELEXON

Deployment Testing Deployment Testing Logica (BPO)



Formal System Testing Factory Acceptance Testing Logica (AM/Dev)

MDD Acceptance Logica (BPO)User Acceptance Testing

MDD File Loader Acceptance ELEXON



MDD Deployment Testing Logica (BPO)Deployment Testing

MDD File Loader Deployment Testing Engage

2.4 Entry/Exit Criteria

A test phase will be deemed to have been completed successfully:

• when all high and medium severity defects have been fixed and re-tested; and

• when low severity defects have either been fixed or a plan in place to address them has been agreed by ELEXON.

Test Phase Entry Criteria Exit Criteria

Unit, Module and System Testing All Design Documentation relating to the change is complete.

Unit, Module and System Testinghas been completed successfully.

When the system works in accordance with the Module Specifications.

Unit and Module test evidence has been compiled for internal Logica review and a plan to resolve outstanding defects before the start of Formal System Testing has been communicated to ELEXON.

Formal System Testing Unit and Module testing has been completed successfully.

Module Testing has completed successfully for the previously

No Severity 1 or Severity 2 Defects are outstanding.

All System Tests in Formal System Testing have been completed



existing defect fixes.

A test environment has been fully prepared.

All System Test Scripts have been prepared (including expected results).

successfully.

When the system works in accordance with the relevant User Requirements Specification (URS), System Specification (SS) and Design Specification (DS).

All defects have been logged on the BSC Services Helpdesk and a plan is agreed and Accepted by ELEXON to resolve any outstanding defects.

A Test Report has been produced by Logica/Engage and reviewed by ELEXON.

The results of Witness Testing have been recorded by ELEXON in a Witness Test Report and this report has been reviewed

BPO Testing The BPO has received the developed software anddocumentation from the AM/Dev.

A test environment has been fully prepared.

No Severity 1 or Severity 2 Defects are outstanding.

All System Tests in BPO have been completed successfully.


The results of Witness Testing have been recorded by ELEXON in a Witness Test Report and this report has been reviewed by



Logica

A Test Report has been produced by Logica and reviewed by ELEXON.

Participant Testing Test Participants (including National Grid Company) have been selected and organised by ELEXON.

Test Participants, ELEXON and Logica are all aware of the times and content of the test flows that will be sent.

All BPO Testing has been completed.

A Test Report has been produced by Logica and reviewed by ELEXON.

The test flows conform fully to the IDDs, test flows are received by Participants and are in accordance with what was sent.


User Acceptance Testing The Factory Acceptance Test phase has been successfully completed.

Test scripts have been followed.

No major defects outstanding.Deployment Testing UAT has been successfully

completedNew software and systems have been successfully deployed without any defects

Witness Testing Design documents and test materials have been agreed with ELEXON and development has been completed.

Successful completion of test activity with no major defects outstanding.


3 P217 Impacted Flows

SAA

National Grid

BSCCo / TOMAS

BMRA

BSC Parties

Non-BSC Parties

BMRA-I002 – BM Data BMRA-I007 (BM Data)

SAA-I014 sub-flow 2

SAA-I014 sub-flow 1 (and can download sub-flow 2 if required)

FPNs

Accepted Bids/Offers BMRA-I014 – Price

Adjustment Data

SAA-I026 – Price Adjustment DataSAA-I014 sub-flow 2 and 3

NETBSAD and DISBSAD

BMRA-I004 – Publish BM DataBMRA-I005 – Publish System Related DataBMRA-I006 (Publish Derived Data)

BSC Website

Best ViewSystem Prices

SAA-I014 sub-flow 2

SAA-I023 (system parameters)

BMRA-I012 (System Parameters))System

All flows coloured in green and red are flows which are amended as part of P217.


4 Phases of Testing Details

Note that this test strategy should be read in conjunction with Logica, IMServ and EngageNovember 09 Release Test Strategies (References 11, 12 and 13).

4.1 Document Review

All documentation produced by the Release, including Project Management deliverables such as this document (which are not otherwise included in the scope of this document), will be subject to review cycles carried out in accordance with the Quality Manual (Reference 8).

All documents indicated as being impacted by P217, P223, P231, P232 and relevant CPs in the Nov09 Release Impact Matrix (Reference 9) will be drafted, and subsequently reviewed, prior to the commencement of development work. All amended documentation will be given an interim version number, and will be reviewed by the ELEXON Release Team on an ongoing basis.

For P231 and P232, in particular, a new BSCP (BSCP201) will be drafted. This will be reviewed by both ELEXON and the industry.

4.2 Walkthrough/Page-Turning

4.2.1 Purpose

The purpose of the Page-Turning and Walkthrough is to ensure that manual business processes and flows affected by the Modifications and Change Proposals in the Nov 09 Release are workable and compliant.

4.2.2 Test Scope

The changes to Documentation will be verified by the Release Team using a walkthrough following a page-turning workshop to ensure that the processes are clear, correct and valid. The Release Team will carry out the walkthrough using the amended documentation after completion of ELEXON review cycles.

For P223, a walkthrough of BSCP510 changes is required. No walkthrough is required for the changes to the PrA Service Description.

For P231 and P232, a walkthrough of the new BSCP201 is required and will focus on two processes detailed in the new BSCP; the market recovery process and the application for compensation process. The walkthrough will be conducted with industry participants, preferably including participants involved in generation.


4.3 Preliminary P217 Participant Interface Validation

4.3.1 Purpose and Scope

Prior to the commencement of any formal testing phase, Logica, Engage and National Grid will carry out additional independent testing of their flows. This preliminary testing will allow the test participants to ensure that the formats of the flows and files that pass between their interfaces are correct and consistent with their system design.

4.4 ELEXON Witness Testing

4.4.1 Purpose

The purpose of Witness Testing is to gain assurance that the tests proposed in the Service Provider’s Test Strategy, Specifications and Scripts are being carried out fully and rigorously.

4.4.2 Test Scope

The ELEXON Release Team may witness any of the testing carried out as part of the Nov 09 changes. ELEXON will review the evidence and results from Testing conducted noting any deviations from Test Strategy, Specification and Scripts.

4.5 Unit, Module and System Testing

4.5.1 Purpose

This is low-level testing conducted to ensure that the application is consistent with the logical designs, physical designs and functional specifications.

4.5.2 Test Scope

Unit and Module Testing will be carried out by the developer as part of the development phase. On successful completion, the application will be released for System Testing to ensure that the new functional requirements have been met. Logica refer to System Testing as Integration Testing and this phase is carried out by the Logica development team.

Unit, Module and System Testing will be required for P223, P217, CP1286, TAI 2349, CP1269, CP1248 and CP1287. For P217, this test phase will be executed and managed by Engage/Logica. Engage will Carry out testing on TOMAS changes and Logica will test changes to Central Systemsusing a full width database. For P223, this test phase will be executed and managed by ELEXON for the internal PrA system and by the PrA for their P223 Management Database. For CP1248 and CP1287, Logica will execute this Test Phase for the PARMS changes. The Release Team will only review and approve test materials.


4.6 Factory Acceptance Testing

4.6.1 Purpose

The purpose of Factory Acceptance Testing (FAT) is to undertake a comprehensive functional test of the system immediately prior to system installation in order to demonstrate full functional verification and conformance with requirements.

4.6.2 Test Scope

FAT will be carried out for P217 functional changes to TOMAS, by Engage and for P217 BMRA and SAA changes by Logica. FAT testing for CP1286 and TAI2349 will take place alongside P217 FAT testing. For P217, Logica carry out Factory Acceptance Testing as part of their ‘Formal System Testing’ phase. This involves Change Specific and Regression Testing, carried out under two specific testing runs – a ‘Dry Run’ and a ‘Main Run’. Logica will also perform FAT for CP1269 MDDand for CP1248 and CP1287 PARMS changes. For P223, the PrA will carry out FAT for changes to the P223 Management Database and ELEXON will carry out FAT on the new internal application that it will develop as part of the change.

4.7 Change Specific Testing

4.7.1 Purpose

Formal testing of the specific changes will be carried out and may be witnessed selectively and by agreement with Logica, and Engage. Change specific testing will be performed for units identifiedin the analysis phase or unit and system testing stages as requiring specific changes and will include verification of results where appropriate.

4.7.2 Test Scope

Change specific testing will be carried out by Logica for P217, CP1286 and TAI2349 specific changes to BMRA and SAA and for CP1269 specific changes to MDD. Logica will also carry out this phase for CP1248 and CP1287 PARMS specific changes. Engage will carry out change specific testing on P217 TOMAS changes. The PrA will carry out Change Specific testing for their P223 Management Database.

4.8 Regression Testing

4.8.1 Purpose

Regression Testing will be performed in order to verify that unchanged functionality is not affected by P217 and CP1269 changes and to demonstrate that the integrity of the Settlement Process as a whole is maintained.


4.8.2 Test Scope

Scripts from the Logica and Engage Regression Test Packs will be updated as necessary and run,in order to test that the pre-P217 functionality is unaffected. Logica will carry out Regression Testing; Engage will also perform regression testing on TOMAS with regard to the P217 changes, ensuring that the integrity of the pre-217 system price calculation and other functionality remains unaffected. This will be achieved by performing selected Settlement Runs for pre P217 systems.

Logica and Engage will also carry out regression testing for CP1269 as part of the FAT testing for CP1269. Logica will carry out regression testing on PARMS for CP1248 and CP1287.

4.9 End-to-End and Participant Testing

4.9.1 Purpose

End to End (E2E) testing demonstrates that the new / changed threads of automated business processes and flows are workable and compliant.

Participant testing demonstrates that new and amended electronic interfaces are correct. Participant testing is intended to provide assurance, additional to that derived from other testing conducted by Logica, BSC Parties, National Grid and BSCCo. Particular attention is focused upon the correctness of the interfaces between systems.

4.9.2 Test Scope

E2E and Participant testing will take place for P217 in conjunction with OAT and Parallel Run Test Phases. Testing of CP1286 will also be covered in this test phase. E2E and Participant Testing will also be conducted for P223 and for the PARMS changes made for CP1248 and CP1287. ELEXON will manage the involvement of participants in this test phase.

Participant testing will include testing of the following flows and files and their associated interfaces:

P217

Flow Name From Participant

To Participant

Change Impact

BMRA-I002 National Grid BMRA Flow to include Pre and post P217 Acceptance and Balancing Services Data including addition of BOALF Data

BMRA-I004 BMRA BSC Parties Addition of BSAA Data for Post P217

BMRA-I005 BMRA BSC Parties Addition of BSAA Data for Post P217

BMRA-I006 BMRA BSC Parties Indicative System Price Stack Data, Derived BM Unit Data


P217


To Participant

Change Impact

BMRA-I007 BMRA BMRA Addition of BOAL

Addition of BOALF

BMRA-I012 BSCCo BMRA Addition of Replacement Price Average Reference Volume (RPAR)

BMRA-I014 National Grid BMRA Addition of BSAA Data for Post P217

BSAD (NETBSAD)

National Grid BSCCo The following fields are sent. These have a zero value post P217

• Net Energy Buy Price Cost Adjustment (EBCA)

• Net Energy Buy Price Volume Adjustment (EBVA)

• Net System Buy Price Volume Adjustment (SBVA)

• Net Energy Sell Price Cost Adjustment (ESCA)

• Net Energy Sell Price Volume Adjustment (ESVA)

• Net System Sell Price Volume Adjustment (SSVA)

BSAA (DISBSAD)

National Grid BSCCo P217 introduces a new DISBSAD message which contains the following fields:

• Settlement Date

• Settlement Period

• SO-Flag

• Adjustment Cost

• Adjustment Volume

SAA-I014 sub flow 1

SAA BSC Parties Addition of BSAA Data for Post P217

Addition of BM Unit Bid-Offer Pair Acceptance Volume Data


P217


To Participant

Change Impact

Addition of SO-Flag under BM Unit Period Bid-Offer Acceptance

SAA-I014 sub flow 2

SAA Non-BSC Parties and BSCCo

Addition of following to Settlement Period Information for all Settlement Periods:

• Price Derivation Code (PDCj)

• Total NIV Tagged Volume (MWh)

Addition of Post P217 Settlement Period Information

Addition of BSAA

Addition of SO-Flag under BM Unit Period Bid-Offer Acceptance

Addition of BM Unit Bid-Offer Pair Acceptance Volume Data

SAA-I014 sub flow 3

SAA BSCCo No specific changes to the flow format.

SAA-I023 BSCCo SAA Addition of Replacement Price Average Reference Volume (RPAR)

SAA-I026 National Grid SAA Addition of BSAA Data for Post P217

BOALF File National Grid BMRA, SAA Addition of SO-Flag

Addition of Amended Acceptance Flag

P223

Flow Reference

From Participant

To Participant

Change Impact

BSCP section 3.1.1

BSCCo PrA Sample Target to be provided to the PrA, broken down by relevant sections.


P223

Flow Reference

From Participant

To Participant

Change Impact

BSCP section 3.1.2

PrA BSCCo Number of existing Sample Participants. This will feed into the internal P223 system.

BSCP section 3.1.3

BSCCo PrA Sample Participant request by MPID. This will be produced using the internal P223 system.

BSCP sections3.1.4 / 4.1.1/ 4.1.6

PrA Supplier Sample Participant request for a particular Supplier, confirmation of existing Sample Participants and confirmation of use of agents.

BSCP section 3.1.5 / 4.1.2

Supplier PrA Additional Sample Participants and their details for the Sample as requested. Confirmation of use of agents.

BSCP sections 3.2.2 / 4.1.3

Supplier PrA Where Supplier shall use agents of its choice, confirm which agents are appointed.


Supplier PrA Where Supplier shall use agents of its choice, confirm that Profile Capable Metering Systems have been successfully installed.


Supplier PrA Provision of validated half hourly data for all Metering System Identifiers (MSID) in one file.


PrA Supplier Provide details of Sample Participants that are customers of the Sample.


Supplier PrA Confirm details provided in 4.1.6.

Testing will be carried out to ensure that transmission of each flow across the relevant interfaces is successful and that the data contained in each flow is correct. Participants will be involved in testing their interfaces with Central Systems.

As part of P217, TIBCO Rendezvous product (RVRD) is being updated to version 8.1. P217 Participant Testing (in conjunction with P217 OAT) will include testing of the updated version of TIBCO data feed and data flow between interfaces in order to test the updated version of the


TIBCO Rendezvous product and gain assurance of its configuration. Logica will test the backward compatibility of TIBCO RVRD version 8.1 with National Grid and with a participant, which are bothoperating an older version of TIBCO RVRD.

4.10 Parallel Run Testing

4.10.1 Purpose

The purpose of parallel run testing is to provide assurance, additional to that obtained viaregression testing, that the revised central systems have not adversely impacted unchanged functionality. This will be achieved by producing correct results for selected functionality in an as-live environment and using live data. This testing will also verify that there is no significant change in performance.

4.10.2 Test Scope

Parallel run testing the pre and post P217 systems will be a part of OAT and will involve running the test environment alongside the production environment. Logica must ensure that its production schedule is given priority and parallel run testing may be delayed if production deadlines are threatened. This test phase will also include testing of CP1286.

Parallel run testing will be carried out upon an OAT test environment, which replicates the live environment, and has installed the application software as amended by P217 changes. Live processing will then be duplicated for a specified test period. The results will be compared with those produced by the live runs. The differences will be inspected in order to ensure that they are as expected given the P217 changes to the pricing calculation. The processing time for each run will be recorded and compared with the timings of the live runs.

The scope of testing will involve:

• Performance of associated processes and flows for both pre and post P217 cases;

• Selected Settlement Runs, performed for pre and post P217 changes;

4.11 Operational Acceptance Testing

4.11.1 Purpose

The purpose of operational acceptance testing (OAT) is to ensure that software is operational and that changes do not impact operational requirements and that service levels can be met.

OAT includes deployment testing, the purpose of which is to ensure that the updated software can be deployed to a test system and, if necessary, backed out.

4.11.2 Test Scope

For P217, Logica will arrange that Operational Acceptance Testing (OAT) will be carried out on the revised SAA and BMRA Systems using the OAT test pack. OAT will be carried out in conjunction with Parallel Run Testing and E2E and Participant Testing phases. Participant Testing may continue after the OAT phase has completed. Engage will also carry out OAT on the updated TOMAS system for P217.


P217 OAT will also include testing of TIBCO data feeds. BSC Central Systems will be upgrading to TIBCO RVRD version 8.1. Logica has received written confirmation from TIBCO that RVRD version 6.7 and higher versions are compatible with TIBCO RVRD version 8.1. Logica will test the backwards compatibility of TIBCO RVRD version 8.1 with a participant and National Grid which are both operating older versions of TIBCO RVRD.

4.12 Deployment and Backout Testing

4.12.1 Purpose

Deployment Testing focuses on ensuring that the Application Software can be installed correctly. Backout Testing focuses on ensuring that the Application Software upgrade can be removed and the previous version reinstated should any problems occur.

4.12.2 Test Scope

For CP1269, P217, CP1286 and TAI2349 Logica will carry out Deployment and Backout Testing for BMRA, SAA and MDD. ELEXON will carry out this phase of testing for CP1248 and CP1287 PARMS Changes. Engage will carry out this test phase for changes to TOMAS. IMServ will carry out Deployment and Backout Testing for P223 systems that it develops and ELEXON will carry out this test phase for the internal P223 system that it develops.

4.13 Volume and Performance Testing

4.13.1 Purpose and Scope

The purpose of Volume Testing is to demonstrate that the revised system continues to function correctly within operationally acceptable performance levels under a full data load. For P217 performance monitoring will be demonstrated for the SAA and BMRA systems throughout all test phases. However, the majority of Volume and Performance testing will take place during the Regression and E2E / Participant testing which uses production like depth and width of data.

In addition to checking actual results, Logica will also check run times and other performance indicators in order to ascertain that there has been no undue degradation in performance. This will make up the performance testing.

P217 Acceptance Testing for TOMAS will use live data volumes and will therefore adequately test volume and performance requirements under full load. CP1286 will be tested as part of this test phase.

4.14 User Acceptance Testing

4.14.1 Purpose

The purpose of User Acceptance Testing (UAT) is to ensure that the delivered system meets the requirements of the relevant BRS and the usability requirements of the end users by testing that the expected outputs match the test inputs for all functions and requirements


4.14.2 Test Scope

ELEXON will perform UAT for PARMS (for CP1248), TOMAS (for P217, the MDD Loader (for CP1269) and the internal P223 system.

5 Test Management

5.1.1 Test Procedures

Defects identified by the Release Team or the end users during product reviews (software testing and documentation reviews) will be notified to Logica, and Engage or the Release Team as appropriate in accordance with the procedures specified in the Quality Manual (Reference 8), and in accordance with the contracts in place with Logica and Engage.

The test procedures to be used by the Release Team will be based on the procedures used in previous ELEXON Release projects and will be tailored to the appropriate quality requirements of the Release.

Logica and Engage will be responsible for ensuring that any test phases carried out and managed solely by them, or in association with the Release (i.e. User Acceptance Testing), will be carried out in a manner consistent with good industry practise and in accordance with their own test procedures.

In general, software defects raised during one phase of testing should be cleared by the start of the next phase, although defects which are of low business impact, on agreement with the Release Manager, may be left outstanding, rather than delay the start of the next phase.

Release deliverables will be approved by Release Manager.

Any defects identified against these products after authorisation will be managed as appropriate to the magnitude of the defect; any defects which would result in incorrect results must be corrected and re-tested prior to implementation; other, less severe defects would be corrected if this did not adversely impact the timely implementation of the Nov 09 Release. These products will have gone through a full ELEXON Quality review procedure prior to authorisation, so the risk of issues is believed to be small.

5.1.2 Test Report

Test Reports will be produced to record the testing performed, to summarise the test results from each type of test carried out, to describe any defects found during testing and the steps taken to resolve them, and to specify any deviations from the relevant test strategies and test specifications.

The following test reports will be produced:

• Logica will write Test Reports which will be submitted to ELEXON as inputs to the Go-live decisions;

• Engage will write and submit to ELEXON a FAT Test Report covering the outcome and results of FAT;

• ELEXON will write an overall Nov 09 Release Test Report to document all testing related activities carried out for the release;


• ELEXON will write a Test Report covering the outcome of FAT on the internal P223 system;

• ELEXON will report on the Acceptance Testing carried out on TOMAS as part of P217; and

• The PrA will provide updates of testing conducted for their P223 Management Database.

5.1.3 Timescales

The timescales and dates for the individual test phases will be defined in the PID and Release Plan and Exception Plan for the November 09 Release (References 2 and 10).

5.1.4 Test Phase Acceptance Criteria

Logica and Engage must also ensure that the scope of testing is adequate to ensure that compliance with the relevant acceptance criteria defined for the release can be achieved.

Completion of each Test Phase will be accepted once all identified defects have been captured and addressed for each test phase.

6 Assumptions, Risks and Issues

6.1.1 Assumptions

The following assumptions have been made:

• There are a sufficient number of Participants who are willing to take part in end-to-end testing.

6.1.2 Risks

The following risks have been identified:

• There is a risk that there will not be a sufficient number of participants available for testingfor P217. This will compromise how comprehensive the end to end testing is.

6.1.3 Issues

There are no issues to date.

7 Document Control

a Authorities

Version Date Author Reviewer Reason for review

V0.1 09/03/2009 SHJ YVW Peer Review

V0.2 07/05/2009 TYM YVW Internal Review

V0.3 08/05/09 SHJ YVW CP1248 additions


V0.4 15/05/09 TYM YVW, SHJ, MIL, JAP, TAK

Internal Review

V0.5 01/06/09 TYM YVW, SHJ, TAK

Internal Review

V0.6 02/06/09 TYM YVW, SHJ, TAK

Updates following review

V0.7 12/06/09 TYM Logica, Engage

External Review

V0.8 23/06/09 TYM YVW Update following review of version 0.7

V0.9 02/07/09 TYM GRW, SHJ Addition of P231, P232 Changes

Version Date Author Approver Signature

V1.0 23/06/09 TYM Yvonne Walsh

Version Date Author Authoriser Signature

V1.0 TYM Alex Grieve

b Distribution

Recipient Version Date Reason

c References

Reference Document

Reference 1 Change Delivery Guidelines for Testing Changes to BSC Systems

Reference 2 PID and Release Plan for November 09 BSC Systems Release

Reference 3 Business Requirements Solutions for Alternative Modification Proposal P217

Reference 4 Business Requirements Solution for Modification Proposal P223 ‘Profile Administration Service’

Reference 5 CP1248 ‘Early release of Meter Technical Details by Non Half Hourly Meter Operator Agent’

Reference 6 CP1269 ‘Publication of Additional Non Half Hourly Combination Data in Market Domain Data’

Reference 7 Change Delivery Quality Plan

Reference 8 Change Delivery Quality Manual


Reference Document

Reference 9 November 2009 Release Impact Matrix

Reference 10 Exception PID and Release Plan for the November 09 BSC Systems Release

Reference 11 Logica November 2009 Test Strategy

Reference 12 IMServ November 2009 Test Strategy

Reference 13 Engage November 2009 Test Strategy

A Appendix A – Terms used in this Document

Other acronyms and defined terms take the meanings defined in the Balancing and Settlement Code (the Code), Section X.

Acronym/Term Definition

AM/Dev Application Management and Developer

BMRA Balancing Mechanism Reporting Agent

BPO Host Business Process Operator Host

BSCCo Balancing and Settlement Code Company (ELEXON)

DS Design Specification

E2E Test End to End Test

MDD Market Domain Data

MDDM Market Domain Data Manager

PrA Profile Administrator

SAA Settlements Administration Agent

SS System Specification

SVAA Supplier Volume Allocation Agent

TOMAS Trading Operation Market Analysis System

URS User Requirements Specification