comprehensive delivery process sdm lite testing phase training facilitator name: state of...

Comprehensive Delivery Process

SDM Lite Testing Phase Training

Facilitator Name:

State of Connecticut Department of Information Technology (DOIT)

( Testing Phase Training

SDM v2.0 Confidential - Internal Use Only


Agenda

Overview of SDM

Governance

Purpose of Testing Phase

Testing Phase Deliverables and Their Owners

Team Members

Testing Phase Activities




Business Issue

•Phase Kickoff Meeting •Project Management Plan•Project Profile •Detailed Schedule for Next-Phase •Phase-End Decision-Point Meeting

Business Requirements

•Bus. and Tech. Rqmts Document•Solution Alternatives Document•Deployment Strategy & Plan•Detailed Schedule for Next-Phase •Phase-End Decision Point Meeting

Design

•System Design Document•System Security Profile•System Design Signoff - TRB•System Design Signoff - PSC•Test Strategy & Plan•Detailed Schedule for Next-Phase

Construction

•Code & Unit Testing•Code Review•Development/Test Environments•Environment Migration Checklist•Software “Golden Build”•System Bill of Materials•System and User Documents•Test Scenarios/Cases•Test Data•Detailed Schedule for Next-Phase

Testing

•Environment Migration Checklist•Integration Testing•System Testing•User Acceptance Testing•Software “Golden Build”•System Bill of Materials•Test Summary Report•Prod. Support & Admin Document•Detailed Schedule for Next-Phase •Phase-End Decision Point Meeting

Post-Implementation

•Project Summary•Project Shutdown

Implementation

•Technology Solution Deployment•Approved Production Turnover•Phase-End Decision Point Meeting

Project Management Plan, Cost Benefit Analysis

Deliverables Updated in Multiple Phases:

SDM Lite v2.03 Deliverables/Events by Phase




Governance – The Project Steering Committee (PSC)

Who are the typical PSC members?

Agency Commissioners or Designee

Executive Director or Designee

Divisional Directors

Not members of the Project Team Wheel(With the exception of the Executive Sponsor)

What does the PSC do?

Sets business strategies within the targeted area;

Sets priorities and allocates resources (people and budgets); and

Makes Go, No-Go, or Redirect decisions for any project in the business area throughout the entire project life cycle.




Testing Phase PurposeThe Purpose of the Testing Phase is to:

Prove that the developed system satisfies the requirements defined in the Business Requirements and Technical Requirements documents. This is accomplished by executing the following test activities:

Integration Tests: Tests shall be executed and evaluated by the Testing team to prove that the various modules/functions within each application component integrate properly. Within each application, the Testing team will verify that the application features/functions, data integrity, and security satisfies the defined requirements of the system.

System Tests: Tests shall be executed and evaluated by the Testing team to prove that all of the applications and infrastructure components work together as a complete system. During system testing, the Testing team will again verify that the application features/functions, data integrity, and security satisfies the defined requirements of the system, but from an “end-to-end” system perspective.

User Acceptance Tests: Tests shall be executed and evaluated by the Business team to prove that the system meets all business requirements and supports the business scenarios of the end-user. During system testing, the Business team will verify that the system features/functions, data integrity, response times, and security satisfies the defined requirements of the system from a business process perspective.

Performance Tests: Tests shall be executed and evaluated by the Testing team to prove that the system will support end-user response time requirements, timeline processing of estimated data volume, the estimated number of end-users, estimated transaction/data growth, and sufficient performance when system resources are “stressed”.

Recovery Tests: Tests shall be executed and evaluated by the Development team to prove that the system can be backed out and restored to a previous state. This will occur in the event of undesirable implementation results requiring a solution back out or, if an unplanned disaster were to occur requiring a system recovery to maintain business continuity.

Engage the Production support team, where the Production Support & Administration Document is viewed and signed off by the Production Support Team to prepare for production turnover.

Develop a detailed plan that commits to the completion of the Testing Phase; verify targets for the completion of the Implementation and Post-Implementation Phases.




Testing Phase Work Breakdown Structure (WBS)5.0 Testing

Phase

5.01 Perform Integration

Testing

5.02 PerformSystem Testing

5.03 PerformUser Acceptance

Testing

5.07 Conduct Production Support

Process Review

Verify the

Integration TestEnvironment

Verify The

System TestEnvironment

VerifyThe UAT

Environment

5.08 Conduct Phase-End Decision Point Meeting

Create the Phase-End Decision Point Meeting

Presentation

Create the System Test

Summary

Create the UAT

Summary

Create the Integration Test

Summary

Approve & SignoffOn the Production Support & Admin

Document

Conduct the Production Support

Process ReviewMeeting

5.04 PerformPerformance

Testing

VerifyThe PerformanceTest Environment

Migrate ApplicationTo Performance

Test Environment

Execute the Communication Plan

Conduct the Phase-End Decision Point Meeting

Plan Remaining Phases

ObtainPerformance Testing

Signoff

Update the Defect Management

Tool

5.06 ConductTest Phase

Review Meeting

CreateTest Phase

SummaryReport

ObtainTesting Phase

Signoff

Conduct theTest Phase

Review Meeting

Conduct Performance

Testing

Complete the Production Support

& AdministrationDocument

5.05 Perform Recovery Testing


Tool

Update theDefect Management

Tool


Tool

ConductUAT

Migrate ApplicationTo

UAT Environment

ConductInter-Operability

Testing

Create the Performance Test

Summary

Conduct Integration Testing

Migrate Application to

Integration TestEnvironment

Conduct System Testing

ConductInter-Operability

Testing

Migrate Application To


Create Recovery

Test Cases

Execute the Backout & Recovery

Test Cases

Execute Disaster Recovery

Test Cases

Establish the Recovery

Test Environment

ObtainIntegration Testing

Signoff

Obtain System Testing

Signoff

ObtainUAT Signoff

Establish the“Golden Build”

For System Testing


For UAT


For Performance Testing


tool

ObtainRecovery Test

Signoff

Create theRecovery Test

Summary

Establish the“Golden Build”For Production

Deployment

Create/Migrate Test Data to

Integration Test Environment




UAT Environment


Performance TestEnvironment


Recovery TestEnvironment




Key Deliverables Created in the Testing Phase

Team Lead Deliverables:

Back Out & Recovery Test Execution,

Disaster Recovery Test Execution,

Environment Migration Checklist,

Testing Phase Summary,

Integration Test Execution,

System Bill of Materials,

System Test Execution,

“Golden Build” of Software for Implementation,

Performance Test Execution,

User Acceptance Test Execution,

Production Support & Administration Document

Project Management Deliverables:

Cost-Benefit Analysis,

Detailed Phase Schedule,

Phase-End Decision Point Presentation,

Project Management Plan, and

Project Team Wheel




Testing Phase Deliverable Snapshot Phase Activity Numbers

Team Lead Deliverables Requirement Owner 01

02

03

04

05

06

07

08

Back out & Recovery Test Execution Mandatory Development Lead S I

Deployment Strategy & Plan Mandatory Deployment Lead I

Environment Migration Checklist Mandatory Infrastructure Lead S S S S

Testing Phase Summary Mandatory Test Lead S

Integration Test Execution Mandatory Test Lead S I

System Bill of Materials Mandatory Development Lead R R R R R

“Golden Build” of Software for Production

Mandatory Development Lead C

Performance Test Execution Mandatory Test Lead S I

User Acceptance Test Execution Mandatory UAT Lead S I

Production Support & Admin. Document Mandatory Prod. Support Lead S

Phase-End Decision Point Presentation Mandatory Technology Manager

S

Updated Project Management Plan Mandatory Technology Manager

R




Project Steering Committee (PSC)Name / AgencyName / Agency

TECHNOLOGY PARTNERS

BUSINESS PARTNERS

* <Name Required>Business Manager

* Name RequiredBusiness Requirements

Lead

<Name Required>Technology Manager

Name RequiredDevelopment Lead

Name RequiredSecurity Lead

* Name(s) RequiredBusiness SMEs

* Name RequiredBusiness Process

Lead

Name RequiredTest Lead

* Staffed from the Business

Name RequiredInfrastructure Lead

Name RequiredProduction Support Lead

* TBDUAT Lead

Name Required Executive Sponsor

Current Phase: Testing

Support/Consulting Roles:Financial Advisor: Enterprise Architect: Name RequiredBusiness Division Director: Name Required

* Name RequiredDeployment Lead

Name RequiredVendor Lead

Instructional Note:•Remove this note before presenting this slide•Remove roles that are not applicable to your project (even is it has “name required”)•If the role IS applicable to your project, then you must specify a name if the role has “name required”•If other standard SDM roles are needed, you may add them to the wheel.•Do not create any new role names; All role names are standardized within the SDM framework




Business Division Director (Technology Role) Assists the Executive Sponsor with establishing a Project Steering Committee, and securing resources for the team, as needed. Point of leadership escalation for technology risks and issue for the Project Team. Ensures that all technology projects being run within their assigned Agencies are aligned with SDM. Monitors the execution of all technology projects within their assigned Agencies.

Business Manager (Business Role) Responsible for coordination and delivery of all Business activities and deliverables; Provides business consultation to the technology team; Provides project plan updates for all Business activities and deliverables; Attends/Participates in all Phase-End Decision Point Meetings

Testing Phase Roles and Responsibilities




Business Process Lead (Business Role) Ensures that User Documentation and Training Materials adequately address all planned business process change; Provides general business process consultation to the project team;

Business Requirements Lead (Business Role) Maintains the Functional Requirements Workbook; Maintains the Non-Functional Requirements Workbook; Provides general business requirements consultation to the project team;

Business Subject Matter Experts (SMEs) - (Business Role) Provides business area consultation to the business team as needed to assist with business deliverable review and creation;

Deployment Lead (Business Role) Maintains the Deployment Strategy & Plan document; Responsible for coordinating all implementation activities at the deployment site(s); Verifies all User Documentation and Training Materials;





Development Lead (Technology Role) Maintains the System Design Document; Maintains the Technical Requirements Workbook; Maintains the Requirements Traceability Matrix; Creates and maintains the System Bill of Materials; Responsible for executing Recovery Testing; Responsible for loading the migrating new software builds across environments; Responsible for coordination of all software defect remediation; Responsible for executing Performance Testing; Responsible for identifying defects, and entering them into the defect management tool; Contributes Performance test metrics to the Test Summary Report;

Enterprise Architect - (Technology Role) Responsible for ensuring that EWTA requirements are addressed with the solution; Provides on-going architecture consultation to the project team;





Executive Sponsor (Business Role) Responsible for providing the project funding source; Attends/Participates in Phase-End Decision Point Meeting; Primary leadership escalation point for significant project risks and issues;

Infrastructure Lead (Technology Role) Single point of contact for the project team for all infrastructure needs (e.g., hardware, software, database, network); Responsible for the planning and coordination of all infrastructure activities; Relies on resources from the various infrastructure areas to deliver the actual infrastructure work; Creates and maintains the Environment Migration Checklist; Responsible for establishing the test environment; Assists the Development lead in loading test data across into the various environments; Assists the Development lead in loading software builds into the various environments;





Production Support Lead - (Technology Role) Creates and maintains the Production Support & Administration Document; Responsible for stakeholdering the new solution with Production Support; Manages and executes the Production Turnover Checklist; Coordinates production turnover signoff with the Production Support team;

Project Steering Committee (Business & Technology Leadership Roles) Provides governance and decision making to the project team; Removes barriers for the project team; Reviews the Phase-End Decision Point presentation material prior to the Phase-End Decision Point Meeting; Renders a Phase-End decision of “Go”, “No-Go” or “Redirect” as it relates to the project team proceeding to the next phase.

Security - (Technology Role) Maintains the System Security Profile Document; Provides security consultation to the project team as needed;





Technology Manager (Technology Role) Responsible for creation and maintenance of the overall project plan (including Business and Technology activities); Responsible for the overall coordination and delivery of all Technology deliverables; Responsible for Change Management on the project; Creates all Phase-Kickoff Meeting presentations; Facilitates all Phase Kickoff Meetings; Updates the Project Management Plan; Responsible for creating the PSC Phase-End Decision Point Meeting Presentations; Hosts and facilitates the PSC Phase-End Decision Point Meetings; Responsible for providing project status reporting at monthly project reviews; Responsible for creating the final Project Summary and formal shutdown of the project.

Test Lead (Technology Role) Maintains the overall Test Strategy & Plan for the project; Responsible for the planning and execution of Integration and System testing; Maintains all Integration and System test cases and scenarios; Responsible for identifying defects, and entering them into the defect management tool; Responsible for mapping Integration & System test cases to business requirements in the Requirements Traceability Matrix; Responsible for creating and presenting the Test Summary report and the conclusion of the Testing phase;





UAT Lead (Business Role) Responsible for the planning and execution of User Acceptance testing; Maintains all User Acceptance test cases and scenarios; Responsible for identifying defects, and entering them into the defect management tool; Responsible for mapping UAT test cases to business requirements in the Requirements Traceability Matrix; Contributes UAT test metrics to the Test Summary Report;

Vendor Lead - (Technology Role) The single point of contact for the Vendor; Responsible for the management and coordination of all Vendor deliverables to the project team;





Activity 5.01: Perform Integration TestingPurpose:

The purpose of this activity is to install the “Golden Build” of the application created at the end of the Construction phase into the Integration Test environment. Verify that all application modules/functions work together as designed. Thoroughly test the functionality of features that could not be tested earlier during code and unit testing. Verify that each application component has achieved the required level of quality before proceeding to system testing.

Integration testing focuses mainly on a logical group of modules/functions WITHIN an application. Integration testing verifies the interaction between such modules/functions to ensure they work properly within an application.

This Activity Produces the Following:

Verify the Integration Test Environment;

Migrate the Application to the Integration Test Environment;

Create/Migrate the Test Data to the Integration Test Environment;

Conduct Integration Testing;

Update the Defect Management Tool;

Create the Integration Test Summary;

Obtain Integration Testing Sign-Off; and

Establish the “Golden Build” for System Testing.




Activity 5.02: Perform System Testing

Purpose:

The purpose of this activity is to validate that all the solution aspects operate and meet the business and technical requirements of the business process, which include interoperability, features/functions, data integrity, security, and installation procedures.

System Testing focuses on the integration and interaction of a complete system, taking all of the individual application components that were tested separately during Integration testing and creating scenarios that string many applications together to validate that the system behaves properly from an “end-to-end” testing perspective.

System testing involves taking a group of test cases (e.g., often a subset of integration test cases) that utilize individual components and running them in sequence to simulate a required business function.


Verify the System Test Environment;

Migrate the Application to the System Test Environment;

Create/Migrate the Test Data to the System Test Environment;

Conduct Interoperability Testing;

Conduct System Testing;


Create the System Test Summary;

Obtain System Testing Sign-Off; and

Establish the “Golden Build” for User Acceptance Testing (UAT).




Activity 5.03: Perform User Acceptance Testing (UAT)

Purpose:

The purpose of this activity is to test the solution and verify that it meets user needs. User Acceptance Testing verifies that the business process and systems work together as expected.


Verify the UAT Environment;

Migrate the Application to the UAT Environment;

Create/Migrate the Test Data to the UAT Environment;

Conduct Interoperability Testing;

Conduct User Acceptance Testing;


Create the UAT Summary;

Obtain UAT Sign-Off; and

Establish the “Golden Build” for Performance Testing.




Activity 5.04: Perform Performance Testing

Purpose:

The purpose of this activity is to ensure that the software achieves required performance levels as defined within the Business Requirements.


Verify the Performance Testing Environment;

Migrate the Application to the Performance Test Environment;

Create/Migrate the Test Data to the Performance Test Environment;

Conduct Performance Testing;


Create the Performance Test Summary; and

Obtain Performance Test Sign-Off.




Activity 5.05: Perform Recovery Testing

Purpose:

The purpose of this activity is to test the established plans to recover information, business processes, and systems to an operational state in the event of disaster or undesirable results.

Recovery testing does not focus on system functionality; it focuses on the following two areas:


Create Recovery Test Cases; Establish the Recovery Test Environment; Create/Migrate the Test Data to the Recovery Environment; Execute the Back Out and Recovery Test Cases; Execute Disaster Recovery Test Cases; Update the Defect Management Tool; Create the Recovery Test Summary; and Obtain Recovery Test Sign-Off.

1. Ensuring that the new system is being deployed can be backed out and restored to a previous state (i.e., a Back out & Recovery procedure) if the new system implementation yields unacceptable results.

2. Ensuring that an effective/operational recovery procedure exists in the event of an unplanned disaster (i.e., a Disaster Recovery plan), so that the effects of a disaster will be minimized and the organization will be able to either maintain, or quickly, resume mission-critical functions.




Activity 5.06: Conduct Testing Phase Review Meeting

Purpose:

The purpose of this activity is to review the final test results from all sub-phases of testing conducted on the project. The project team assesses whether or not the final testing results satisfy the Testing phase exit criteria as defined in the Project Management Plan document.


Create the Testing Phase Summary Report;

Conduct the Test Summary Review Meeting;

Obtain Testing Phase Sign-Off; and

Establish the “Golden Build” for Production Deployment.




Activity 5.07: Conduct Production Support Process Review Meeting

Purpose:

The purpose of this activity is to document the support structure required to support the new application once it is deployed to production. The support structure will describe the available support services, the responsible support organizations, and the support processes and procedures that will be utilized.

Within the State of Connecticut, it is expected that some, or all, resources from the application development team, who have now constructed the new system, will also become part of the production support organization once the solution has been implemented. The production support model should be established in “tiers”, such that problems are escalated from lower tiers to higher tiers based on the complexity of the problem or required problem solving skill set (e.g., if network-related, escalate to the network team). A common tier-based model is to establish the following tiers:

Tier 1: Call Center/Help Desk (most basic problem resolution, such as user-id setups); Tier 2: Development Team (more complicated application problem diagnosis; requires in-depth knowledge of

the application and data); Tier 3: Infrastructure Team (infrastructure problem diagnosis; requires in-depth knowledge of the hardware,

software, data structures, or networks).

Document the production turnover criteria so it is clear that when the solution is declared ready for production turnover, the project team can be dissolved.

If an existing Service Level Agreement (SLA) with a business unit(s) is expected to be impacted, additional sign offs may be required to support the deployment of the initiative. Any SLA impact must be documented in this support plan.

Once placed under change control, this document must be maintained until deployment is complete.

(Cont.)




Activity 5.07: Conduct Production Support Process Review Meeting (Cont.)


Complete the Production Support and Administration Document;

Conduct the Production Support Process Review Meeting; and

Approve and Sign-off on the Production Support and Administration Document.




Defect Severity LevelsDefect severity levels assign a numeric value representing the impact that the incident/defect has on the system/environment. Severity levels will be assigned to defects based on the severity level definitions described below.

Severity Level

1

2

3

4

• Critical Outage;

• System or key component of a system is unavailable for use;

• Emergency situation requiring immediate action overriding all planned actions.

• System is available but a key component is not functioning correctly;

• Significant defect to core functionality within the system;

• Manual workaround may or may not be available.

• System is available but a component is not functioning correctly;

• Does not necessarily result in a failure;

• Workaround is available.

• System is available but a minor or infrequently used function is not processing correctly;

• Cosmetic defects.




Activity 5.08: Conduct Phase Decision Point MeetingPurpose:

The purpose of this activity is to obtain a “Go”, “No-Go”, or “Redirect” decision from the Project Steering Committee (PSC).

Minimum Suggested Attendees for the Phase Decision Point Meeting:

Project Manager,

Executive Sponsor,

PSC,

Technology Manager, and

Business Manager


Distribute the Testing Phase-End Decision Point Presentation to the PSC;

Conduct the Phase-end Decision Point Meeting; and

Execute the communication plan.




Continuous SDM Support & Training

SDM Resources:

SDM website: www.ct.gov/doit/sdm

SDM Overview & Training

SDM Sample Deliverables

SDM Email Box:

[email protected]

Inbox is reviewed on a regular basis (e.g., weekly, bi-weekly, etc.)

Will add input for ‘FAQ’ section of website

PMO Team will respond to feedback

SDM Continuing Education:

SDM open communication forums

SDM Brown Bags

Specific discussion on ‘hot’ SDM topics

Scheduled assistance as requested

The purpose of the System Development Methodology (SDM) is to establish procedures, practices, and guidelines governing the initiation, concept development, planning, requirements analysis, design, development, integration and test, implementation, and operations, maintenance and disposition of information systems (IS) within the State of Connecticut.




PMO Team MembersPlease contact the PMO Team with any questions that you have. The PMO Team will be reviewing the SDM Feedback Mailbox on a regular basis.

PMOPMO

PMO Team Members:

Lois Bryant 860-622-2981 [email protected]

Tricia Johnson, PMO Director 860-622-2549 [email protected]

Dawn Keiper 860-622-2232 [email protected]

Jim Rutushni 860-622-2271 [email protected]

Lori Violette 860-622-2365 [email protected]

Larry Williford 860-622-2465 [email protected]

The PMO Team also has a group telephone # 860-622-2085. If you receive the voicemail, please leave a message as the voicemail will be

checked on a regular basis.

PMO Web site: www.ct.gov/doit/pmo




Thank you for attending!

Any questions, comments, or suggestions?

comprehensive delivery process sdm lite testing phase training facilitator name: state of...

Documents

testing phase purpose

phase slide

phase activities

testing phase training

testing team

developed system

project team wheel

sdm lite v2