Enterprise Performance Life Cycle (EPLC) Stage Gate ReviewsKevin Lyday & Robin Eggleston

February 6, 2009

22

Agenda

Enterprise Performance Life Cycle• Overview

• Framework

EPLC Stage Gates• Purpose

• Benefits

• Types

• Thresholds

• Roles and Responsibilities

Critical Partners

Summary

Questions & Answers

2

33

EPLC Framework Overview

• Enhances Information Resource (IR) governance through rigorous application of sound investment and Project Management principles and industry best practices

• Establishes a structure for investments and projects to consistently achieve successful outcomes

• Improves the quality of project planning and execution, reducing overall project risk

HHS EPLCHHS EPLC

CD

C U

PC

DC

UP

44

HHS EPLC Framework

55

Stage Gate Review – Purpose

Emphasis of Stage Gate

reviews

Emphasis of Stage Gate

reviews

The successful accomplishment

of the phase objectives

The successful accomplishment

of the phase objectives

The risks associated with moving into the next life cycle

phase

The risks associated with moving into the next life cycle

phase

The plan for the next life cycle

phase

The plan for the next life cycle

phase

The Stage Gate Review is:

An evaluation process by which a project is authorized to progress from one life cycle phase to the next

A collaborative practice in which key stakeholders play an important role in assessing the project’s overall health and quality of execution

• A phase-driven go/no-go decision point where project activities are reviewed to assure it is appropriate to proceed to the next phase

66

Stage Gate Review - Benefits

• Ensures that sound project management, quality and technical practices are followed

• Confirms completeness and accuracy of deliverables

• Ensures compliance with Federal, HHS and CDC policies

• Facilitates risk and issue mitigation and resolution

• Minimizes project delays and cost overrun

• Can save money!

77

Stage Gate Review - Types

A. Initiation

D.Requirements Analysis

F. Development

G.Test

I. Operations & Maintenance

J. Disposition

B. Project Selection (end of Concept)

C. Project Baseline (end of Planning)

E. Preliminary Design (middle of Design)

H. Operational Readiness (middle of Implementation

CDC IR GovernanceCenter IR Governance or delegated body

88

Stage Gate Review - Thresholds

IR Governance

Council

Enterprise Planning &

Review

Public Health Domain

Coord Centers & National

Centers Threshold A Threshold B Threshold C Threshold D

CPIC Requirements

Major >=$10M Tactical >$3M & <$10M Supporting >$1M & <$3M <$1M Additional Factors CDC (as an enterprise) Mission Criticality

High Medium Low

Variance in Cost, Schedule & Performance (EVM)

>=20% >=10% & <20%

99

Role Responsibility

IR Governance Organization or delegated authority

• Reviews recommendations from the Stage Gate Review Team• Decision authority to pass the project to the next phase.

Federal Project Officer or Manager

• Determines that the project is prepared for the Stage Gate Review. • Requests scheduling from the appropriate Stage Gate Review Lead.• Provides deliverables for the Stage Gate Review.

Stage Gate Review Lead

• Makes arrangements for the Stage Gate meeting and communicates the schedule with other participants.

• Collects, reviews, and distributes the appropriate documentation. • Ensures that the Stage Gate Review Team has achieved its purpose

and agreed upon next steps.

Stage Gate Review Team Member

• Stage Gate Review Team consists of the Business Owner and stage appropriate Critical Partners and Stakeholders.

• Complete Stage Gate Review evaluations using provided templates and submit them to the Stage Gate Review Lead in advance of the Stage Gate Review Team meeting for compilation and distribution.

Stage Gate Review – Roles and Responsibilities

10

Critical Partners

*Critical Partners

Enterprise Architecture

Security

Acquisition Management

Finance

Budget

Human Resources

Section 508

CPIC

Performance

Others such as Health Scientist or Epidemiologist as needed

During Stage Gate Reviews, the Critical Partners will review documents for completeness, accuracy, adequacy and make recommendations to the appropriate IR governance organization.

During Stage Gate Reviews, the Critical Partners will review documents for completeness, accuracy, adequacy and make recommendations to the appropriate IR governance organization.

* Indicates roles only; not full time employees * Indicates roles only; not full time employees

11

The Stage Gate Review Process

PM

Critical Partners

• Deliverables• Exit Criteria Compliance• Known Issues

Next Phase

Discontinue Project

Approve

Prepare/Revise

ReviewIssues

Reject

Recommendations

Conditions

Not Approve (Issue Resolution)

Unresolved Issues

IT Governance

Approve w/Conditions

GateGate

* Indicates federal FTE that is responsible for the project * Indicates federal FTE that is responsible for the project

**

12

Stage Gate A - Initiation

Purpose: The Initiation Stage Gate Review considers whether the Business Needs Statement justifies proceeding to the Concept Phase for development of a full Business Case and preliminary Project Management Plan.

Example Initiation Assessment Questions:Example Initiation Assessment Questions:

Has the business owner defined the business need?

Have the stakeholders been identified and informed of the Business Needs Statement for the potential project?

Is the Business Needs statement sound and consistent with the Enterprise Architecture?

Is there a rough order of magnitude on cost and schedule in the business needs statement?

May be DelegatedMay be Delegated

13

Stage Gate B – Project Selection Review

Purpose: The Project Selection Review (PSR) is a formal inspection of a proposed IT project by the IT governance organization to determine of it is a sound, viable, and worthy of funding, support and inclusion in the organization’s IT investment Portfolio.

Example Project Selection Review Questions:Example Project Selection Review Questions:

What are the key objectives of the project?

Does the Business Case details the business need and expected performance outcomes?

Has an internal (government) configuration management process been developed?

Has the approach to risk management been tailored to suit the scale of the project?

IR Must PerformIR Must Perform

14

Stage Gate C – Project Baseline Review

Purpose: The Project Baseline Review (PBR) is a formal inspection of the entire project and performance measurement baseline initially developed for the IT project. The PBR includes review of the budget, risks, and user requirements for the investment.

Example Project Baseline Review Questions:Example Project Baseline Review Questions:

Are all the activities included in the plan?

Does the budget contain all the resources required for successful completion of the project? This would include any interfaces with external systems and projects.

Have performance goals been established and a monitoring mechanism implemented to assure goals are achieved?

Is the WBS based on deliverables or tasks? Are the estimate assumptions clear and up front?

IR Must PerformIR Must Perform

15

Stage Gate D – Requirements Analysis

Purpose: The documented requirements are validated and further analyzed and decomposed into functional and non-functional requirements that define the automated system in more details with regard to inputs, processes, outputs, and interfaces.

Example Requirements Analysis Assessment Questions:Example Requirements Analysis Assessment Questions:

Are meetings conducted with the End Users to elicit requirements?

What is the single most important requirement for the project?

Are the requirements detailed enough and with enough specificity to be measurable?

What is the quality assurance process for the business requirements?

May be DelegatedMay be Delegated

16

Stage Gate E – Preliminary Design Review

Purpose: The Preliminary Design Review (PDR) is a formal inspection of the high-level architectural design of an automated system, its software and external interfaces, which is conducted to achieve agreement and confidence that the design satisfies the functional and non-functional requirements and is in conformance with the enterprise architecture.

Example Design Assessment Questions:Example Design Assessment Questions:

Has a formal review of the high-level architectural design been conducted?

Does the Design Document provide an overview of the entire hardware and software architecture and data design, including specifications for external interfaces?

Does the test plan define all the types of tests (unit, functional, integration, system, security, performance (load and stress), user acceptance, and/or independent verification) that are to be carried out?

Are backup procedures and responsibilities well-designed and fully documented? Are post-disaster recovery procedures included in the design?

IR Must PerformIR Must Perform

17

Stage Gate F - Development

Purpose: The Development Stage Gate Review evaluates whether the project should proceed to the Test Phase.

Example Development Assessment Questions:Example Development Assessment Questions:

Have the types of tests, the acceptance criteria for those tests, and the manner of testing been finalized?

Have the Test Plan and Test Cases been finalized? Does the Test Plan include evaluation of Performance Metrics?

Does the Operations and Maintenance Manual provides the operations and support staff, including the Help Desk, the information necessary to effectively handle routine production processing, ongoing maintenance, and indentified problems, issues, and/or change requests?

Does the independent Verification & Validation (IV&V) Report adequately document the findings obtained during a specific IV&V assessment that was conducted by an independent third party?

May be DelegatedMay be Delegated

18

Stage Gate G - Test

Purpose: The Test Stage Gate Review evaluates whether the project should proceed to the Implementation Phase.

Example Test Assessment Questions:Example Test Assessment Questions:

Has the final implementation plan been developed?

Does the implementation plan describe how the business product will be installed, deployed, and transitioned into the operational environment?

Was a summary report created at the end of the test phases that completely documents the overall test results, including summarizing the test activities and describing variances?

Were any applicable additional tests conducted to validate documentation, training, contingency plans, disaster recovery, and installation?

May be DelegatedMay be Delegated

19

Stage Gate H – Operational Readiness Review

Purpose: The Operational Readiness Review (ORR), is a formal inspection conducted to determine if the final IT solution or automated system/application that has been developed or acquired, tested, and implemented is ready for release into the production environment for sustained operations and maintenance support.

Example Operational Readiness Review Questions:Example Operational Readiness Review Questions:

Has a system certification, including management, operational, and technical security certifications, ensures compliance with information security requirements been successfully completed?

Has the Operations & Maintenance Manual been updated based on results from the Test Phase?

Does the training plan adequately describe the goals, learning objectives, and activities of the information that has been implemented?

Have all stakeholders been notified of the implementation, including information on the schedule, the benefits, the changes, and the impact on end-users?

IR Must PerformIR Must Perform

20

Stage Gate I – Operations & Maintenance

Purpose: The Operations & Maintenance (O&M) Stage Gate Review evaluates whether the project should be released into the full-scale production environment for sustained use and operations/maintenance support.

Example O&M Assessment Questions:Example O&M Assessment Questions:

Does the plan for the Annual Operational Assessment include a review of the CPIC evaluation and the performance metrics during operation to determine whether the business product is meeting original user requirements and any new requirements or changes?

Is the annual system re-accreditation up-to-date, fully documenting the official management decision to authorize continued operation of an information system?

Have the operations manual, business case analysis, and contingency/disaster recovery plan been updated as required?

Does the privacy impact assessment (PIA) for all systems associated with this project/investment include information on any changes?

May be DelegatedMay be Delegated

21

Stage Gate J - Disposition

Purpose: The operation of an automated system or other IT solution is formally ended in accordance with organization needs and pertinent laws and regulations.

Example Disposition Assessment Questions:Example Disposition Assessment Questions:

Are the Project Archives that preserve vital information, including both documentation of project execution and the data from the production system, appropriately preserved?

Is Lessons Learned included in the Project Archives?

Has a final phase-end review been conducted after the system retirement to ascertain if the system and data have been completely and appropriately disposed of?

Are completed contracts closed appropriately?

May be DelegatedMay be Delegated

22

CDC Stage Gate Review Plans

CDC projects will be integrated into the EPLC Stage Gate Review process by September 2009

• Current investments are being reviewed to determine the active projects with their cost and schedule baseline

• Projects are being assessed to determined their “life cycle position” and will be assigned to the appropriate level of CDC IR Governance

• Begin stage gate reviews for major and tactical projects in January & February 2009

• Begin stage gate reviews for all other projects in April 2009

Projects will have one stage gate review by September 2009 and the required reviews for CPIC

23

Summary

1. Stage Gate Reviews are an integral component of the EPLC

2. Coordinating Centers, Coordinating Offices & Offices of the OD must have a IR governance structure and critical partners identified

3. There is latitude for deciding who does the 6 gates that can be delegated

4. Tools and training are being prepared to assist IR governance structures and critical partners in establishing processes and achieving required competencies

5. All projects at CDC should have at least 1 Stage Gate by September 30, 2009

24

Questions & Answers