assessment of nctd program management … implementation of the ptc system is complex and dynamic,...

0 icfi.com/transportation © 2013

Prepared for:

Brad Hansen, M.S., PMP Director, PMO Capital Projects

Assessment of NCTD Program Management Framework for Positive Train Control Program Subtask 2: Analysis – Gap Analysis

May 2013


Subtask Objective EXECUTIVE SUMMARY

Identify gaps in the PTC Program project management processes, artifacts, tools and staff resources required for effective management Assess the PTC Program’s Program Management Plan (PMP) compliance with:

– PMI PMBOK – 49 Code of Federal Regulation (CFR) § 633.25 - Contents of a project

management plan – ICF’s experience and lessons learned from analogous programs, tailored to the

NCTD PTC Program environment Review priority components of the PMP, as requested by NCTD:

– Program Management Organization – the organizational structure and associated resources

– Requirements Management – the processes and tools NCTD employs to manage requirements

– Risk Management – the process and tools employed by NCTD to manage programmatic risk

Recommend prioritized improvement areas to NCTD’s program management

approach based on expected benefit to the NCTD organization


Key Themes EXECUTIVE SUMMARY

The implementation of the PTC System is complex and dynamic, placing stress on NCTD’s organizational capabilities

In many instances, the PTC Program’s “current state” activities are inconsistent with the Program Management Plan’s (PMP) documented processes, tools, and structures

Formally documented program-level controls, integrating all PTC-related activities and components, do not exist or are not being exercised

– The program-level controls that do exist are largely specific to the HTI PTC System contract—which represents only 40% of the total program budget


Gap Analysis Results EXECUTIVE SUMMARY

PMBOK Knowledge Area NCTD PTC

Ratings Summary PMP Current State

Integration Management Little evidence of integration processes or tools were observed

Scope Management The entirety of the PTC Program scope is not defined and documented

Cost Management No cost management baseline exists and inconsistencies between PTC Program and NCTD cost management processes are present

Schedule Management Lack of a schedule baseline or a single integrated master schedule

Risk Management Undefined risk management process, but recently instituted project-level processes will enhance Program level risk management

Quality Management An organizational quality capability does not exist, nor is there a Quality Management System

Human Resource Management The current organizational structure is not the most effective approach

Procurement Management Mature organizational procurement capabilities with few needed improvements

Stakeholder Management Level of stakeholder identification, analysis and engagement for a Program of this complexity is not evident

Communications Management Strong organizational capability, though ineffectively integrated into the PTC Program

The color scale reflects the degree of compliance with (i.e. all, most, some, few) the processes, artifacts and tools prescribed within the PMBOK knowledge areas

All

Most

Some

Few

How well does the PTC PMP stack up to PMBOK? How well does the Current State of the Program follow PMBOK?


Recommended Improvements EXECUTIVE SUMMARY

Baseline the Program

Enhance program control

Re-align program organization

Implement a coordinated stakeholder strategy

Dedicate resources to prepare for post-implementation operations and maintenance

2

1

3

4

5

5 icfi.com/transportation © 2013 5

Methodology


Scope of Gap Analysis METHODOLOGY

Evaluation of PTC Program Management Plan

Assessment of Numerous Program artifacts, Program staff interviews, PTC Program organization charts, and observations – all constitute the “current state”

Standards used :

A Guide to the Project Management Body of Knowledge (PMBOK) – Fifth edition (with emphasis on the ten Knowledge Areas)

The Standard for Program Management – Third Edition


FRA§ 633.25 PMP Content Requirements are addressed by PMI’s PMBOK Knowledge Areas

METHODOLOGY

PMBOK Knowledge Area § 633.25 PMP Content Requirement

Integration Management

(a) description of adequate recipient staff organization, complete with well-defined reporting relationships, statements of functional responsibilities, job descriptions, and job qualifications

(d) document control procedure and recordkeeping system

(i) plan for internal reporting requirements including cost and schedule control procedures

Scope Management (e) change order procedure which includes a documented, systematic approach to the handling of construction change orders

Cost Management (b) budget covering the project management organization, appropriate consultants, property acquisition, utility

relocation, systems demonstration staff, audits, and such miscellaneous costs as the recipient may be prepared to justify

Schedule Management (c) construction schedule

Risk Management Not required (but recommended, by the assessment team)

Quality Management

(g) quality control and quality assurance programs which define functions, procedures, and responsibilities for construction and for system installation and integration of system components

(h) material testing policies and procedures

(j) criteria and procedures to be used for testing the operational system or its major components

Human Resource Management (f) description of organizational structures, management skills, and staffing levels required throughout the construction phase

Procurement Management Not required (but recommended, by the assessment team)

Stakeholder Management Not required (but recommended, by the assessment team)

Communications Management Not required (but recommended, by the assessment team)


Rating METHODOLOGY

The color scale reflects the degree of compliance with (i.e. all, most, some, few) the processes, artifacts and tools prescribed within the PMBOK knowledge areas

All

Most

Some

Few

PMP Current State

Assesses the PMP against PMBOK Knowledge Areas

Assesses the Program’s “current state” processes, artifacts and activities against PMBOK Knowledge Areas

Our color coded scale provides a rating for each knowledge area:

Ratings are provided for the PMP and the “current state”


Gap Analysis Results


“… includes the activities needed to identify, define, combine, unify, and coordinate multiple components within the program.”

Integration Management GAP ANALYSIS RESULTS

Rating: PMP Current State

Analysis Identified Gaps Several key integration-related artifacts exist

– PMP, last revised on March 22, 2012 – Project charter – Organizational charts – Program management information systems

Recent project-level standard processes,

templates, and reporting requirements have been implemented and are contributing to integration management

Inconsistencies between the “current state” organizational structure and the structure defined in the PMP

Documented process and tools that integrate Program-level scope, schedule, schedule and budget (i.e., no Integrated Master Schedule, Program WBS, or time-phased budget)

Many current state activities do not follow processes documented in the PMP

Process for maintaining the PMP

Process for program-level monitoring and control

Program Charter lacks clear articulation of the purpose, scope, budget, schedule and roles/responsibilities

Process for integrated change control (scope, schedule, cost, requirements, system configuration


“… defines the work required to deliver a benefit (major product, service, or result with specified features and functions) [and] includes all of the activities involved in planning and managing the program’s scope.”

Scope Management GAP ANALYSIS RESULTS

Analysis Identified Gaps System requirements for PTC are provided in

49 Code of Federal Regulation (CFR) § 236.1005 – Requirements for Positive Train Control systems – Additional due diligence is required to ensure

PTC Program requirements are compliant with this CFR

The PMP contains elements of scope

management including, but are delegated to HTI: – Contract WBS – Scope change control process – Requirements management and traceability

A scope statement that defines the entire PTC Program – (note: an “original Scope Statement” was

referenced in the materials reviewed during the Discovery subtask, but not found)

A Program-level Work Breakdown Structure (WBS) to include ALL Program work (vendor-implemented and NCTD-implemented projects and tasks)

Control of Program scope

Program-level requirements management and traceability for all PTC Program projects



“… identifying the program’s financial sources and resources, integrating the budgets of the program components, developing the overall budget for the program, and controlling costs throughout the duration of both the components and the program.”

Cost Management GAP ANALYSIS RESULTS

Analysis Identified Gaps Total program budget of $87.3M is funded

through nine sources

The program’s contingency fund decreased by 70% during the period from July 2011 through January 2013. – It is not clear whether the remaining

contingency funds will be sufficient

There are differences in the budget management practices used by the Program and the NCTD Finance Division, resulting in at least $21.2M (~24% of the total program budget) of funds that cannot be currently reconciled by the Finance Division

Program cost management processes do not reconcile with organizational cost management practices (15 accounts versus 5 accounts)

Post-implementation operations and maintenance cost estimates have not been documented - current estimates appear to be anecdotal and not analytically based

Cost management process




“… includes determining the order in which the individual components are to be implemented, the roadmap for the program, and the milestones to be measured to keep the overall program on track and within defined constraints.”

Time Management GAP ANALYSIS RESULTS

Analysis Identified Gaps Activities of individual contracts and certain

Program elements appear in multiple schedules

Project Managers are assigned to monitor contractor progress against project-level schedule baselines

The PMP defines an approach to schedule management using a three-level hierarchy, each with a different level of detail - NCTD is not executing the approach

A Program-level schedule baseline

A single integrated master schedule that encompasses ALL Program activities Dependencies Predecessor/successor relationships Critical Path Schedule hierarchy

Schedule management process

Objective basis for NCTD’s ability to achieve

the December 31, 2015 completion date



“… includes the processes of conducting risk identification, analysis, and response planning in order to reduce the likelihood of occurrence and/or impact an individual, or series of risks, have on a program.”

Risk Management GAP ANALYSIS RESULTS

Analysis Identified Gaps There are currently four versions of risk

registers within the program, some more current than others

Risk management discussion in the PMP and the Program Risk Management Plan have major inconsistencies

Project-level risk management plans and processes have recently been implemented

A documented risk management process to ensure risk management consistency within all Program elements

A single integrated risk register

Risks are currently assessed and prioritized using only qualitative measures


“… includes the activities of the performing organization that determines program quality policies, objectives, and responsibilities so that the program will be successful.”

Quality Management GAP ANALYSIS RESULTS

Analysis Identified Gaps Current quality management processes in the

PMP are specific to the HTI contract

NCTD is required to document “a complete description of the safety assessment and Verification and Validation processes applied to the PTC system” in an FRA-mandated PTC Safety Plan (PTCSP) prior to FRA-testing of the system – Rail Safety Consulting LLC is developing

NCTD’s PTCSP

Program-level quality management processes as part of a Quality Management System

Clarity as to who is accountable within the PTC Program or NCTD organizations for quality management

In-house staff accountability and capability for program quality management



“… includes the process that organize, manage, and lead the project team—comprised of the people with assigned roles and responsibilities for completing the project.”

Human Resource Management GAP ANALYSIS RESULTS

Analysis Identified Gaps The program manager has authority to assign

priorities, apply resources, and direct the work of individuals assigned to the project

The PTC Program organization is largely geared to oversee the work of contractors and consultants – is this precluding attention to internal work elements?

Five Project Managers oversee 15 projects (contracts)

The organizational chart in the PMP does not match 1) the associated narrative description or 2) NCTD’s organizational chart

The PMP’s listing of contractor and consultant resources is incomplete and does not match the contents of the PTC Vendor List

Job skills, experience, and qualifications of personnel fulfilling positions within the PTC program are not documented – This makes it difficult to assess whether the

PTC Program has been appropriately and adequately staffed



“… addresses specific procurement needs that are unique to managing the overall program and the needs of the constituent projects/components.”

Procurement Management GAP ANALYSIS RESULTS

Analysis Identified Gaps An NCTD Contract and Procurement

representative is ‘matrixed’ to the program and provides contracting/procurement support

Procurement management processes are integrated with, and leverage, NCTD’s organizational procurement capabilities

Processes associated with formally closing out each contract after all deliverables and payments have been made are not defined

The PTC Vendor List does not include “all vendors and suppliers of each PTC Program system, subsystem, component, and associated product, and process in use system-wide.”



“… includes the activities associated with identifying, categorizing, and tracking all stakeholders who will interact with the program or be affected by the implementation of the program.”

Stakeholder Management GAP ANALYSIS RESULTS

Analysis Identified Gaps PMP identifies three stakeholder groups:

– Communities – Media Outlets – Government Entities

Preliminary analyses conducted by NCTD’s

Government Affairs and Communications Division identified 15 stakeholder groups in three categories: – Internal – General Public – Targeted

A dedicated PMP section for stakeholder management

A complete list of stakeholders to manage

Coordination and collaboration between the PTC program and NCTD’s Government Affairs and Communications Division—this does not appear to take full advantage of the agency’s capabilities




“… includes the activities for facilitating timely and appropriate generation, collection, distribution, storage, retrieval, and ultimate disposition of program information.”

Communications Management GAP ANALYSIS RESULTS

Analysis Identified Gaps The approach outlined in the PMP is largely

focused towards external communications

NCTD has a robust internal communications capability that has developed key communication-related elements regarding PTC

Stakeholder management and communication is a critical aspect of the PTC Program and its success

The PMP fails to address how communications will be managed and disseminated to internal stakeholders regarding the status of the program

Lack of internal communication, resulting in an insufficient level of understanding and engagement within the PTC Program among project team members


Prioritized based on expected benefit to the NCTD organization

Recommendations for Improvement


Recommended Improvements RECOMMENDATIONS FOR IMPROVEMENT

Baseline the Program

Enhance program control

Re-align program organization

Implement a coordinated stakeholder strategy

Dedicate resources to prepare for post-implementation operations and maintenance

2

1

3

4

5



Scope: Identify all work that’s required to be accomplished by the PTC Program and develop an associated program-level WBS

Cost: Reconcile current cost estimates and resource requirements with the newly developed WBS; develop risk-based budget contingencies

Schedule: Develop a program-level integrated master schedule that encompasses all currently-identified tasks



Evaluate and implement a program management information system sufficient for the size and complexity of the PTC Program

Review and update the Risk Management plan (e.g. implement an integrated master risk register that includes quantitative analyses)

Revise the PTC’s Quality Management plan so it aligns with the future PTCSP and newly established Quality function within NCTD

Identify and define configuration management requirements specific to the PTC Program

Implement program-level variance analysis and reporting mechanisms

Implement a process for maintaining and updating the PMP for the duration of the program



Strengthen the organizational relationship between the PTC Program and NCTD’s Project Management Oversight Office, Capital Projects

Establish an independent Quality management capability within NCTD

Formalize cross-organizational representation within PTC Program

Establish formal job descriptions that include skills, experience, and qualification requirements for each role within the PTC Program organization

Reissue an updated Program Charter to all NCTD staff that clearly highlights the purpose, objectives, governance structure, and roadmap for the PTC Program



Review and expand upon the stakeholder analysis currently drafted by NCTD’s Government Affairs and Communications Division

Implement a coordinated internal and external communications strategy based on the updated stakeholder analysis

Explore opportunities to link the PTC outreach strategy with Board of Directors and external project champions engagement



Document on-going maintenance requirements for PTC-related components and ensure adequate maintenance plans are developed, implemented, and the appropriate personnel trained

Identify additional organizational training needs, for example: – Locomotive operators: work with Herzog to understand and document initial and

recurrent training plans – Maintenance of Way: determine which PTC-related components require

familiarity with and understanding of by MOW personnel

Plan, document, and integrate newly implemented PTC-related components into NCTD’s configuration management capability

Develop and document one-time, on-going, and incremental operating costs related to PTC and implement processes for reviewing and revising these costs (and assumptions) on a regular basis.


Next Steps


Plan of Action NEXT STEPS

To include: – Implementation plan for the recommendations identified in this task – Current and proposed project management organization chart

assessment of nctd program management … implementation of the ptc system is complex and dynamic,...

Documents