Schedule Assessment and Risk Analysis

Richard M. GreathouseLead Programmatic Analysis Group

Heidemarie S. Borchardt, PMPLead Schedule Analyst

Contributors: Kelley Cyr, Michele King, Kelly Moses, Robin Smith

Independent Program Assessment Office (IPAO)

Overview

• The schedule is a powerful tool for project management. Schedule risk analysis: Identifies potential risks Identifies impacts of risks to project schedule

• IPAO has the opportunity to meet with many programs and projects: Lessons Learned Future Plans

2

IPAO Forensics AnalysisExcerpt

Strengths:• Risk management, budget, and schedule tend to gravitate to very few

(typically one) overall strengths for each program. • Resources Other Than Budget received the fewest strengths.

Issues:• Schedule issues were the most frequent issues identified and may be an

opportunity for improvement.– Schedule issues may be caused by poor planning, poor schedule

(duration) estimates, or an overall lack of emphasis on good schedule practices.

– Insufficient time may have been allowed to achieve Level I milestones, perhaps complicated by insufficient funding.

3

IPAO Forensics Analysis performed byKeith Burleson, 2008-2009; 2009-2010

IPAO Schedule Analysis Results

• IPAO schedule analysts typically have to amend the schedules in order to be able to assess the schedule.

• Schedules received for assessment are typically not of good quality due to poor scheduling practices. Missing predecessors and successors Containing constrained data Risks are not mapped to the schedule Lack of a true critical path Inappropriate use of lags/leads hides actual work being done which

does not allow the schedule to be properly loaded with resources• NASA Schedule Management Handbook and NPR 7120.5D (NID), NM

7120-81, and NPR 7123.1• Schedule management may be the area that presents the best

opportunity for improvement -- There are immediate actions we can take to improve!

4

Improving Schedule Management

• Schedule Health/Quality Check

• Schedule Analysis vs. Reporting

• Schedule Risk Analysis

Schedule Health/Quality CheckDCMA 14 Point Assessment

An Integrated Baseline Review (IBR) will assess the metrics outlined in the DCMA 14 point assessment. The biggest areas for improvement are in logic and constraints.

GAO Criteria

Government Accounting Office (GAO) Scheduling Best Practices

Metric DefinitionScope of Work Capturing all Activities - WBS

Logic Network Sequencing all activities – All activities have a predecessor and successor excepting the start and finish activity.

Resource Loaded Schedule Assigning resources to all activities – labor, material, and overhead

Durations Establishing the duration of all activities – Basis of Estimate to include resources

Schedule Integration Integrating schedule activities horizontally and vertically

Critical Path Method Establishing the critical path for all activities – Project master schedule and critical path are defined

Float/Slack Identifying reasonable float between activities – calculated by schedule analysis

Schedule Risk Conducting a schedule risk analysis

Statusing Updating the schedule using logic and durations to determine the dates – not constraints; Schedule analyzed continuously to determine changes in critical path and variances

Baseline Schedule (new metric) Creating and managing a baseline schedule

7

GAO audits many of the programs and projects that IPAO assesses. It is important to understand and adhere to their criteria.

GAO is in the process of developing a Schedule Estimating BestPractices Guidebook – Draft Release March 2011

GAO Cost Estimating and Assessment Guide, GAO-09-3SP

Schedule Health/Quality CheckNASA STAT Tool

8

STAT is a NASA-developed schedule integrity check tool for MS Project-based schedules that aids in the development, assessment, analysis and reporting of project schedules. 

Many NASA schedulers do not seem to be aware of the STAT tool. This analysis should be performed as part of the update cycle.

To get your copy of the STAT software and a registration form, please contact Ken Poole, MSFC, at Kenneth.W.Poole@nasa.gov256.544.2419

Schedule Analysis

9

The schedule is an informative management tool. It should be used for analysis as well as reporting actual status.• Critical Path Analysis• Schedule Performance Trend Analysis• Baseline vs. Current Comparison Analysis – Variance• Resource Analysis• What-if Analysis• Earned Value Management (EVM) Analysis• Schedule Margin Assessment• Schedule Risk Analysis• Joint Confidence Level (JCL) Analysis

Reference: NASA/SP-2010-33403NASA Schedule Management Handbook

Critical Path Analysis

Many projects will report what the critical path is; however, the reported path is not reflected in the project and in some cases is a misrepresentation of the true critical path. It is imperative that the critical path come from the IMS and that it and the other merging paths are properly managed.• Critical Path – “the sequence of schedule activities that determines the

duration of the project.” PMI• Importance of C/P Analysis – helps determine which activities can impact

the project completion date (excludes LOE tasks). Understand schedule drivers.

• The critical path needs to be monitored as it can change. Monitoring the critical path will show the project manager where resources are required to prevent schedule delays. Activities with higher slack values can be postponed in order to work activities that must be completed.

10

Schedule Performance Trend Analysis

Types of Analysis:• Schedule Performance Work-off – compares baseline

dates against actual and forecasted dates• Schedule Baseline vs. Actual Finish• Schedule Performance Efficiency (Baseline Execution

Index)• Projection of Actuals Based upon Schedule Performance• Total Slack Trends• Baseline vs. Current Comparison Analysis

Schedule Milestones Schedule Variance Reports

• Earned Value Management

11

Studies show that once a schedule slips more than 15%, it cannot get back on track.

Resource Analysis

Resource loading of schedules is a tool to manage resources effectively.• Are the right people, material, and facilities available to do the

job when needed?• Workforce planning is dependent on the project’s schedule -

Brooks’s Law. • Most NASA schedules are resource and time constrained.• Cost and schedule analysts review how the resources are

loaded or allocated in the IMS that supports the JCL Time dependent/variable (i.e. program management

staffing) Time independent/fixed (i.e. material costs, resources)

12

What-if Analysis

The schedule is a powerful management tool, as well as, a reporting vehicle.• Tool for assessing potential schedule problems and

mitigations.• Used to plan for corrective actions.

13

Schedule Margin Assessment

Margin needs to be applied to critical path and high risk activities.

• Margin – “a provision in the project management plan to mitigate cost and/or schedule risk.” PMI

• Margin is not float/slack. Float/slack only appears on non-critical paths; critical paths for NASA projects tend have 0; cost reserves can be applied to both float/slack on a given path as well as schedule margin on the critical path and it is important to know the difference.

• Margin can be moved around the schedule as needed for high risk activities – if this is not the case, for instance, if the time is allocated to working a specific task/risk mitigation, it is no longer considered margin.

• A probabilistic risk assessment is recommended as a basis for determining required margins.

14

Schedule Risk Analysis

The schedule is a powerful tool for assessing and mitigating risks. Project Manager, Risk Manager and Scheduler work together to closely monitor risks and schedule impacts.• Map risk ID to schedule task that is affected by the risk.• Determine if mitigation is required (mitigation tasks,

additional resources, use of schedule margin if on the critical path)

15

Schedule Risk Analysis (SRA) and Simulation - GAO

1. Does the schedule reflect all work to be completed?

2. Are the P/p critical dates used to plan the schedule?

3. Are the activities sequenced logically?

4. Are activity interdependencies identified and logical?

5. If there are constraints, lags, and lead times, are they required, and is documentation available to justify the amounts?

6. How realistic are the schedule duration estimates?

7. How were resource estimates developed for each activity and will be the resources be available when needed?

8. How accurate is the critical path and was it developed with scheduling software?

9. How reasonable are float estimates?

10. Can the schedule determine current status and provide reasonable completion date forecasts?

11. What level of confidence is associated with the P/p schedule completion date? Does it reflect a schedule risk analysis and the organization’s or stakeholder’s risk tolerance?

GAO-09-3SP Appendix X

Project Inputs

Projects deliver products at SR-100. SRB members understand that programmatic documents are works in progress.• Documents (Requested of Project):

Project Management Plan, Risk Management Plan, Schedule Management Plan Work Breakdown Structure (WBS) and WBS Dictionary Resources Plan (staffing, workforce, facilities) Project management status reports and EVM Data

• Best Practice Master Schedule / Integrated Master Schedule (IMS) Project runs NASA STAT health check and schedule should be green for delivery Identification of Critical Path and Schedule Margin/Reserve with basis Removal of constraints from schedule for purposes of performing SRA Provide basis for any constraints not removed from schedule Identification of risk mitigation activities in the schedule Qualitative uncertainty ratings by WBS (Ref. schedule risk consequence chart)

• Risk List Likelihood and Consequence Probabilities for Risks Mapping of Risks to Specific Schedule Activities

• Joint Confidence Level (JCL) and assumptions (if required)

-40-60-100 0

Draft IPA Plan and Methodology; Review

with PAG Lead; Get signatures from IPAO

Director, Review Manager, and PAG Lead

Program/Project (P/p)

ProgramAnalysis

Group (PAG)

Review Programmatic Documents

Brief SRB on Initial Analysis

Communicate with the P/p Regarding IPA

Process

Create SRB /IPA

Briefing and Internal

Peer Review Charts

Create Cost/Schedule Risk Model

Collect Initial SRB Risk and Uncertainty

Assessment

Perform Cost/Schedule Analysis and Work with

Project to Resolve Issues, Identify

Uncertainty, and Map Risks

Review IMS and Work with P/P to resolve any

quality issues

Collect Updated SRB Risk and Uncertainty

Assessment

Apply P/p Risks & Initial SRB Risks and

Uncertainty Values to Model

Apply Updated SRB Risks

and Uncertainty

Values to Model

Run Final SRB Risk Updates and P/p Risk Updates in Model and

Provide Chair with Results

Present Updated IPA Findings to the SRB

Inform PAG Lead of Any Updates to the Model

Results

Complete IPA Report

Independent Programmatic Analysis (IPA) Timeline

-20 -7

* Required if the P/p has a Joint Confidence Level (JCL) Requirement

Update Analysis per Peer Review

Standing Review Board (SRB)

Receive Initial Programmatic Data

Drop from P/p (Master Schedule & Integrated Master

Schedule (IMS))Due at SR-100 days

Receive Updated Data Drop form P/p (Final

IMS Delivery and Initial JCL Model Delivery*)

Due at SR-60 days

Receive Final JCL Model* and

Programmatic Updates from P/p

Due at SR-20 days

Receive Final Risk Updates from P/p

Due at SR

Initial Review

with PAG

PAG Peer

Review

Collect Final SRB Risk and Uncertainty

Assessment

Kickoff Meeting

with SRB

Begin Drafting IPA Report

Brief SRB on Updated Analysis

Continue Updating

IPA Report

Continue Updating

IPA Report

Site Review

(SR)

The project timeline provides an opportunity for the project to deliver a best-practices schedule to be used in performing the schedule risk assessment.

Schedule Risk Assessment Output

The schedule risk assessment is a tool to mitigate project risks:

Schedule Health/Quality Assessment Critical Path Analysis Tornado chart depicting the impact of risks occurring S-Curve depicting the impact of risks on the schedule confidence level Risk Assessed Stochastic Critical Path Frontier Curve showing the schedule or schedule and cost confidence level (or

JCL if required)* Sensitivity Analysis (if needed) implemented to analyze the various risks to the

project by looking at all their potential impact on the overall goal

Future Plans

• Schedule Analysis Working Group• Connect and Discover the NASA Planning & Scheduling

Community of Practice: To obtain access to the NASA P&S CoP contact:Jennifer Poston, Jennifer.L.Poston@nasa.gov (301) 286-7492

• Continuous improvement of IPAO Standard Operating Procedure Instruction (SOPI) which contains cost, schedule, and risk analysis processes.

• Supporting the development of 7120.5E• Visiting NASA centers and providing overview of

programmatic assessment process. Ames – November 2010

• Consistency between reviews and across centers

20

References

• NASA Schedule Management Handbook, NASA/SP-2010-3403, January 2010

• DOD Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide Version 0.9, October 21, 2005

• GAO Cost Estimating and Assessment Guide, GAO-09-3SP

21