1A Case Study of Earned Schedule to do Predictions

By Lewis HechtConsultant in EVMS

Vestal, NY 13850607-765-2731

LCHECHT@AOL.COM

2Agenda of Presentation Introduction to earned schedule Summary of technical references Our case study Defense Contract Management Agency concerns Mathematical modeling details Review of contractor performance Prediction accuracy Summary and conclusions Addendum:

Bio of the author

3Introduction to earned schedule-1

Earned schedule methodology was developed by Walt Lipke in 2003. Kym Henderson from Australia has published additional work in 2004 and 2005.

Traditional independent estimate at completion formulas do not predict the final duration of a contract.

4Introduction to earned schedule-2

In traditional EVMS analysis, cumulative schedule variance is zero at the end of the period of performance by definition.

Lipke proposed new measurements to track a contractors schedule performance. These are SV(t) schedule variance with respect to time, and SPI(t) schedule performance index with respect to time.

5Introduction to earned schedule-3

Henderson in 2003 suggested using the concepts PD (planned duration) and PDWR (planned duration of work remaining) as predictors for an independent estimate of completion date for the contract.

Jacob and Kane have published additional earned schedule concepts in 2004.

The IPM conference in the last 2 years has had additional papers supporting ES concepts.

6Summary of technical references-1

Earned Schedule: A Breakthrough Extension to Earned Value Management,Proceedings of PMI Global Congress Asia Pacific 2007, January 2007 by Kym Henderson.

Why EVM Is Not Good for Schedule Performance Analyses (and how it could be), The Measurable News, Winter 2006-2007, by Radenko Corovic.

Applying Earned Schedule to the Critical Path and More, The Measurable News, Fall 2006, by Walt Lipke.

A Simulation and Evaluation of Earned Value Metrics to Forecast Project Duration, Journal of Operational Research Society, September 2006, by Mario Vanhoucke & Stephan Vandevoorde.

Can You Tell the Time?, Journal of the Association of Project Management, August/September 2006, by Mick Higgins.

7Summary of technical references-2

A Comparison of Different Project Forecasting Methods Using Earned Value Metrics, International Journal of Project Management, May 2006, by Stephan Vandevoorde & Dr. Mario Vanhoucke

Earned Schedule in Action, The Measurable News, Spring 2005, by Kym Henderson.

"Further Developments in Earned Schedule,The Measurable News, Winter 2004, by Walt Lipke.

"Further Developments in Earned Schedule, The Measurable News, Spring 2004, by Kym Henderson.

"Earned Schedule: A Breakthrough Extension to Earned Value Theory?, A Retrospective Analysis of Real Project Data, The Measurable News, Summer 2003, by Kym Henderson.

"Schedule is Different,The Measurable News, March & Summer 2003, by Walt Lipke.

8Our case study-1 Purpose of this contract was to build a second avionics

maintenance trainer for Navy helicopters Navy decided to implement schedule incentive to expedite

deployment constraints with moving previous maintenance trainer from East Coast to West Coast

Problems encountered by the contractor in first trainer build effort were ignored deliberately

Navy customer wanted DCMA to generate predictions as to when the contractor could make the enhanced schedule delivery date

This contract was picked to test out the successful prediction capability of earned schedule methodology

9Our case study-2

Contract start date was 12/29/2003 Contract end date was 11/16/2005 Major milestone was shipment by 7/1/2005 Schedule incentive was substantial for

shipping earlier than 7/1/2005

10

Defense Contract Management Agency Concerns

No lessons learned by contractor on this contractprevious experience in similar contracts was ignored by contractor program management

CSSR data not accuratetoo many serious mistakes made for the first three months in cost reports

Traditional EV analysis not able to predict long term schedule issues to customers satisfaction

Contractor decision on work package structure not acceptable

Contractor used work packages whose duration was for entire period of performance for this contract

Too many changes of key contractor personnel

11

Mathematical Modeling Details

Earned schedule calculator was used from ES website to predict completion date

Running comparison was kept between contractors LRE and estimated delivery date vs. DCMAs independent estimates

All formulas were from ES website

12

Review of contractor performance

Plot #1 shows % CV and %SV as parameters of time

Plot #2 shows a typical box plot of % CV and %SV

Plot #3 shows contractors own estimate of schedule slippage vs. DCMAs estimate

13

Traditional EVMS Parameters

-50.00%

-40.00%

-30.00%

-20.00%

-10.00%

0.00%

10.00%

20.00%

Jan 04CSSR

Feb 04CSSR

Mar 04CSSR

Apr 04CSSR

May 04CSSR

Jun 04CSSR

Jul 04CSSR

Aug 04CSSR

Sep 04CSSR

Oct 04CSSR

Nov 04CSSR

Dec 04CSSR

Jan 05CSSR

Feb 05CSSR

Mar 05CSSR

Apr 05CSSR

May 05CSSR

Jun 05CSSR

Time (CSSR report date)

P

e

r

c

e

n

t

a

g

e

c

u

m

%CV

%SV

14

Box Plot of %CV vs %SV

-8.00%

-6.00%

-4.00%

-2.00%

0.00%

2.00%

4.00%

6.00%

-10.00% -5.00% 0.00% 5.00% 10.00% 15.00%

Under cost, ahead of schedule

Under cost, behind schedule

Over cost, behind schedule

Over cost, ahead of schedule

15

Comparison of estimated days behind schedule

-120

-100

-80

-60

-40

-20

0

Months Jan 04 thur June 05

D

a

y

s

b

e

h

i

n

d

s

c

h

e

d

u

l

e

Contractor's estimate behind schedule(days)

DCMA estimated behind schedule(days)

Linear (DCMA estimated behindschedule (days))

Confidence factor for linear regression line on DCMA data is R squared = 0.598

16

Prediction Accuracy-1

Actual history of contract schedule performance was that 1) contractor worked multiple shifts and with

extra help to keep delivery date within 10 day slip in last 2 months of final contract work

2) shipment was actually 7 days late 3) contractor did not win any schedule

incentives

17

Prediction Accuracy-2

Traditional EV analysis would state that contractor was under staffed and running behind schedule

Due to changes in key contractor personnel, a learning curve had to be factored into schedule performance considerations

18

Prediction Accuracy-3

DCMA estimates of completion date were much more realistic than contractor.

DCMA, acting for benefit of Navy customer, forced contractor to implement recovery plan and to speed up work effort to make delivery

19

Summary and conclusions

Earned schedule proved to be very useful in predicting how much work remained to be completed.

US Navy customer was very satisfied with ES predictions

20

Addendum: Bio of the author

BS in physics from Ohio State; MS in physics from Rensselaer Poly Tech

20+ years in electronic packaging technology, mainly at IBM Microelectronics

6 years at Defense Contract Management Agency as GS 12 electronic engineerretired in February 2007

Certified level 2 in Defense Acquisition Workforce Improvement Act education requirements

EVMS expert on Navy helo programs for DCMA at that office

CD-ROM Opening ScreenIntroductionTable of ContentsSearchHelpPractice SymposiaPS 02 Introduction to the NDIA EVM Standard and GuidesPS 03 Recognizing EVM GamingPS 04 A Success- Oriented and Collaborative Approach to EVM System ValidationPS 05 Interfacing Earned Value Management and Risk Management ProcessPS 06 Efforts to Improve Air Force Cost AnalysisPS 07 A Case Study of Earned Schedule to do PredictionsPS 08 NDIA EARNED VALUE MANAGEMENT SYSTEMS APPLICATION GUIDEPS 09 Statistical Methods Applied to Project ManagementPS 10 Department of Defense Earned Value Management Policy and InitiativesPS 11 Lessons Learned in Implementing and Certifying and EVMS in Europe On the Presidential Helicopters ProgramPS 12 What is New with EVM at NASA?PS 13 Relationship Diagramming MethodPS 14 NDIA EVMS Intent GuidePS 15 Adoption the Real Challenge in Implementing EVMSPS 16 Lessons Learned in Implementing EVM within the Federal EnterprisePS 17 Exploration of Cultural ChangePS18 Electronic Facilitation of NASA Integrated Cost Risk for Improved Cost ManagementPS 19 Implementing EVMS in a Dynamic Project EnvironmentPS 21 FAA Portfolio/ Program Performance MetricsPS 22 The Risk Based Estimate at Completion (EAC)PS 23 Earned Schedule in ActionPS 24 GAOs Cost Assessment Guide & How It Defines EVM Best PracticesPS 25 Applying Earned Value Management to Project Formulation?PS 26 NDIA EVMS System Acceptance GuidePS 27 Achieving ANSI/ ESI Standard 748 Compliance at VA using PrimaveraPS 28 Integrating EVM into Project AssessmentsPS 29 Building a Professional Project Planning and Control CommunityPS 30 Linked Control Account Analysis and the Use of Risk IndexesPS 31 Estimating EVMS Implementation & Execution RequirementsPS 32 EVMS SurveillancePS 33 The Project Cost Variance Model: a Project Management ToolPS 34 Program Office Execution of an Integrated Baseline ReviewPS 35 Earned Value in an Agile EnvironmentPS 36 Using EVM Reporting to Develop Cost Estimating Models: The Value and Pit Falls

Professional Education Program (PEP) (Training Seminars)CPM- 100 Principles of Project Management Special Topics100- A Integration, Scope, Time, and Cost Management100- B Overview of PMI EVM Practice Standard100- C Capital Asset Plan and Business Case (OMB Exhibit 300)100- D Earned Value Management System Surveillance100- E Human Resources and Communications Management100- F Developing and Implementing an Integrated PMIS

CPM- 200 Principles of Schedule Management200- A Critical Path Method Scheduling200- B Introduction to Resource Loading / Analysis Techniques200- C Schedule Status Updates and Analysis Techniques200- D EV Cost and Schedule Performance Management200- E Schedule Analysis Techniques200- F Basic Approach to Cost and Schedule Risk Analysis

CPM- 300 Principles of Earned Value Management300- A EVM Principles and Terminology (Part 1)300- B EVM Principles and Terminology (Part 2)300- C Control System Design (Part 1)300- D Control System Design (Part 2)300- E Establishing an Earned Value Management System (Part 1)300- F Establishing an Earned Value Management System (Part 2)

CPM- 400 Principles of Earned Value Metrics & Analysis400- A Developing and Maintaining the PMB (Part 1)400- B Developing and Maintaining the PMB (Part 2)400- C Understanding Resource Management through EVM400- D Integrated Baseline Review400- E Earned Value Data Analysis Techniques (Part 1)400- F Earned Value Data Analysis Techniques (Part 2)

CPM- 500 Principles of Technical Management500- A Development of the WBS500- B Technical Risk Management500- C Integrating Systems Engineering with EVM (Part 1)500- D Integrating Systems Engineering with EVM (Part 2)500- E Material and Subcontract Issues in EVMS500- F Implementing Technical Performance Measurement

CPM- 600 Principles of Project Integration600- A Integrating the Projects Technical Components600- B Integrating Scheduling and EVM Metrics600- C IMP and Its Impact on IMS600- D Integrating Risk into PMB Development600- E Project Integration: Buyers Perspective (Case Study)600- F Project Integration: Sellers Perspective (Case Study)

Tools TrackTT 04 Effective Assessment Tools EVM and PROJECTRxTT 05 AT& L Knowledge Sharing System (AKSS), Acquisition Community Connection (ACC) and EVM Community of Practice (EV CoP)

WorkshopsWS 01 Validation Reviews: Whats the big deal?WS 02 Earned Value Management for ServicesWS 05 Well now that we have an NDIA Validation Guide, how is it going?WS 06 DCMA Trip Wire Metrics: Preparing and Analyzing