11/03/091 integrating cost and schedule estimation in the cosysmor model/tool john e. gaffney, jr....

11/03/09 1

Integrating Cost and Schedule Estimation In The COSYSMOR Model/Tool

John E. Gaffney, Jr.301-721-5710

[email protected] Center For Process Improvement Excellence

Corporate Engineering and TechnologyLockheed Martin Corporation

November 3, 2009

(c) Copyright Lockheed Martin Corporation 2009

11/03/09 2

Focus Issues and Questions-1• An issue of considerable importance to proposal managers, program

managers, technical planners and to software engineering and systems engineering managers is how schedule compression or stretch-out* affects engineering costs or overall project costs

• They are also interested in the opposite question, how engineering costs affect schedule. Indeed, a project might be cost driven or schedule driven or some combination of the two. More generally, managers and other stakeholders cited above are interested in the relationship between cost and schedule (project duration).

• Thus, cost can imply schedule or vice versa.

*Schedule compression (or stretch-out) can be defined as the amount or percentage of reduction (increase) of a project or software or systems engineering schedule with respect to some ideal or nominal value as related to cost or productivity


Focus Issues and Questions-2

• Knowing the relation between cost and schedule can help us to answer questions such as:1. What is the schedule corresponding to this (estimated) cost value?2. What is the (estimated) cost value corresponding to this schedule

value?3. Can cost and schedule be traded off (can one “buy off” the other)?

Sometimes this is possible, and sometimes, it is not

11/03/09 (c) Copyright Lockheed Martin Corporation 2009

3

Background/Reference To Earlier Work

• The approach described here has been implemented in the COSYSMOR (“COSYSMO Risk and Reuse”) tool and the COCOMOR (“COCOMO Risk”) tool developed at Lockheed Martin. COSYSMOR evolved from the COSYSMO tool, developed by Dr. Ricardo Valerdi, and is available to anyone upon request. COSYSMOR was described at the 23rd International COCOMO forum in October, 2008

• More generally, the approach presented applies to estimating models such as COSYSMO and COCOMO that produce cost (labor) estimates as their principal outputs

• This presentation expands upon work described in a talk given at the 23rd International COCOMO forum in October, 2008 by John Gaffney , and in a talk (and paper) given at the 7th Annual Conference on Systems Engineering Research 2009 (CSER 2009), by John Gaffney, Ricardo Valerdi, and Michael Ross. Dr. Valerdi gave the presentation. The former was “Looking At Schedule Vs. Effort In Software Engineering, Systems Engineering and Systems.” The latter was “Approaches to Calculating Systems Engineering Schedule in Parametric Cost Models”


4

(c) Copyright, Lockheed Martin Corporation, 2008

5

Concerning The “Risk” Aspect of COSYSMOR COSYSMOR provides the ability to estimate Cost/Effort and Schedule Uncertainties/Risk and Confidence: Provides quantification of the impacts of uncertainties in the values of key model parameter values. Provides multiple cost and schedule values with associated probabilities.

Risk=Prob [Actual Effort Will Be >Estimated Effort]Confidence=100%-Risk

Approach Summary

Implemented in COSYSMOR (and COCOMOR):1. Estimate cost (labor) using COSYSMOR. Call it K1.

2. Obtain the value of schedule, call it T1, that corresponds to K1.

3. The person doing the estimate decides whether T1 is satisfactory. It may or may not be, depending on: project requirements, the desires of management, marketing, and/or other authorities.

4. If T1 is not satisfactory (perhaps within some tolerances), but T2 is the desired value, then determine the value of K, effort, call it K2, that corresponds to the desired schedule value, T2.

• Perform these steps at one or more risk levels, e.g., 80%,of interest, say the risk level at which you desire to make a proposal or at which the project is to be managed

• The values of the parameters in the relationships used to obtain these values (see next several pages) are obtained as part of the tool calibration process


6

Details of Step 2: Determining T1 As Function of K1 • Objective: Determine T1, the ”natural” schedule corresponding to K1

estimated by COSYSMOR at some risk level.• Use Function: T1=b*(S1

v)*(K1z); S=size (“equivalent requirements” in

COSYSMOR and “equivalent new SLOC” or ESLOC in COCOMOR).– The parameters b, v, and z are obtained by regression*– In the example, b=2.54683,v=-0.0300,z=0.315;R2=0.7664

• This function, as it includes the effect of the variable S, is preferred over the perhaps more familiar form:

T1=c*K1d

-The parameters c and d are obtained by regression-In the example, c=2.6333,d=0.2928;R2=0.7897

* Note : The data sets used in the regressions to determine the values of the parameters are preferably are from the domain for which the tool is being used, as is the case for the other parts of calibration


7

Details of Step 4: Determining K2 When T1T2; 1 of 2

• Use Function: S=A*(Kp)*(Tq)• Function is of the form of a Cobb-Douglas Production function• The variables K and T are “factors of production” and may enable

tradeoffs to be made between K and T, effort and schedule • This function characterizes the development process performance of

the organization performing the project that is being estimated. It is a “process performance model.” *

• In the example, a=0.086298, p=1.0990, q=-0.301; R2=0.8081 * A Process Performance Model (PPM) is a description of the relationships among attributes of

a process and its work products that are derived from historical process performance data and calibrated using collected process and product measures from the project and that are used to predict results to be achieved by following a process. The reference for this definition is the CMMI-DEV® v1.2 glossary. CMMI ® is a registered Trademark of Carnegie Mellon University.


8

Details of Step 4: Determining K2 When T1T2; 2 of 2

• Consider two instances: <S1,K1,T1> and <S2,K2,T2>; • Write: S2/S1 and then for S1=S2, we have: 1=((K2/K1)p)*((T2/T1)q) . • This can be rewritten as: ((K2/K1)p)=1/((T2/T1)q) and setting q/p=r, we obtain: (K2/K1)=1/((T2/T1)r ;• Recalling p=1.0990,q=-0.301, the r=q/p=-0.273885• Thus, to estimate the affect on K , effort, if the schedule were to be

changed from T1 to a more desirable value, T2, use this relationship to obtain the “new” value of K,K2; that is: K2=K1/((T2/T1)r;


9

COSYSMOR Effort/Schedule Inputs and Outputs

10

Systems Engineering Schedule/Effort Relationship Estimator,SCHEDESTSE

1. Enter calendar Weeks per Month that you wish to use to transform a schedule given in months to one given in weeks.

Weeks per Month 4.3

2. Estimate baseline schedule. Typically, this is one corresponding to one of effort values in the range of such values that you estimated using COSYSMOR. Enter Person Hours value and Size,Equivalent Requirements value and the tool provides the corresponding schedule.Equiv Reqmts Person Hours Schedule (Months) Schedule (Weeks) Unit Effort (PHrs /EReq) Productivity (EReq/100 PHrs)

632 6219.4 32.9 141 9.8 10.2

Processing In Box 1.5170

3. Decide if the schedule value calculated above is satisfactory. If so,go to step 5.Else, if it is not satisfactory, go to step 4.

4. If you desire a different ("better") schedule, enter the desired value here, and obtain the corresponding person hours estimate. It is suggested that the "better" schedule be within the range of +/-50% that of the baseline schedule, K1 in the table below, that was calculated in Step 2 above. Base estimated schedule, T1= 32.9 Months 141 Weeks Better schedule, T2= 25.0 Months 108 Weeks And, given that K1= 6219 Person HoursTherefore, corresponding effort, K2= 5769 Person Hours

Therefore, T2/T1= 76.02% Therefore, K2/K1= 92.77%

Project User 1 of 3: User Data Entry

11

Project User 2 of 3 Shows relationship between K and T about baseline

12

Table SCHEDEST 1 Table SCHEDEST 2 For Baseline Effort = 6219

T2/T1 K2/K1 and Schedule= 32.950.00% 82.71% T, Schedule (Months) K, Effort (Person hours)

60.00% 86.94% 16.4 5144

70.00% 90.69% 19.7 5407

80.00% 94.07% 23.0 5641

90.00% 97.16% 26.3 5851

100.00% 100.00% 29.6 6042

110.00% 102.64% 32.9 6219

120.00% 105.12% 36.2 6384

130.00% 107.45% 39.5 6538

140.00% 109.65% 42.8 6683

150.00% 111.75% 46.0 6820

49.3 6950

The tool produces these tables and the corresponding plots onthe next page based on the model parameters derived from regression as previously described and on the values entered by the user on the data entry (previous) page

% Systems Engineering Effort Change,K2/K1 Versus % Schedue Change,T2/T1

0%20%40%60%80%

100%120%

50% 60% 70% 80% 90% 100% 110% 120% 130% 140% 150%

T2/T1 Percent

K2/K

1 Pe

rcen

t

K, Effort (Systems Engineering Person Hours) Versus T, Schedule (Months), For Baseline K and

T 6219 and 32.9

5000550060006500700075008000

20.0 25.0 30.0 35.0 40.0 45.0 50.0

T, Months

K, P

erso

n Ho

urs

Project User 3 of 3

13

Show plots of relationship of K and T about baseline, same informationas in tables on previous page.

Summary• Determining the relationship between schedule and effort is

important for project planning and management– A project may be effort (cost) or schedule driven or some combination of

both• COSYSMOR can provide such a relationship

– This enables sensitivities in effort/ and schedule to be evaluated• A production equation of the form: S=A*(Kp)*(Tq)

is employed as part of the tool calibration process– The production function defines the ability of the organization for a

certain domain to produce S as a function of the “factors of production,” K and T.

– The methodology can be used in other venues, such as software development and for businesses more generally, say to produce a certain sales volume as a function of labor applied and invested capital

05/14/2311/03/09 14

Backup

15

11/03/09 16

Cobb-Douglas Production Function• The Cobb-Douglas Production Function relates an output of a process to

factors of production– An example is a C-D P F that relates software source statements, S, produced

by a software development process, to factors of production, such as labor, K, and time or project duration, T

– General Form: O=A*(F1Q1)*(F2

Q2)*…..*(FNQN); O=Output; the Fi are the factors of

production• Examples of Use of the Production Function Form

– Inputs: labor and capital; output=automobiles. From this form can be derived a “productivity,” actually a “unit cost,” labor hours per automobile. Notice the indicated possible tradeoff between capital (appropriately applied in terms of training and technology) and labor; more capital less labor to obtain the same output

– Inputs: invested capital, labor; output=$ profit From this form can be derived a “productivity,” return-on-invested capital”

– Inputs: invested capital, labor; output=$ sales From this form can be derived a “productivity,” “sales per employee”

– Inputs: inspection time, meeting time, amount of material inspected; output= number of defects discovered. Can use as a basis for “tuning” the inspection process and for identifying opportunities for process improvement


11/03/09 17

Some Software Engineering and Systems Engineering Examples-1

• General Form: S=A*(Kp)*(Tq)• Some Actual Cases That Exemplify Different Situations That Can

Be Experienced:– 1.S=A1*(K0.6288)*(T0.5555); q/p=0.8831– 2.S=A2*(K0.3333)*(T1.3333); q/p=4.0000– 3.S=A3*(K0.929)*(T0.079); q/p=0.0807– 4.S=A4*(K1.0990)*(T-0.301); q/p=-0.273885

• Observations Concerning These Equations:– Case 1,Examples 1&2 :If q&p both >0; K,T can be traded off for a

given value of S: increasing T means decreasing K, for a given S– Case 2, Example 3: Low values of q/p , especially low q, mean little

change in K, for a change in T, for a given value of S– Case 3, Example 4: If either q or p is <0, this means that the equation

for S is defined in a region for which schedule increases/decreases are matched with cost/effort increases/decreases, i.e., changes of each variable are of the same sign; that is, they are not traded off: increasing T means increasing K, for a given value of S


11/03/09 18

Some Software Engineering and Systems Engineering Examples-2

• Application of these equations:– An organization might experience any one or all three behaviors: for T

increasing (illustrated in the plot on the next page): 1.K increasing, 2.K flat or 3. K decreasing The equations relating S to K and T are one-sided and might have to be fitted separately over various ranges

– Always look at the data; don’t just apply/fit equations !

• For small incremental changes in any one or all of the variables, e.g., ΔK/K, % change in K,:

ΔS/S= (ΔA/A)+(p*(ΔK/K))+(q*(ΔT/T)); If ΔS/S= (ΔA/A)=0; Then, ΔK/K)=-(q/p)*(ΔT/T); For Example 3, a 4% reduction in schedule

would result in only a 0.3% increase in effort. However, for Example 1, that same schedule reduction would result in a 3.5% increase in cost/effort. For Example 2, there would be 16% increase in effort; some would consider this to be not realistic


11/03/09 19

Illustrative Relative Effort Vs. Relative Schedule (100%=Baseline/Natural)

99%100%101%102%103%104%105%106%

50% 75% 100% 125% 150% 175% 200%

Relative Schedule

Rel

ativ

e Ef

fort

Region of Schedule Flexibility

Region of Schedule/Cost Tradeoff,Schedule Compression Region of Schedule and Cost

Increase


Admin User 1 of 2

20

SCHEDESTSE Administrative User Instructions

OverviewSCHEDESTSE provides several relationships amongst the COSYSMOR variablesS (size, Equivalent Requirements), K (effort, person hours), and T (schedule, months).The program or project user employing COSYSMOR uses the SCHEDESTSE tool functionfunction implemented on the SCHEDESTUSERSE COSYSMOR tool page. The tool is basedon these relationships and determines the value of T that corresponds to the value of Kproduced by COSYSMOR for a size, S. The program or project user decides whether he likes this value of T, based on requirements that the program/project is to meet. If the value of schedule thus produced is not satisfactory, then the program user employs anotherrelationship provided to determine the value of K that corresponds to the value of T that he desires.The constants in the relationships implemented in SCHEDESTSE are based on regressions on actual systems project data, sets of values of S,K and T. Most desirably, such data setsare from projects that are similar to the ones that COSYSMOR is "tuned' to or calibrated for. estimating systems engineering effort. These regressions and the determination of the constantsused in the SCHEDEST estimates are performed as part of the COSYSMOR calibrationprocess. The Administrative User performs the regressions described in the next section for a particular domain and enters the values of the constants in the table at the bottomof the page. The tool, implemented on the SCHEDESTSEUSER page, uses these constsnts.

Regression/Calibration and Estimation Parameter Values Estimation Process1. Using at least three sets of project data,<S,K,T> values, determine values for the constants A, p and q by regression of Log(S) on Log(K) and Log(T), where:Log(S) is the dependent variable and Log(K) and Log(T) are the independent variables."Log" means logarithm to the base 10. Here, you are using the logarithmic formof the "production function": S=a*(Kp)*(Tq).The regression equation is: Log(S)=A+(p*Log(K))+(q*Log(T)), and A=Log(a), and thus, a=10^A. This is Estimation Equation 1.2. Using at least three sets of project data,<S,K,T> values, determine values for the constants B, v and z by regression of Log(T) on Log(S) and Log(K), where:Log(T) is the dependent variable and Log(S) and Log(K) are the independent variables."Log" means logarithm to the base 10. Here, you are using the logarithmic formof the function: T=b*(Sv)*(Kz).The regression equation is: Log(T)=B+(v*Log(S))+(z*Log(K)), and B=Log(b), and thus, b=10^B. This is Estimation Equation 2.3. If the program user wants to determine the affect on effort of a schedule change from a value T1 to another value T2, then, he uses Estimation Equation 3, where:Estimation Equation 3 is: k2=k1/((T2/T1)r), where: K2 is the value of K associated withschedule value T2; r=q/p, see Estimation Equation 1 concerning p and q; and K1 is theoriginal value of K. Estimation Equation 3 is derived from Estimation Equation 1 by writing the equations for S2 and S1, dividing the one for S2 by the one for S1 and setting S2=S1.

Admin User 2 of 2

21

Program/Project User Procedure1. Enter S=S1 and K1 in Estimation Equation 2, and solve for T=T1.2. Determine if T1 is satisfactory relative to program/project requirements.a. If "yes," and you want to obtain the distribution of K1 over the period of time, T1, then enter K1 and T1 in the COSYSMORLABSHED tool page in cells Q91 andT99, respectively. If you do not want such a distribution,you are through.b. If "no," then decide what the desirable value of T, T2, should be. Enter T2, T1, and K1 in Estimation Equation 3 and solve for K2. Do a "sanity check" to make sure that K2 makessense. If the values of T2 and K2 are satisfactory, then enter them in the appropriate cells in COSYSMORLABSCHED as described in 2a.

As a general rule, any results from applying the procedure described here that uses or yields a value of S, K, or T outside of the ranges of the values that were used in the calibration process might be suspect (i.e, their validity should be questioned).

SCHEDESSE Regression/Model Constants

Name Value

A -1.0640

a=10^A 0.0863

p 1.0990

q -0.3010

r=q/p -0.2739

b=B 0.4060

10^B 2.5468

v -0.0300

z 0.3150

11/03/091 integrating cost and schedule estimation in the cosysmor model/tool john e. gaffney, jr....

Documents