demonstrating early value in software system projects using software and system cost models...

Demonstrating Early Value in Software System Projects

Using Software and System Cost Models

Anca-Juliana Stoica, IT Dept, UU26th International Forum on COCOMO and

Systems/Software Cost ModelingUniversity of Southern California, Los Angeles, CA

November 2-4, 2011

Info

rmat

ions

tekn

olog

i

Institutionen för informationsteknologi | www.it.uu.se

Agenda•Research Background

• Value-Based Software Engineering• Software and System Cost Models• Balanced Scorecard Method

•Algorithm for Applying VBSE •Collecting Project Information and Questionnaire

•Experience Database•Case Study•Final Remarks and Future Work

Info

rmat

ions

tekn

olog

i


What is value?

- Value: worth in usefulness or importance to the possessor; utility or merit

- Value of a system/software project: elicited by the project stakeholders’ values

- Early value: value generated early in the project lifecycle

Info

rmat

ions

tekn

olog

i


Value-Based Software Engineering

- Traditional Software Engineering (SE) is not enough to make projects succeed

- Value-Based Software Engineering (VBSE) is the integration a software system’s stakeholder value propositions into the system’s definition, design, development, deployment, evolution• Includes aspects not covered by SE that are critical to the system’s

success

- VBSE integrates value considerations into current and emerging software engineering principles and practices while developing an overall framework in which these techniques compatibly reinforce each other.

Info

rmat

ions

tekn

olog

i


VBSE Key Elements

Key components: - Benefits Realization Analysis- Stakeholder Value Proposition Elicitation and Reconciliation- Business Case Analysis- Continuous Risk and Opportunity Management- Concurrent Software and System Engineering- Value-Based Monitoring and Control- Change as Opportunity

Info

rmat

ions

tekn

olog

i


Problem/Question/Goal

- Validating VBSE compared to SE by demonstrating early value in software system projects

- Can early value be demonstrated in software system projects, if so, how?

- Demonstrate value Using data collection and projects

developed in academic environment for educational and commercial users

Info

rmat

ions

tekn

olog

i


COCOMO II Model

- Helps planning and reasoning about the cost and schedule implications of a software project• Software investment decisions

• Project budgets and schedules

• Cost/schedule/performance tradeoffs

• Software improvement decisions– Reuse, tools, process maturity, outsourcing

Info

rmat

ions

tekn

olog

i


COSYSMO Model

- COnstructive SYStems engineering cost MOdel

- COSYSMO helps plan the systems engineering

costs of a project

Info

rmat

ions

tekn

olog

i


Balanced Scorecard Method

- How do customers see us? (Customer perspective)- What must we excel at? (Internal perspective)- Can we continue to improve and create value? (Innovation and learning perspective)- How do we look to shareholders? (Financial perspective)

Info

rmat

ions

tekn

olog

i


Algorithm for Applying VBSE1. Elaborate the Results Chain based on Benefit Realization Approach

2. Identify success-critical stakeholders, their roles and value propositions

3. Analyze the Model Clash Spider Web based on the previous two steps

4. Verify that the project uses the prototyping techniques mentioned and/or a

relevant process model

5. Identify milestones according to the process model used ( like: iterative

process model with: i) major and minor milestones; ii) objectives to be tested

against at each milestone; iii) risk-driven).

6. Estimate costs using COCOMO II and/or COSYSMO models for

software and/or systems

7. Make market projections for the next time horizon (ASA, APR) (years)

8. Calculate Expected Benefits (EB)

9. Calculate Cumulative Expected Benefit (CEB) and Return On

Investment (ROI)

10. Compare expected and actual values using the measures presented

above in steps 6-9

11. Identify the critical risks and try to figure out how much risk is

enough by applying for example sweet points, utility functions

and people factors

12. Use the Results Chain and the milestones in order to monitor the

project progress and see if the project is progressing according to plan.

Goal-Question-Metrics (GQM) or the Balanced Scorecard Method (BSM)

can also be used. Apply corrective actions in case value is not realized

13. Apply architecture-based and refactoring techniques can be used in order

to validate Change as Opportunity

Info

rmat

ions

tekn

olog

i


Experience DatabaseExperience Database

- In the database all the information gathered from the projects in order to perform the analysis is stored

- Online access via website

Info

rmat

ions

tekn

olog

i


Database ModelDatabase Model

1x

1

1

x

1

xxxx

Project

ID

Name

Location

Description

Role

ID

Name

Deliverables

Id, name, filename, project_id, deliverabletype_id, keyword

Stakeholder

ID

Name

Deliverable_type

ID

Name

Project metrics

ID

Name

Project_id

Info

rmat

ions

tekn

olog

i


Case Study:Web-Based Time-Reporting System

• Developed for a company that provides other companies with consultants

•Online, anytime, from anywhere

• Keeps track of consultants working with several customers at the same time

• Security issues- no one could break it from outside

Info

rmat

ions

tekn

olog

i


ContributionOUTCOMEINITIATIVE

Implement an Online System for Consultants’ Time Reporting

Reduces time to work

Saves time for all system users

OUTCOMEContribution

Increased system efficiency

Saves operational costs

ASSUMPTION

Implementing a new online IT System is an important efficiency criterion

Step 1 – Results Chain based on BRAStep 1 – Results Chain based on BRA

Info

rmat

ions

tekn

olog

i


Step 2 – Stakeholders

Step 4: Process Model

Stakeholders:- Acquirers (Users) - R2M- Mentors- Project Group- KTH

Process Model:- The Rational Unified Process

(RUP) Iterative Major and minor milestones Formal objectives to be

tested against Risk-driven Prototyping technique: HTML

prototype of the system

Info

rmat

ions

tekn

olog

i


Step 3 – Model Clash Spider WebStep 3 – Model Clash Spider Web

PP/S

PD/SPP/PD

PC/PCPP/S

PD/PD

PD/SPP/PD

PC/PC

PD/PD

PD/PD Limited development budget, schedule

Development visibility and control

RUP Process

Acquirers/Usersrs

Mentors and KTH

Project group

Applications compatibility

Changeable requirements

Many features

Early availability

Project cost/effectiveness

Stable requirements

Ease of meeting budget and scheduleFreedom of choice COTS/reuse

Freedom of choice process

PC: Process

PD: Product

PP: Property

S: Success

Info

rmat

ions

tekn

olog

i


Step 5 – Major Milestones

Milestone Due date Milestone details

Inception readiness Oct 1, Year1 -

Life Cycle Objective (LCO) Oct 26, Year1Win- Win Negotiation, Vision, Glossary, Software and System Requirement Specification, Use Case Specification, Supplementary Specification

Life Cycle Architecture (LCA) Dec 21, Year1System Architecture, Software Architecture Document, Rose Analysis and Design Models, Data Base Model, User Interface Prototype (HTML), Architectural Prototype

Initial operational capability (IOC): hardwareDec 21, Year1

System up and running

Core capability drivethrough (CCD)Feb 1, Year2

Major Use Cases up and running

IOC: softwareFeb 25, Year2

All preliminary Use Cases up and running

Developed IOCMarch 1, Year2

-

Responsive IOCMarch 5, Year2

-

Info

rmat

ions

tekn

olog

i


Step 6 – Budget, COCOMO, COSYSMO

- Overall estimated development budget: 22 PM (11 persons: 7 programmers, 4 networking engineers; 2 full-time months)

- COCOMO II results for the project

- COSYSMO results•System Engineering Person-Months Estimated: 26•System Engineering Hours Estimated:3951•Total Normalized SE Size (expressed as EREQ): 553

Info

rmat

ions

tekn

olog

i


Step 7 – Market Projections

jjj ASHASIASA * jj ASAAPR *1.0

Year

ASIj

(Annual market SIze)

MSEK

ASHj (Annual market SHare)

%

ASAj (Annual market SAles)

MSEK

APRj Annual market PRofits

MSEK

0 400 000 0 0 0

1 500 000 0.1 50 5

2 600 000 0.5 300 30

3 700 000 1 700 70

4 800 000 1.5 1 200 120

5 900 000 2 1 800 180

6 1 000 000 2.3 2 300 230

7 1 100 000 2.3 2 530 253

Info

rmat

ions

tekn

olog

i


Step 8 – Expected Benefits

oldjnewjj APRAPRCPR ,, jjj CPROCSEB

Year APRj,new

(Annual market PRofits)

MSEK

APRj,old (Annual market PRofits)

MSEK

CPRj (Change in PRofits)

MSEK

OCSj (Operational Cost Savings)

MSEK

EBj (Expected Benefits)

MSEK

0 0 0 0 0 0

1 5 0 5 22.5 27.5

2 30 0 30 22.5 52.5

3 70 0 70 22.5 92.5

4 120 0 120 22.5 142.5

5 180 0 180 22.5 202.5

6 230 0 230 22.5 252.5

7 253 0 253 22.5 275.5

Info

rmat

ions

tekn

olog

i


Step 9 – Cumulative Expected Benefits and ROI

j

jjj CC

CCCEBROI

k

jkk EBCEB

1

Year EBj (Expected Benefits)

MSEK

CEBj

(Cumulative Expected Benefits)

MSEK

CCj (Cumulative Cost)

MSEK

ROIj (Return On Investment)

%

0 0 0 40 -1

1 27.5 27,5 55 -0.47

2 52.5 80 75 0.067

3 92.5 172,5 100 0.725

4 142.5 315 130 1.423

5 202.5 517.5 165 2.136

6 252.5 770 205 2.756

7 275.5 1 045.5 250 3.182

CEBk = 1045.5 MSEK

ROIk = 318.2%

Info

rmat

ions

tekn

olog

i


Return on Investment

Info

rmat

ions

tekn

olog

i


Step 10 – Expected and Actual Values

- Expected and Actual Values are compared using the measures presented in steps 6-9

- System up and running- Analyses performed show good ROI, COSYSMO and

COCOMO results- Future expansion has a lot of possibilities if they were

exploited

Info

rmat

ions

tekn

olog

i


Step 11 – Risks/Opportunities at Milestones

Major risks:- Team member withdrawal- Inadequate resources, personnel- Personnel competence- Time management during exams

Info

rmat

ions

tekn

olog

i


Risk Examples

Info

rmat

ions

tekn

olog

i


Steps 12 and 13 – Project progress and corrective actions

To monitor project progress, the deadlines for milestones, their pass-fail criteria, and risk management are reviewed

Milestone Due date Risks Eliminated

Inception readinessOct 1, Year1

Life Cycle Objective (LCO)Oct 26, Year1

Life Cycle Architecture (LCA) Dec 21, Year1

R1,R2, R6, R7, R9

Initial operational capability (IOC): hardwareDec 21, Year1 R11

Core capability drivethrough (CCD)Feb 1, Year2

R8

IOC: softwareFeb 25, Year2

R10

Developed IOCMarch 1, Year2

Responsive IOC March 5, Year2 R3, R4, R5, R12, R13

Info

rmat

ions

tekn

olog

i


Step 14 – Architecture

Is only applicable if there is a high-level architectural design like patterns and views

Info

rmat

ions

tekn

olog

i


Discussion-Algorithm and method for value-based software and system monitoring and control applied for a large number of commercial and educational applications developed in an academic environment

-Projects data were collected in the Experience Database

-Value-based questionnaire was designed and the results were analyzed

-Questionnaire results not included, but were used for validation purposes

-Projects developed for commercial applications like the presented case study showed larger ROI than those for educational applications

-Applying the above approach helped demonstrating early value in software and system projects motivating the need for further research in the area.

Info

rmat

ions

tekn

olog

i


Conclusion

Our approach for demonstrating early value in software and system projects is based on:

- Survey questionnaire - Real data collection via market research- Algorithm for applying VBSE

Software and system cost estimation models Other models / metrics

- Case studies- Experience database- Website

Info

rmat

ions

tekn

olog

i


Future work- Monitoring value creation in modern software processes- Applying patterns and views- Extended metrics set- Strategic IT decision-making- Models and model integration- Use project follow-up- Further case studies- Education and communication of scientific and practical

results

Info

rmat

ions

tekn

olog

i


References Biffl, S., Aurum A., Boehm, B., Erdogmus, H., Gruenbacher P.(Eds.), “Value-Based Software Engineering”,Springer, 2006 Boehm, B.,“Value-Based Software Engineering”, ACM SIGSOFT Software Engineering Notes vol 28 no 2, 2003. Boehm, B., and Huang, L., “Value-Based Software Engineering: A Case Study”, IEEE Computer, March, 2003. Jacobson, I., Booch, G., and Rumbaugh, J., “The Unified Software Development Process”, Addison Wesley, 1999. “RUP 2000: Rational Unified Process”, Rational Software, 2000. Stoica, A.J., “IT Project Management, Methods and Tools”. Lecture Notes, KTH, Stockholm, Sweden, 2002. Stoica, A.J., “An Internet-Intranet Solution for Software System Estimation with Use Cases”, Proceedings of the 18th International Forum on COCOMO and Software Cost Modeling, 2003.

Stoica, A.J., “Software Engineering and Security Architecture”. Lecture Notes, KTH-DSV, Stockholm, Sweden, 2004.

Stoica, A.J., “Aspects of Value-Based and Strategic Software Engineering Research”. Invited technical presentation, SINTEF Research Institute, Oslo, Norway, Sept. 2005. Stoica, A.J., “Value-Based Software Engineering”. Lecture Notes, KTH-DSV Stockholm, Sweden, 2007.

demonstrating early value in software system projects using software and system cost models...

Documents

system engineering value

value considerations

merit value

software system projects

project slide

project lifecycle slide

opportunity slide

outsourcing slide