se 470 software development processes james nowotarski 19 may 2003

SE 470Software Development Processes

James Nowotarski

19 May 2003

Course Map

Overview. Introduction. History

Content. Rational Unified Process. Extreme Programming

Implementation. Tools, Training, Roles. CMM, Metrics. Selection & Evaluation

Briefings (Term Papers)

1 2 3 4 6 7 8 9 10 115

Assignments

Quizzes

Week

Mem

ori

al D

ay

• Recap the Capability Maturity Model (CMM)

• Understand key issues of methodology selection and evaluation

• Discuss term papers (last chance)

Today’s Objectives

Topic Duration

• CMM Recap 15 minutes

• CMM Jeopardy 45 minutes

• CMM Quiz 35 minutes

• *** Break 15 minutes

• Methodology Selection: Organization 50 minutes

• Methodology Selection: Project 20 minutes

• Project Methodology Adequacy 15 minutes

Today’s agenda

CMM Levels

Repeatable(2)

Repeatable(2)

Defined(3)

Defined(3)

Managed(4)

Managed(4)

Optimized(5)

Optimized(5)

Initial(1)

Key process areas (KPAs)

Maturity levels

Process capability

Indicate

Key process areas

Contain

Key practices

Contain

Goals

Achieve

CMM Appraisal Method

TeamSelection

ResponseAnalysis

On-site visit

Interviews &documentreviews

Findingsbased on the CMM

1

MaturityQuestionnaire

23

4 5

KPAProfile

6

Appraisal Methods

• Software Process Assessments (SPA)– Performed in open, collaborative environment– Focuses on improving the organization’s software

process– Now called CMM-Based Appraisal for Internal Process

Improvement (CBA-IPI)• Software Capability Evaluations (SCE)

– Performed in a more audit-oriented environment– Focuses on identifying risks associated with a

contractor– Team’s recommendation will help select contractors or

set fees

CMM Issues in the Real-World

• “Level envy”• Areas not addressed

– Business strategy and linkage to IT– Operations, help desk, support– Management of the IT human resource– Application portfolio– Tools– Risk

• Many question whether it is worth the effort to pursue levels 4 and 5

CMM Maturity Profile

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0%

Initial

19.3%

Repeatable

43.2%

Defined

23.4%

Managed

7.3%

Optimized

6.8%

% o

f O

rga

niz

ati

on

s

August 2002

Based on assessments from 1998-2002 of 1124 organizations

USA and Offshore Profiles

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0%

Initial

23.3%

Repeatable Defined Managed Optimized

% o

f O

rga

niz

ati

on

s

August 2002

Based on 645 U.S. organizations and 479 offshore organizations

14.0%

47.4%

37.6%

22.0%

25.3%

5.3%

10.0%

2.0%

13.2%

USA Offshore

CMM

• Marketing tool to win clients, who are based predominantly in US and Europe• Clients using Indian service providers should have certain key processes in place:

– service level agreements– identifying business requirements– scoping requirements– managing changes

Approximately 50 of the 70 or so publicly-acknowledged Level 5 CMM-certified organizations are in India

Time to Move Up100

0

75

50

25

1 to 2

23 22

2 to 3

28

3 to 4

17

4 to 5

Largest observed value thatis not an outlier

75th percentile

Median (50th percentile)25th percentileSmallest observed value thatis not an outlier

Recommended time between appraisals (18-30 mos)

Number of months to move to nextmaturity level

CMM-based Software Process Improvement (SPI)

• Time and cost often exceed expectations– 18-24 months to advance 1 level– Can cost $2K per software engineer per year– 1-2% full-time resources (e.g., 5-10 in a 500-person organization)– 2-4% of rest of organization’s time

• Difficult KPAs– Planning and tracking

• Key success factors– Senior management is engaged– Participation and buy-in at all levels, including middle management

and technical staff– Clearly stated, well understood SPI goals– Clear assignment of responsibility– SEPG staffed by highly respected people

Software Process Improvement Models

• ISO 15504• ISO 9000-3• TickIT

IT specific models

A number of models enable software development organizations to compare their practices to a set of “best practices”

• Total Quality Management (TQM)• Six Sigma

General models


• International collaborative effort (including SEI)• Sparked by an investigative study sponsored by the

U.K. Ministry of Defense (MOD) • Objective: To develop a standard in the area of

software process assessment – establish a common framework for expressing the

process capability ratings resulting from a 15504-conformant assessment

– provide a migration path for existing assessment models and methods wishing to become 15504-conformant

ISO 15504


• Over time, proliferation of CMMs:– Capability Maturity Model for Software (SW-CMM®)– Systems Engineering Capability Model (SECM) (may or may not include software)– Integrated Product Development Capability Maturity Model (IPD-CMM)– Software acquisition

• Many organizations wish to integrate improvement efforts across disciplines

• Differences among these multiple models made integration difficult

• SEI developed common framework to support integration of current and future discipline-specific maturity models

• The common framework is called the Integrated CMM (CMMI)– “Each CMMI model is designed to be used in concert with other CMMI models, making

it easier for organizations to pursue enterprise-wide process improvement at their own pace”

The Integrated CMM (CMMI) has emerged as the latest thinking from SEI

CMMI

• Incorporates and extends:– Capability Maturity Model for Software (SW-CMM®)– Systems Engineering Capability Model (SECM)– Integrated Product Development Capability Maturity Model (IPD-CMM)– Supplier sourcing

• CMMI-SW model released August 2002• SW-CMM® to be sunsetted by end of 2003 (stay tuned)• What’s different about CMMI-SW:

– Stronger linkage to business objectives and customer needs– Greater alignment with relevant ISO standards– Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) V1.1 as a replacement for CMM-

Based Appraisal for Internal Process Improvement (CBA IPI) and Software Capability Evaluation (SCESM)

• Objective: To develop a standard in the area of software process assessment – establish a common framework for expressing the process capability ratings resulting from a

15504-conformant assessment – provide a migration path for existing assessment models and methods wishing to become

15504-conformant

•

CMMI integrates process improvement models for product and service development and maintenance

CMMI

• What’s different about CMMI-SW:– Stronger linkage to business objectives and customer needs– Greater alignment with relevant ISO standards– Standard CMMI Appraisal Method for Process Improvement

(SCAMPISM) V1.1 as a replacement for CMM-Based Appraisal for Internal Process Improvement (CBA IPI) and Software Capability Evaluation (SCESM)

– New process areas– New names for maturity levels

• Still outside scope of CMMI:– People CMM (P-CMM)– Software Acquisition CMM (SA-CMM)

CMMI

• New names for maturity levels:

Level Old New

1 Initial Initial

2 Repeatable Managed

3 Defined Defined

4 Managed Quantitatively Managed

5 Optimizing Optimizing

CMMI

• http://www.sei.cmu.edu/publications/documents/02.reports/02tr029.html (600+ page pdf)

For more information

Topic Duration








Today’s agenda

Methodology Selection Process

Methodology Selection Process

1. Pre-RFP

2. RFP

3. Proposal Submissions

4. Proposal Evaluations

5. Vendor Selection

6. ProcurementMethod

7. ROI Analysis

8. NegotiateContract

Objective: Identify best solution to meet stated business need while minimizing cost and risk

1. Pre-RFP

• Also known as Requirements Definition

• Preliminary analysis for management (not given to vendor)

• Serves as basis for Request for Proposal (RFP) and evaluation criteria

• May be a simple presentation (small firm) or a formal report

• Most important step in the system procurement process

1. Pre-RFP

• Factors to consider

– Organization and project sizes

– Locations/Degree of distribution

– Diversity of application developers

– IT culture

– Cost/Budget

• Request for Information (RFI)

– Vendors are called/consultants consulted/research conducted

– Breadth of alternatives is identified

– Vendors identified to participate in future stages

1. Pre-RFP

Sections of the Pre-RFP Report

• Problem statement

– Current state

– Gaps

– Risks

• Alternative solutions

• Ratings (of each alternative)

• Range of costs and benefits

• Recommended alternative and rationale

1. Pre-RFP

Categories of Methodology Providers

• Developed and delivered by consultants

• Software tool vendors

• Industry consortia or other groups

2. RFP

• Blueprint

• Confirms in detail the exact requirements stated in both business and technical terms

• Limited distribution (e.g., 3-5 vendors)

– Protect confidentiality

– Keep selection process manageable

2. RFP

• Business need/Requirements

• Statement of Work to be done

– Methodology characteristics

– Implementation plan

– Training strategy

– Maintenance and support

– Cost/Budget

• Procedural details

– Form and structure of proposal

– Schedule (meetings, demos, selection)

– Key contacts

• Selection criteria

Sample Contents of an RFP

2. RFP

• Multiple solutions available that will fit the need

• Multiple vendors can provide the same solution

• Corporate policy requires it

When should an RFP be used?

2. RFP

• RFP team develops better understanding of their needs from both a technical and business perspective

• Compels vendors to create competitive solutions

• Does not favor one vendor over another (in theory)

• “Everybody singing from same hymn book”

– Vendors working from same set of rules and requirements

• Facilitates evaluation of competitive solutions

– Provides a foundation on which to base a more rigorous evaluation of a vendor

Advantages of Using an RFP

3. Proposal Submissions

• Forums to answer vendor questions (written, oral)

– Vendor conferences before proposal submission

• Response content and format

• Sometimes requires "proof" statements, such as "This feature was implemented 12 months ago and is currently installed at 10 sites. Names and addresses are provided in the reference section."


• Business and Technical Solutions

– Rating scale: (0=unresponsive, . . ., 5=exceptional)

• Vendor qualifications (site visits, reference checks)

• Preliminary cost, value, and risk analysis

• Cost proposal may be a separate document from technical proposal

• Vendor demo

• Personnel assignment

• May be a two-stage process, with only a “short list” of 2-3 vendors doing demos and making “best and final” offers

• Question: Who are the key stakeholders in this process?


• Ability and track record of vendor to meet commitments?

• Satisfaction levels of vendor’s current customers, particularly long-term customers?

• Vendor’s track record for providing support?

• Any litigation pending against vendor?

• Is the vendor financially stable?

In selecting a vendor, there are major management and technical considerations

Management considerations


• Ability and track record of vendor to meet technical challenges of project?

• Level of vendor’s expertise in your industry (e.g., Financial Services)? With your type of IT organization?

• Quality of vendor’s past work? Are metrics available?

In selecting a vendor, there are major management and technical considerations (cont.)

Technical considerations

• Content– building blocks (processes, deliverables, etc.)– pre-defined routes

• Delivery vehicle (e.g., browser)• Tools for authoring and publishing content• Tools for applying the methodology to a specific project

– project planning and estimating tools– process management tools– project management tools

• Deliverable templates tightly coupled with a development and/or execution platform• Training and support• Maintenance

What does a methodology product consist of?

• Content• Methodology tools• Fit with development tools• Deployment and Support• Vendor • Cost

Small groups

Identify criteria to be considered in each of the following categories when considering methodology products for your organization

Small groups

Examples:

Content

- Which life cycle models are supported?- Does the methodology support component-based

development?

5. Vendor Selection

• Vendor site visits

• Weighted score method

• Final cost, value, and risk analysis

– Costs

-- one-time vs. recurring

-- fixed vs. variable

– Benefits

-- tangible vs. intangible

6. Procurement Methods

• Purchase

– not that popular because of fear of obsolescence

– longest-term commitment of these 3 methods

• Subscription

– usually 12-36 months in duration

– often done with an option to buy

7. ROI Analysis

• Metrics to justify investment in methodology remains a weak link

• Vendors supply very little help for customers developing ROI models for methodology

• Managers must commit to developing custom cost, benefit, and risk models to justify their methodology investments

investment Totalinvestment on Earnings

ROI

8. Contract Negotiation

• Do’s

– Include vendor responses to RFP in the contract

– Consider intellectual property issues

– Leverage outside expertise in negotiations

– Provide incentives/penalties

• Don’ts

– Buy vaporware instead of proven solutions

– Purchase low bid unless the value is there

– Settle on final offer prematurely

Topic Duration








Today’s agenda

Context

Content

Course ofAction

Methodology Selection: Project

• The Course of Action (the selection of the system development process) is driven by:– Content factors: the result to be achieved– Context factors: internal and external influences

Decision Tree

Buyvs.

Build

Packaged

Agile

Trad.

Assetvs.

Non-Asset Technology

Generation

Asset-Based

Legacy

See Table 1 Context 1




Buy

Build

Agilevs.

Traditional

Asset

Non-Asset Netcentric

Legacy

Table 1 - Buy versus Build

• Content FactorsBuy Build Comments:

Shorter initial time to market X

Higher responsiveness to market changes X

Lower project risk X

Lower budget for development (X) In case of some enterprise wide packaged software

this is not/ should not be a driving factor

Greater variety of users (discretionary,

consumers, suppliers, etc.)

(X) Although some packages can also support

Greater need for competitive advantage X

Greater need for multi-media content Not a distinguishing factor in this decision

Greater acceptability of constraints on

organization and processes

X

Context 1 - Buy versus Build

• Context Factors– Is a suitable package available?

– Availability of tools (development tools, plug-ins, etc.)

– Previous success with system development process

– Management understanding of and commitment to one of the approaches

Table 2 - Asset versus Non-Asset

• Content Factors

Asset

Non-

Asset Comments:

Shorter initial time to market X



Lower budget for development (X) Sometimes the asset is considered an evolving

asset, and requires an eventual total development

cost equal to custom developed systems.



Not a distinguishing factor in this decision

Greater need for competitive advantage (X) Assets can sometimes be modified over time to

provide as much of a competitive advantage as a

custom developed system




X

Context 2 - Asset versus Non-Asset

• Context Factors– Is suitable asset available?


– Availability of asset-experienced resources



Table 3 - Technology Generation

• Content Factors

NC

C/ S or

Host Comments:

Shorter initial time to market Not a distinguishing factor in this decision

Higher responsiveness to market changes Not a distinguishing factor in this decision


Lower budget for development Not a distinguishing factor in this decision



X

Greater need for competitive advantage Not a distinguishing factor in this decision

Greater need for multi-media content X




Context 3 - Technology Generation

• Context Factors– Availability of tools (development tools, plug-

ins, etc.)


– Need for the client to be considered leading edge


Table 4 - Agile vs. Traditional

• Content Factors

Agile Trad. Comments:

Shorter initial time to market Not a distinguishing factor in this decision



Lower budget for development X




Greater need for competitive advantage Not a distinguishing factor in this decision





Context 4 - Agile vs. Traditional

• Context Factors



– Experience with object technology

– Need for the client to be considered leading edge

– Result will be used as an asset


Words of Caution

• A system development process is not all-exclusive. In all probability the project will use concepts, tasks, deliverables from more than one system development process

• One project, however, will follow primarily one system development process

• For specific projects some factors may not apply, or additional factors can have an influence on the decisions.

Topic Duration








Today’s agenda

Project Methodology Adequacy

Content

• Was there an adequate set of systems development methodology and standards to guide project activities?

– What existed?

– Was it adequate?

Key Questions

Deployment

• Was the set of systems development methodology and standards rolled out to developers so that they had awareness and understanding of how to use them?

Usage

•Did project personnel adhere to the standards and methodology?

• First half of term papers to be presented

Next Class: June 2

Extra Slides

se 470 software development processes james nowotarski 19 may 2003

Documents