psp tutorial - conclusion filethese data convince them to take more care in their work. this saves...

© 1998 Carnegie Mellon University PSP Seminarl - 2 1

PSP Tutorial -Conclusion Watts Humphrey / Jim Over Speaker: Daniel M. Roy (STPP, visiting scientist SEI) Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213

Sponsored by the U.S. Department of Defense


Introducing the PSP PSP results

•course results•industry results

Introducing the PSP•management considerations•Team Software Process•introduction strategy

The PSP and process improvement


PSP Experience -1 The PSP is now being introduced in a growing number of companies and universities.

Over ninety PSP instructors have been authorized to teach PSP in both industry and government organizations..

Four companies offer PSP training and support.

Universities in the U.S., Canada, South America, Europe, and Australia offer PSP.


PSP Experience -2 The following data are from several sources:

•14 students at Carnegie Mellon University •a 1996 study of 104 engineers from industry and university PSP courses

•a 1997 study of 298 engineers from industry and university PSP courses

•industry data from Motorola PPG and Advanced Information Systems (AIS)


Size and Time Starting with PSP0 at program 1, the engineers gather size and time data on 10 programs.

They learn how to gather and track size and time data.

They also learn to project the size and development time for new programs.

By program 10, the engineers are using the full PSP3 process.


Ac tu a l S iz e Ra n g e

P rogram N um ber

Lin

es O

f C

od

e

0

1 00

2 00

3 00

4 00

5 00

6 00

7 00

8 00

9 00

1 2 3 4 5 6 7 8 9 1 0

Ma x

Avg

Min

Source: 14 students at Carnegie Mellon University


Ac tu a l T im e Ra n g e

P rogram N um b er

Ho

urs

02468

101214161820

1 2 3 4 5 6 7 8 9 1 0

Ma x

Av g

Min



Size Estimation Accuracy - 1

1110987654321030%

40%

50%

60%

70%

80%

90%

Mean Size Misestimation

PSP Level Average

Size Estimation Accuracy Trend

Program Number

Source: 1997 study of 298 engineers


PSP 0

PSP 1

PSP 2

100%0%-100%-200%-300%0

20

40

100%0%-100%-200%-300%0

20

40

100%0%-100%-200%-300%0

20

40

Size Estimation Accuracy - 2

Many extreme underestimates

Underestimates are less extreme

Much more balanced and accurate estimation of size



Effort Estimation Accuracy - 1

1110987654321020%

30%

40%

50%

60%

70%

Mean Time MisestimationPSP Level Average

Effort Estimation Accuracy Trend

Program Number



PSP 0

PSP 1

PSP 2

100%0%-100%-200%0

20

40

100%0%-100%-200%0

20

40

100%0%-100%-200%0

20

40

Effort Estimation Accuracy - 2

Majority are under-estimating

Estimation accuracy and balance of over- and under-estimates is improving

Majority have balanced estimates of time with far better accuracy



Defects With PSP, engineers gather data on every defect they find•in compiling•during unit test•and during personal design and code reviews

They see how many defects they inject and how long it takes to find and fix those defects.

These data convince them to take more care in their work. This saves compiling and testing time.


T o ta l D e fe c ts - Ra n g e

P rogram N um ber

Def

ects

/KL

OC

0

5 0

1 00

1 50

2 00

2 50

1 2 3 4 5 6 7 8 9 1 0

Ma x

Avg

Min



C o m p ile T im e Ra n g e

P ro g ram N u m b er

% o

f T

ota

l Tim

e

0

5

1 0

1 5

2 0

2 5

3 0

1 2 3 4 5 6 7 8 9 1 0

Ma x

Av g

Min



T e s t T im e Ra n g e

P ro g ram N u m b er

% o

f T

ota

l Tim

e

05

1 01 52 02 53 03 54 04 55 0

1 2 3 4 5 6 7 8 9 1 0

Ma x

Av g

Min



Defect Analysis With data on their defects, engineers can determine the effectiveness of various defect removal methods.

They find that they can remove •2 to 4 defects per hour during initial testing•3 to 5 defects per hour in design review•5 to 10 defects per hour in code review

They see that reviews save them compile and test time.


D e fe c t Re m o v a l Ra te s - C la s s

P ro g ram N u m b e r

Def

ects

/Ho

ur

0

5

1 0

1 5

2 0

2 5

1 2 3 4 5 6 7 8 9 1 0

D e s .R e v .

C o d eR e v .

C o m p ile

T e s t



Defects Removed During Compile

One defect per 20 lines of code is very common.

One defect per 20 lines of code is rare.

PSP 0

PSP 1

PSP 2

30020010000

40

80

120

30020010000

40

80

120

30020010000

40

80

120

Defects/KLOC Removed During CompileSource: 1997 study of 298 engineers


Defects Removed During Test

One defect per 30 lines of code is very common.

One defect per 30 lines of code is rare.

PSP 0

PSP 1

PSP 3

20010000

50

100

150

20010000

50

100

150

20010000

50

100

150

Defects/KLOC Removed During Test



Quality Measures Cost of quality measures process efficiency.

•fix time is percent time in compile and test•appraisal time is percent time in reviews

A/FR is the ratio of appraisal time to fix time.

Yield measures process quality.•percentage of defects removed before compile

Defect density measures product quality.•defects per KLOC in compile and test


Appraisal to Failure Cost R atio (A/FR ) -C lass

Prog ram N um ber

Ap

pra

isal

Fai

lure

Co

st R

atio

0

0 .5

1

1 .5

2

2 .5

3

3 .5

4

4 .5

5

0 1 2 3 4 5 6 7 8 9 1 0

Ma x

Av g

Min



T e s t D e fe c ts v s . A /F R - C la s s

Ap p ra isa l to F ailu re C o st R atio

Tes

t D

efec

ts/K

LO

C

0

2 0

4 0

6 0

8 0

1 0 0

1 2 0

1 4 0

1 6 0

1 8 0

0 1 2 3 4 5



Process Quality

111098765432100.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Mean YieldPSP Level Mean

Pre-Compile Defect Yield

Program Number



Product Quality

111098765432100

10

20

30

40

50

60

70

80

90

100

110

120

Mean Compile + Test

PSP Level Mean Comp + Test

Defects Per KLOC Removed in Compile and Test

Program Number



Productivity The 1997 Impact study of 298 engineers showed that there is no statistically significant change between an engineer’s productivity at the beginning of the PSP course and the end.

With the PSP, LOC/hour rates tend to increase for larger programs.

The principal PSP benefit is increased quality at no cost in personal productivity.


Productivity

1110987654321020

22

24

26

28

30

Mean LOC/HourPSP Level Average

Lines of (New and Changed) Code Produced Per Hour of Total Development Time

Program Number



Trade-off Analysis: Productivity for Process Quality?

9876543210%

20%

40%

60%Pre-compile Defect Yield

Assignment

Yie

ld

98765432120

22

24

26

28

30

Assignment

LO

C /

Hou

r

Productivity

The solid line is the PSP level average.The broken line is the assignment average.

PSP0 PSP1 PSP2 PSP0 PSP1 PSP2


PSP Industrial Introduction The PSP is being introduced in a number of software organizations.

Since few organizations have pre-PSP data, improvement comparisons are rarely possible.

The following charts show some of the data and experiences to date.


* Results of Applying the Personal Software Process, IEEE Computer, May 1997

Overview of Industry Results In-practice results are similar to course results.

One study* of three companies reported improved schedule, cycle time, and quality. •34 development and maintenance projects•produced and delivered 43 KLOC of new and modified code

•four defects were found in the field•combined defect density for the 43 KLOC is less than .1 defects/KLOC


Teradyne -1 Teradyne is a supplier of automated test equipment.

They sent a manager to the SEI’s PSP instructor authorization program.

This manager started PSP introduction with a PSP course for several teams.

They now have several teams that are using PSP in product development.


Teradyne -2 Using a PSP-based team process, one team built a system of 90,000 lines of code.

They used PSP methods to plan and track the entire project.•It was planned for 77 weeks, and completed in 71 weeks (8% early).

•It was estimated at 16,000 hours, and came in at 14,711 hours (8% under the estimate).

PSP quality methods were used.•Quality levels improved 100 times over their prior projects.

•Field trial defect level was 0.02 defects/KLOC.


Hill Air Force Base -1 The Hill Air Force Base software group develops avionics and support software for the US Air Force.

They sent several engineers to the SEI’s PSP instructor authorization program.

They recently were assessed at CMM level 5.

They recently completed a pilot project using a PSP-based team process.


Hill Air Force Base -2 The TaskView project was 25,820 lines of code.

The product was delivered a month ahead of the originally committed date, at almost exactly the planned cost.

Final test phases: system and operational tests•only one high-priority defect found•reduced from 22% to 2.7% of project schedule

The product is currently in acceptance testing with the customer; no defects have been found to date.


The AIS Corporation -1 Advanced Information Services (AIS) is an independent software contracting organization in Peoria, Illinois and Madras, India.

AIS has been working with SEI on process improvement and the PSP since 1992.

AIS has two SEI-authorized PSP instructors to train their engineering staff and are also licensed by the SEI to deliver PSP training.


Error in Estimated Weeks ofWork

Component Number

Err

or

%

-100

0

100

200

300

400

500

1 2 3 4 5 6 7 8 9

Before PSP After PSP

Source: AIS


Acceptance TestDefects/KLOC

Product

Def

ects

/KL

OC

00.10.20.30.40.50.60.70.80.9

A B C - P S P

Before PSP After PSP

Source: AIS


Start Date Size Sys Test Duration Sys Test Defects Language

8/91 30 req (est) 2 months 36 C

3/94 19 req 3 cycles 7 C

8/94 30 req 2 months na C

1/95 15795 LOC 1.5 months 8 C++

4/95 11681 LOC 1.5 months 1 C++

8/95 24 req 5 days 8 C

3/96 2255 LOC 2 days 0 C

7/96 6196 LOC 4 days 0 PL *SQL

7/96 1400 LOC 4 days 0 VB

10/96 13263 LOC 2 days 4 PL *SQL

1/97 4650 LOC 5 days 5 LOTUS MACR

NOMODEL

CMM

CMM +PSP

System Test Duration Comparing pre and post-PSP products of similar size, AIS data reflect

an order of magnitude reduction in system test duration—from months to days.

Source: AIS

BeforePSP

AfterPSP


Getting Started with PSP An organization will not see many useful benefits from a few PSP-trained engineers.

Installing PSP in the organization requires•management involvement•a team software process•an introduction strategy and plan


Management Involvement Convincing managers and obtaining resources

•takes a minimum of one day•can take forever

If the managers do not give a high priority to software process improvement, there is no point in trying to introduce the PSP.


Management Considerations Managers have to support PSP introduction by providing sponsorship and remaining actively involved.

Middle and first-line managers may need to change the way they manage.•schedule versus quality•coaching versus managing•using data•maintaining confidentiality•improvement-based incentives•team management


A Team Software Process A team software process is needed to provide a framework within which individual PSPs can operate.

A team process framework for PSP•defines the team-PSP interface•establishes standards and measurements•specifies where and when to use inspections •establishes planning and reporting guidelines•assigns team member roles•establishes team goals and quality criteria


The Team Software Process SEI is developing the Team Software Process (TSP), a process for PSP-trained software engineering teams with 2 to 20 members.

TSP supports•development, enhancement, and repair•self-directed teams•interdisciplinary teams•isolated software teams•statistical process control

TSP is a SW CMM Level 5 process for teams.


Team Software Process Goals The goals of the TSP work are

•accelerate software process improvement for all CMM maturity levels

•build high-performance teams in months not years

•provide improvement guidance for high-maturity organizations

•make Level 5 behavior normal and expected•promote a disciplined quantitative approach to software engineering


TSP Structure and Flow

Launch

Relaunch

Relaunch

Relaunch

Postmortem

Requirements

High-LevelDesign

Implementation

Integrationand Test

TSP has four principal phases•Requirements•Design• Implementation• Integration and Test

The project commitment may be for• the entire project•any single phase

A revised plan is produced at the beginning of each phase.


TSP Team Launch Each TSP phase starts with a team launch.

•form team•establish project objectives from a customer, business, and team perspective

•define team roles•define team process•set quality goals•establish team plan and individual plans•identify risks and assign responsibility •review and approve plan with management


TSP Planning At each phase, the engineers each plan their own work.

If their plans do not meet the project plan, the issue is immediately addressed.•the project plan is adjusted or•work scope is revised or•additional resources are applied

The engineers work to meet commitments they have planned for and personally set.


TSP Quality Certification With TSP, engineers certify that their programs are of high quality.•use PSP quality methods•evaluate with PSP data•call for team assistance as needed

The team certifies that each component is of high quality.•preliminary inspection acceptance•reinspection as needed•integration/system test acceptance


Introducing the PSP - 1

PSP introduction has been successful when •qualified PSP instructors are used•managers and engineers are trained•the full PSP course is followed•training is by engineering teams •line management are directly involved•senior management actively support and track the introduction plan

•mentoring is provided for managers and engineers


Introducing the PSP - 2 Assume your organization is currently working on CMM process improvement.

You need an introduction strategy and plan to •obtain management support•train the key project personnel on the PSP •provide training on managing PSP-trained engineers

•build a team of qualified trainers/change agents


Introducing the PSP - 3 SEI recommends

•start with executive seminar/planning session•identify key areas for initial introduction •train affected managers and engineers •train PSP instructor-change agents•establish needed support for pilot projects•conduct pilot projects using PSP and TSP•plan and initiate rollout across the organization


PSP and Process Improvement -1 The PSP is a CMM-based process improvement tool that works for•large and small organizations•organizations working towards CMM Level 2•organizations at CMM Level 2 and above

The PSP provides•a breakthrough technology for mature organizations

•a basis for establishing effective team and project-level processes for less mature organizations


PSP and Process Improvement -1 PSP supports organizational process improvement in several ways.

With PSP, engineers are able to have Level 5 personal process.

With TSP, teams are able to have a Level 5 team process.

•establishing and sustaining sponsorship

•developing improvement experience and skills

•convincing engineers and line managers to use defined processes and gather data

•generating measurable improvements in the near-term

•helping engineers improve their performance

•exposing engineers and managers to the principles of all CMM Levels


CMM KPAs in PSP and TSP

5 Optimizing

4 Managed

3 Defined

2 Repeatable

Continuous processimprovement

Product and processquality

Engineering process

Project management

✔Defect prevention✔Technology change management✔Process change management

✔Quantitative process management✔Software quality management

✔Requirements management✔Software project planning✔Software project tracking✔Software quality assurance✔Software configuration management

Software subcontract management

Level Focus Key Process Areas (KPA)

✔Organization process focus✔Organization process definition

Training program✔Integrated software management✔Software product engineering✔Intergroup coordination✔Peer reviews

✔CMM Key Process Area fully or partially addressed in PSP or TSP


Think of the CMM, the TSP, and the PSP as tools for improving your organization’s software process.

The CMM provides•an orderly framework in which to do good engineering

•the infrastructure and resources to define and improve the organization’s processes

•an outline of the sound management and engineering practices

Process Improvement Tools -1


Process Improvement Tools -2 The PSP provides

•the principles and methods engineers need to define, use, and improve their performance

•the knowledge, skill, and conviction to use these methods in practice

The TSP provides•a framework for operating teams of PSP-trained engineers

•a way to start teams at maturity level 5


Process Improvement Tools -3

CMM - Buildsorganizational

capability

TSP - Buildsquality products

on cost and schedule

PSP - Buildsindividual skill and discipline

By using all the tools you will accelerate software process improvement.


Messages to Remember 1. The PSP shows engineers how to use process methods in their work.

2. It will help them to understand and to better manage their own performance.

3. PSP training will also accelerate organizational process improvement.

4. PSP introduction takes concerted study and active management support.


Reference Material SEI Support for PSP

•PSP Transition Collaboration•PSP Executive Seminar•PSP for Software Managers•PSP for Engineers Part I and II•PSP Instructor Training•PSP Transition Partners


SEI Support for PSP SEI provides support to organizations that want to transition PSP into practice.•on-site support for the initial introduction of PSP on pilot projects

•public PSP training courses for engineers and managers

•instructor authorization/change agent program

•transition partner program


PSP Transition Collaboration -1 A transition collaboration is a tailorable program to install PSP and develop transition capability within the organization.

Limited availability

For more information contact Janice MarchokRyan.

Collaborations include:

•transition planning

•on-site training for executives, managers, and engineers

•PSP instructor authorization

•SEI-coached TSP launches on pilot projects


PSP Transition Collaboration -2 Example timeline

•based on 9-12 month pilot projects• initial results are within first year• final results are within 18 months•demonstrates costs and benefits of PSP and TSP•builds high-performance teams rapidly

Ta s k Q1 Q2 Q3 Q 4 Q5 Q 6

E xe c u tive tra in in g /kic ko ff s e s s io n ◆

S e le c t p a rtic ip a n ts , d e ve lo p s c h e d u le ◆

Tra in m a n a g e rs , e n g in e e rs , in s tru c to rs ◆ ◆ ◆

C o n d u c t p ilo t p ro je c ts with TS P ▲ ▲

P la n a n d in itia te ro ll-o u t ▲ ➡


PSP Executive Seminar A one day PSP seminar for software executives and middle managers

Describes the PSP from a software business perspective

Builds support for introducing PSP

Availability•public offerings at SEI•on-site delivery

Topics PSP Executive Seminar

• PSP and the software business• The baseline process• PSP planning methods• PSP quality methods• Managing with PSP• PSP and the organization

For more information contactLinda Shooer or SEI customer relations.


PSP for Software Project Managers Five day course for software project managers

PSP from a software project perspective

Builds knowledge and skills for managing engineers that are PSP trained

Availability•on-site delivery only

Topics PSP for Software Project

Managers• PSP and the software business• Introduction to the PSP• Measurement in the PSP• Estimating and planning

methods in the PSP• Defect management methods

in the PSP• Project cost, schedule, and

quality management with the PSP

• Managing and coaching PSP-trained engineers

For more information contactJanice Marchok Ryan.


PSP for Engineers Part I & II Two one-week courses for software engineers

Builds the discipline and skills to use and adopt PSP

Availability•public offerings at SEI•on-site delivery

Topics Part I: Planning

• Introduction to personal process

• Size measurement• Size estimating• Proxy-based estimating• Resource estimating• Process measurement

Part II: Quality• Defect management• The design process• Design verification• Scaling up the PSP• Process development• Using the PSPFor more information contact

Linda Shooer or SEI customer relations.


Contact information

Daniel M. Roy STPP Inc. 20 Forest Rd. Bradford Woods PA 15015 Voice: (724) 934 0943 Fax: (724) 934 3805 [email protected]

psp tutorial - conclusion filethese data convince them to take more care in their work. this saves...

Documents