look before you jump: the trials and triumphs of implementing standard software development...

Look Before you JUMP:The Trials and Triumphs of Implementing Standard Software Development Processes

Presented By:Michael Stefanini, Jet Propulsion Laboratory, California Institute of Technology

Grady Booch Speaks…

“People are more important than any Process.

Good people with a good process will out perform good people with

no process any time.”

- Grady is a Chief Scientist for Rational at IBM, author, and distinguished SW Expert, and the creator of UML

Everyone Cares about Process.

1 2 3 4 5

0% 0% 0%0%0%

1. Project Managers2. Technical

Development3. QA and Support4. Operations5. Other

4

Why do Projects Fail?

1. Ad hoc requirements management2. Overwhelming complexity3. Failure to address risks early on4. Ambiguous communication5. Brittle architecture that fails to perform under stress6. Undetected inconsistencies in requirements, designs,

and implementations7. Insufficient testing8. Subjective assessment of project status9. Uncontrolled change propagation10. Insufficient automation

5

How can Projects Succeed?

1. Pure team talent2. Clear team direction3. Flexible yet predictable environment4. True senior management support5. Maintain a project structure to provide planning and quick response6. Ensure pre-project readiness assessment & overall project planning7. Create a partnership with your partners and your stakeholders8. Adequately resource your project9. Communicate and manage expectations10.Encourage functional ownership of the project11. Develop dependency-driven project schedules that can be tracked

and managed to provide early warnings and help avoid crises

Why does Software Process Adoption Fail?

In other industries, process innovation has been the contributing factor in explosive growth.

Software development continues largely unchallenged with a record of late deliveries, budget overruns, missed expectations, and low quality.

6

Antagonists to Adoption

Developers and Managers are Undisciplined»Reward, hire, and promote people who demonstrate

discipline. Bloated Process Bureaucracy

»When people are given the assignment to develop a process area: they behave as if they are writing a Ph.D. thesis.

Ineffective Process»Just because you have invested a great deal of time

developing and deploying your process, it does not mean it is good or effective .

courtesy of the YouWantItWhen.com, Bill Miller

7


Management Priority & Commitment» Employees take serious what management demonstrates to be

important. » Don’t expect a good process to be introduced in less than 18 months.

Projects and Customers Are Un-Supportive» Software teams misuse process as a way of defending slips or holding

off customers. » Customers are often satisfied with informality. » Developers pit Customers against the process. Customers are told the

process is the reason they aren’t getting what they want or need.


8


Poor Process Audits»Auditors may find it uncomfortable writing up a negative

audit report. The only question the auditor should set out to answer is, is the team following the documented process?

Doesn’t Promote Change»The first version of your process will be buggy. You’ll need

to tweak it, and maybe even overhaul parts of it.


9


Been There Before»Usually the process experience isn’t positive: management

removes its sponsorship just as things were actually starting to improve. People become cynical about these initiatives, and they come to know if they hold management off long enough, they will abandon the initiative.

Process Fads»Adopting and rolling out process shares many

characteristics with keeping to a diet program.


10

What is the biggest hurtle in your area?

0%0%0%0%0%

1 2 3 4 5

1. Discipline2. Bad/Bloated Process3. Management Support4. Developer Support5. Poor Implementation

12

JUMP – IT Software

Processes at JPL

Project Strategy – The Deming Model

13

Planning(Plan)

Delivery (Do)

Metrics(Check)

Baseline(Act)

Improvement over Time

BASELINE

CHANGE

No one has to change.

Survival is optional.

Components of JUMP

Roles and Phases

14

PhasesRole

sAuthorityDelivery

ResponsibilityChecklists

ArtifactsReviews

15

Reviews

Review Goals and Intent

Goal One: Achieve Commitment and ConsensusThe four formal JUMP Reviews

are Confirmation and Commitment Reviews, not Design Reviews

»Reviews will focus on the completion of Checklists and Artifacts

»Reviews will assure commitment from responsible team members and interfaces

»Reviews also align projects with the organization’s directives and enforce policies

»Only the Configuration Manager can “FAIL” a project16

How do you document review decisions?

1 2 3 4

0% 0%0%0%

1. Reviews? What Reviews?

2. Informally / Show of Hands

3. Email / Signature4. Formal Scorecard

with Feedback and Metrics

Scorecard for Domain Architect

Inception Phase Review Metrics

Project Manager

Line Manager

Architect

Sponsor

CM

SQA

0

2

4

Elaboration Phase Review Metrics

Line Manager

Architecture

Technology

Operations

Developers

Project ManagerSponsor

IT Security

SQA

Dev Services

CM

0

2

4

21

Key Process Concepts

JIT Training and IT Accelerators

22

JUMP Tools: JUMP Schedule

Purpose: Track major schedule and delivery milestones

The provided schedule already includes all required (and many optional) JUMP milestones – a turn-key project schedule!

Dedicated Project Scheduler helps to keep projects on track and provide a Master Schedule Rollup

23

Technology Positions

24

25

Measuring Success

The Impact of ProcessInformal Formal

Compliance 33% 90%

Defect Density 11.74 3.06

Productivity 25 30

Comment Density 0.37 1.06

Task with more robust process had 74% fewer defects for every 1000 lines of code and was 20% more productive.

Comparison is for period from July 2005 to July 2007

Results from a 2 year study on Mission Design and Navigation Software by William Taber

Time Spent in Elaboration

27

Planning < 5% of project costs 150% - 200% Overruns Planning > 5%, but <10% of project costs 100% - 120% Overruns

Planning > 10% of project costs 0% - 50% Overruns

Front-End Planning includes1. Defining the scope

2. Developing the requirements3. Preliminary design

Source: NASA study performed in 1990

NASA Study

PMI Study

In the US, < 5% of total project time is spent planning with 150-200% overruns—NOT ENOUGH

In Japan, > 30% of total project time is spent planning with 0-20% overruns—TOO MUCH» Based on a 1995 study by C. Christensen

PMI suggests >20% of total project time be spent planning» Based on Generic Industry Studies

28

PMI suggests

>20%

Typical Resource Use by Phase

29By: Yuri Gomes

http://gp.embarcadero.com/authors/edit/1895.aspx

http://gp.embarcadero.com/authors/edit/1895.aspx

Do you track your own Project Performance Metrics?

1 2

0%0%

1. Yes2. No

Project Metrics

31

UX Evaluation Report

32

Weekly Project Status Reports

33

34

Best Practices

Time Boxing: Set Expectations Early!

35

Scope(Quality)

Schedule(Time)

Budget(Resources)

You can’t get it faster, better and cheaper!

36

Development Services Catalog

37

Software Quality

Assurance

QA Config Mgmt

Manual Testing

Automation

Environmental Architecture

Hardware Engineering

Development Hosting

Project Support

Technical Documentation

Training Development

JUMP Config Mgmt

Project Management

Project Scheduling

Project Coordination

Customer Support

Service Desk

38

JUMP TOOLS

39

Tales from the Pit

Each Phase Builds upon the Previous

40

ElaborationConstruction

InceptionElaboration

Transition

Construction

Initial Agreement

Scope Creep

Shifting

Expectations

Undocumented Requirements

Build and Fix

Project Failure results from not being in position to

meet the customer’s needs

Schedule-Induced

Delivery (De-Scoped)

41

Focus on People and Not Artifacts!

How Many Faces Did You See?

Other Lessons Learned

Large Training Classes do NOT work for most people Training is provided “Just In Time” IT Accelerators provides experienced coaching Affinity Groups provide more coaching, as well as Mentoring,

on the job Training, and Career Guidance Inject QA Early – Especially during Requirements Gathering Hold Architecture Reviews Early and Often Keep the Review Board Lean

43

44

Thank You

[email protected]

45

Traps

How Managers Destroy Projects

Usually, the Managers drive to deliver on time and on budget will be at odds with the development team’s desire to create a quality product

Management is well meaning and may be using proven techniques, but some have the potential for disaster

Here are four ways managers may unwittingly sabotage a project

46courtesy of the TechRepublic’s Robert

Bogue

Trap #1: False Dates Project managers must put dates out in front of the group to motivate

them but when the dates aren’t realistic and they are consistently missed, it’s time to reevaluate the plan

Solution:

Developers need to be involved in setting the schedule Developers need to be held accountable to the schedule Managers need to aggressively defend the project schedule

and the developers


Bogue

Trap #2: Pretend All Is Well

When it comes to project management, ignorance is not bliss

Solution

Frequent and open status reports: Try the 2-minute status report format on a daily or weekly basis

Utilize Project Management Office resources or your line management to report on an issue within a project.

You can report anonymously if you need to


Bogue

Trap #3: Ignoring Dependencies In software development we have a great number of

techniques for delaying dependencies

Solution

»The Task Plan»Developers must communicate with the project manager» If the Project manager ignores or defers any issues, the

development team must elevate the problem to another level (see Trap #2) or the project could fail

49

courtesy of the TechRepublic’s Robert Bogue

Trap #4: Time Boxing Getting top honors in the list of things which can destroy

software quality is the practice of time boxing

Time boxing works—most of the time—because it does three things» It forces the developer to be creative in finding a solution

that fits within their budget» It eliminates unnecessary frills and scope-creep that don’t

necessarily add value» It ensures iteration remain smaller and manageable while

deferring good ideas for later

Time-Boxing is used to manage scope-creep and defend the project plan – NOT to manage the schedule or deadlines


Bogue

Additional Material

52Time-Boxed

RUP

Tailored

Toolset

Iterative

Sample Mock-Ups

53

Mock-Ups (Wire Framing)

54

Customer Involvement Keep customers isolated from development and

testing efforts as well as internal policies»Minimize disruptions and avoid having the customer

become a member of the development team (or Management Team!)

Keep the customers involved with»UX reviews and screen-shot updates»Conducting Training»Communications and Outreach»Change Control

Give them what they pay for»Build creep into your margin

55

Scope Will Creep

It’s Human Nature: The more you think about a project, the more you will discover new ways to use or improve it

To manage scope creep» Include only clearly negotiated features in Inception Phase» Plan your resources to include Margin for these extra items» Ensure constant communication with your sponsor and customers» Manage your scope with a process that evaluates the impacts of new

additions or changes» Manage developer’s “Gold Plating” – this is self-imposed scope creep!

56

http://www.graphicsbydave.com/cartoon%20samples/cartoon%20samples2.htm

Example of ITIL Success Criteria

Discussion and Comments from Users ITIL Analysis and Next Steps

I wouldn't need this system, but design and fabrication on this project are so overlapped that we don’t know when drawings are being updated with red-lines or ECIs.

Trigger: Overlapping activities make it difficult to quickly identify design changes.

All the designers need an e-mail, spreadsheet, something! Something that lets me know all the redlines and ECIs that happened.

Results: A published listing of all the ECI’s and redlines on a project sorted by date.

Customers: All MSL designers need access to this list.

If we can get these lists on a daily basis, that would be great.

Metrics: Updated lists will be delivered every day until the project enters phase E. Five spot checks will be performed during the first 30 days that will validate the lists have the most up-to-date information from PDMS. Process issues will be handled by the project.

ITIL

57

Trap #1: False Dates Project managers must put dates out in front of the group to motivate

them but when the dates aren’t realistic and they are consistently missed, it’s time to reevaluate the plan» The problem is that when a really important date comes up there will be

little drive to hit it because an expectation has been set that the date isn’t important

» After all if the team misses 10 dates in a row, how important can the 11th date be?

If timelines are being set that have no penalty behind them and people aren’t meeting them it’s time to put some teeth behind them or move the whole timeline» Developers need to be able to concentrate on their work» The desire to meet the date and the confusion about whether the date is

real or not may lead to developers skipping critical steps in the development process and, in doing so, creating problems that will be hard to find 58


Trap #1: Solution

Developers need to be involved in setting the schedule Developers need to be held accountable when schedules are

missed Managers need to aggressively defend the project schedule

and the developers


Bogue

Trap #2: Pretend All Is Well

When it comes to project management, ignorance is not bliss» Risks turn into a reality and then panic sets in» Everyone scrambles to put together the rest of the pieces of the project

and the quality of the project will suffer from the hastiness of the final assembly

» Of course, this problem isn’t fully realized until the rush that is caused when the business learns about the true state of the project after believing that nothing was wrong for a long time


Bogue

Trap #2: Solution

Solution» Frequent and open status reports: Try the 2-minute status report format

on a daily or weekly basis» Utilize Project Management Office resources or your line management

to report on an issue within a project.» You can report anonymously if you need to


Bogue

Trap #3: Ignoring Dependencies In software development we have a great number of techniques for delaying

dependencies» We can stub out functions, remove connecting infrastructure, or bypass

extensive error handling» All of these techniques when used correctly can be helpful to moving a project

along» However, when it becomes required to get the project completed and when the

costs of these techniques are not factored into the planning, trouble sets in» Especially when it comes to unforeseen dependencies, the clean up cost can

often become a non-trivial part of the project’s overall cost—and one that isn’t discovered until the very end

External interfaces pose even greater risks» Project managers and developers often either defer discussing interface

agreements or each assumes the other is working to ensure the interface will be ready

» For unforeseen dependencies, external cooperation to provide support on the project’s time-table may be tough to get

62


Trap #3:Solution

The solution resides with the Responsible Developer»The Task Plan should detail any internal and external

interfaces that need to be managed»The developers must then communicate with the project

manager if they need any political support in ensuring external interfaces are ready, but the technical aspects of the interfaces lie with the development team

» If the Project manager ignores or defers any issues that arise, the development team must elevate the problem to another level (see Trap #2) or the project could fail


Bogue

Trap #4: Time Boxing Getting top honors in the list of things which can destroy software quality is the practice of

time boxing Used at its extreme it often means that the code isn’t complete, it’s merely pushed along the

process Time boxing works—most of the time—because it does three things

» It forces the developer to be creative in finding a solution that fits within their budget» It eliminates unnecessary frills and scope-creep that don’t necessarily add value» It ensures iteration remain smaller and manageable while deferring good ideas for later

The intent is to get the thing working and rely on a QA phase where detailed testing will hopefully reveal any problems that there may be with the code

Time boxing doesn’t always work if» The problem is unknown or the technology isn’t proven» The box is made so small there’s no possible way to complete the objective within the

allotted time» Used for research and development, problem solving, etc.

When time boxing is used correctly it shouldn’t result cut corners and poorly developed software

It should be used with moderation to ensure the lowest cost, quickest and best quality software possible


Bogue

Trap #4: Solution

Time-Boxing works well when the schedule is well designed and significant margin has been allocated

It is the responsibility of both the Project Manager and the Responsible Developer to ensure that the schedule and Task Plan are both realistic and have sufficient margin

Sticking to a failed Time-Boxed schedule for the sake of meeting a dead-line is not a good idea» Time-Boxing is used to manage scope-creep and defend the project

plan – NOT to manage the schedule or deadlines Time-Boxing results in removing functionality or adding resources – not

sacrificing quality If the schedule runs out of margin – Re-Plan and hold a review


Bogue

JUMP Tools: Risk List Purpose: Catalog and

Track all identified project risks and mitigations.

Severity is automatically calculated

Performance indicators are keyed to react to the status and severity of risks

Un-Mitigated Red-Risks are automatically identified

66

Status reports and views are generated

All standard risk

information is tracked in this

list.

JUMP Tools: To-Do List Purpose: Assign and

track Action Items Customize the list to meet

your project needs Individual and group

assignments Multiple To-Do Lists can

be created for several major project areas

E-Mail Notification and Outlook Integration

67

The list is pre-populated with tasks required to complete your JUMP Project

Setup!

JUMP Tools: Estimates to Complete

Purpose: Monitor budget status Designed to easily provide Estimate-to-Complete (ETC)

information Auto-calculates variance and ETCs Easily allows for budget baselines and status Provides graphs Track hard and soft liens Performance Indicators keyed off of budget status

68

Automatically creates budget graphs on the

resources pages

JUMP Tools: Issues/RFA List

Purpose: Monitor and Respond to Project Issues

Works like a Specialized To-Do List

Used to monitor RFAs, Customer Issues, and other problems

E-Mail Notification and Outlook Integration

69

Track Related Issues

Key Performance Indicators Purpose: Monitor and

report on overall project health

Tracks 11 key performance indicators (KPI)

8 KPIs are tracked automatically

70

KPI details for key areas are

provided

JUMP Tools: Document Tracking

Purpose: Track and Control Project Documentation

Directory Structure provided by section IM

Integrates with Office Applications

Full workflow and reporting capability

A separate Documents area is provided for JUMP Templates

E-mail documents directly to the site

71

Enable Drag-and-Drop, Alerts, and RSS feeds

Requirements in Quality Center

72

Details of the Requirements

are tracked

Methodologies and Process

look before you jump: the trials and triumphs of implementing standard software development...

Documents

good process

words software process

process methodology

process innovation

process area

software process adoption

unwelcome friend of

process level maturity