balancing agility and discipline (part 2) -...

Balancing Agility and Discipline (Part 2)

CS 566 – Software Management and Economics

Lecture 10 (Boehm and Turner Tutorial 2004;

Chapters 4 – 6, Boehm and Turner Book 2004)

Ali Afzal Malik

LUMS 2

Two Case Studies

• Show how agile and plan-driven methods can be extended

• Provide examples of success and lessons learned

• ThoughtWorks Lease Management– Agile extended with plan-driven

• CCPDS-R– Plan-driven overlaid with agile concepts

LUMS 3

Thoughtworks Lease Management

• XP replaced ineffective traditional development

• Problems when project moved beyond XP assumptions– The effort to develop or modify a story really does not

increase with time and story number– Trusting people to get everything done on time is compatible

with fixed schedules and diseconomies of scale– Simple design and YAGNI scale up easily to large projects

• Disciplined practices enabled XP to scale up– High-level architectural plans to provide essential big-picture

information– More careful definition of milestone completion criteria to

avoid “finishing” but not finishing– Use of design patterns and architectural solutions rather than

simple design to handle foreseeable change

LUMS 4

CCPDS-R

• USAF Command Center Processing and Display System Replacement for early missile warning

• Government contract environment required plan-driven approach, project needed agility

• Practices implemented to provide agility mapped well to the Agile Manifesto – Individuals and Interactions over Processes and Tools

• Milestone content was redefined • Architecture was organized around the performers’ skill levels

– Working Software over Comprehensive Documentation• Later PDR demonstrated working software for high-risk areas

– Customer Collaboration Over Contract Negotiation • Used COCOMO for cost and schedule negotiations

– Responding to Change Over Following a Plan • Automated plans• Architecture for re-use saved effort

LUMS 5

CCPDS-R (2) Cost of Change

RATIONALS o f t w a r e C o r p o r a t I o n

Project Development Schedule15 20 25 30 35 40

40

30

20

10

Design Changes

Implementation Changes

MaintenanceChanges and ECP’s

HoursChange

RATIONALS o f t w a r e C o r p o r a t I o n

Project Development Schedule15 20 25 30 35 40

40

30

20

10

Design Changes

Implementation Changes

MaintenanceChanges and ECP’s

HoursChangeHours

Change

Cos

t of C

hang

e

Time

Beck

LUMS 6

Five Critical Decision Factors

• Size, Criticality, Dynamism, Personnel, Culture

Personnel

Dynamism (% Requirements-change/month)

Culture (% thriving on chaos vs. order)

Size (# of personnel)

Criticality (Loss due to impact of defects)

5030

105

1

90

70

50

30

10

3

10

30

100

300

35

30

25

20

15

Essential Funds Discretionary

Funds Comfort

Single Life

Many Lives

(% Level 1B) (% Level 2&3)

0

10

20

30

40

Agile

Plan-driven

Personnel





5030

105

1

90

70

50

30

10

3

10

30

100

300

35

30

25

20

15


Funds Comfort

Single Life

Many Lives


0

10

20

30

40

Agile

Plan-driven

LUMS 7

Lease Management CCPDS-R

Comparing the Case Study Projects

LUMS 8

Misconceptions

LUMS 9

Outline

• Tutorial learning objectives– Survey of assumptions about participants

• General software trends and implications– Sources of perplexity about agile, plan-driven methods

• Overview of agile and plan-driven methods– XP, TSP “days in the life”– Comparisons of differences, strengths, weaknesses

• Risk-based balance of agility and discipline– Small, medium, large examples

• Observations and Way Forward

• Conclusions; review of learning objectives

LUMS 10

Using Risk to Balance

Discipline and Agility - Overview

Step 5. Execute and Monitor

Step 4. Tailor Life Cycle

Step 3. Architecture Analysis

Step 1. Risk Analysis

Step 2. Risk Comparison

Rate the project’s environmental, agility-

oriented and plan-driven risks.

Uncertain about

ratings?

Buy information via prototyping, data

collection and analysis

Compare the agile and Plan-

driven risks

Go Risk-based Agile

Agility risks dominate

Plan-driven risks dominate

Architect application to encapsulate agile parts

Go Risk-based Agile in agile

parts; Go Risk-based Plan-

driven elsewhere

Yes

No

Go Risk-based Plan-driven

Tailor life cycle process around risk patterns

and anchor point commitment milestones

Monitor progress and risks/opportunities,

readjust balance and process as appropriate

Neither dominate

Deliver incremental capabilities according to

strategyNote: Feedback loops present, but omitted for

simplicity

Step 5. Execute and Monitor

Step 4. Tailor Life Cycle

Step 3. Architecture Analysis

Step 1. Risk Analysis

Step 2. Risk Comparison

Rate the project’s environmental, agility-

oriented and plan-driven risks.

Uncertain about

ratings?

Buy information via prototyping, data

collection and analysis

Compare the agile and Plan-

driven risks

Go Risk-based Agile

Agility risks dominate

Plan-driven risks dominate

Architect application to encapsulate agile parts

Go Risk-based Agile in agile

parts; Go Risk-based Plan-

driven elsewhere

Yes

No

Go Risk-based Plan-driven

Tailor life cycle process around risk patterns

and anchor point commitment milestones

Monitor progress and risks/opportunities,

readjust balance and process as appropriate

Neither dominate

Deliver incremental capabilities according to

strategyNote: Feedback loops present, but omitted for

simplicity

LUMS 11

Risks Used in Risk Comparison

• Environmental risks– Technology uncertainties– Many stakeholders– Complex system-of-systems– Legacy compatibility

• Agility risks– Scalability– Use of simple design– Personnel turnover– Too-frequent releases– Not enough agile-capable people

• Plan-driven risks– Rapid change– Need for rapid results– Emergent requirements– Not enough discipline-capable people

LUMS 12

Three Examples

• Agent-based systems

• Small – Event Managers application

• Medium – SupplyChain.com application

• Large – National Information System for Crisis Management (NISCM) application

• Each example results in a development strategy based on risk analyses

LUMS 13

Event Managers Project

• Small startup company

• Diverse set of smaller agent-based planning applications

• This project: support for managing the infrastructure and operations of conferences and conventions

• Widening variety of options and interdependencies, makes an agent-based approach highly attractive

LUMS 14

Event Managers Profile

Personnel





5030

105

1

90

70

50

30

10

3

10

30

100

300

35

30

25

20

15


Funds Comfort

Single Life

Many Lives


0

10

20

30

40

Risk Items Risk Ratings

Event Managers

Environmental Risks

E1. Technology uncertainties �

E2. Many stakeholders �

E3. Complex system of systems �

Risks of using Agile Methods

A1. Scalability �

A2. Use of simple design �

A3. Personnel turnover ��

A4. Too-frequent releases �

A5. Not enough agile-capable people �

Risks of using disciplined methods

D1. Rapid change ��

D2. Need for rapid results ��

D3. Emergent requirements ��

D4. Not enough discipline-capable people

�

Risk rating scale: � - minimal risk � - moderate risk

�� - Serious but manageable risk �� - Very serious but manageable risk �� - Show-stopper risk

LUMS 15

Event Managers Strategy

1. StartupTeambuilding and Shared

Vision

2. Design, Development and Deployment

• Establish CRACK joint management team

• Develop shared vision, results chain, life cycle

strategy, infrastructure

choices

• Establish commitment to proceed, incremental

capabilities, backlog

• Develop next-increment

features

• Test, exercise, deploy new increment• Re-prioritize next-increment features

• Analyze lessons learned; prepare strategy for next increment

Customer Management; Representative Users

Joint Developer-Customer Agile Team

LCA

LUMS 16

SupplyChain.com Profile

• Turnkey agent-based supply chain management systems

• Distributed, multi-organization teams of about 50 people

• Parts of applications are relatively stable, while others are highly volatile

• Architectures are driven by a few key COTS packages that are also continually evolving

LUMS 17

SupplyChain.com Profile

Personnel





5030

105

1

90

70

50

30

10

3

10

30

100

300

35

30

25

20

15


Funds Comfort

Single Life

Many Lives


0

10

20

30

40


SupplyChain.com

Environmental Risks

E1. Technology uncertainties ��

E2. Many stakeholders �

E3. Complex system of systems �


A1. Scalability ��

A2. Use of simple design ��

A3. Personnel turnover �

A4. Too-frequent releases �

A5. Not enough agile-capable people �






�



LUMS 18

SupplyChain.com Strategy

Startup 1. Teambuilding and Shared

Vision

3. Design, Development and Deployment2. Systems Definition and Architecting

Stakeholders

Furnish CRACK representatives

and alternates

Staff and organize to cover

all success-critical risk areas

• Develop benefits-realization results

chains

• Identify missing

stakeholders• Enhance mutual

understanding

• Explore goals, options, technology,

prototypes, risks

Review; Commit

to

proceed

• Elaborate supply

chain operational concept, prototypes,

transition strategy• Evaluate and determine COTS,

reuse, and architecture choices

• Prioritize and

sequence desired capabilities,

outstanding risks

• Establish project organization, overall

process, and feature

teams

Review; Commit

to

proceed

• Ensure representative exercise of incremental capabilities

• Monitor progress and new operational

development

• Monitor project progress,

risk resolution, and new technology

developments

• Identify and work critical project-level actions

• Develop and integrate

new feature increments

• Communicate

proposed redirections

• Prepare for and execute

acceptance

tests, installation,

operational

evolution

• Analyze

feedback on current

version• Reprioritize

features

• Prepare strategy for

next

increment• Consolidate

process

lessons

learned

• Use risk to determine content of artifacts

Project Manager andRisk Management Team

Agile Feature Team LCA LCO

LUMS 19

NISCM Profile

• Broad coalition of government agencies and critical private-sector organizations

• Support cross-agency and public-private sector coordination of crisis management activities

• Adaptive mobile network, virtual collaboration, information assurance, and information integration technology

• Private-sector system-of-systems integration contractor

LUMS 20


NISCM

Environmental Risks

E1. Technology uncertainties ��

E2. Many stakeholders ��

E3. Complex system of systems ��


A1. Scalability �� — ��

A2. Use of simple design �� — ��


A4. Too-frequent releases ��

A5. Not enough agile-capable people �� — ��






��



NISCM Profile

Personnel





5030

105

1

90

70

50

30

10

3

10

30

100

300

35

30

25

20

15


Funds Comfort

Single Life

Many Lives


0

10

20

30

40

LUMS 21

NISCM Strategy

Startup Teambuilding and Shared

Vision

Design, Development and Deployment

Systems Definition and Architecting

Stable, Higher-criticality Module Teams

Stakeholders

Furnish CRACK

representatives

and alternates

• Staff and organize

to cover all

success-critical risk areas

• Prepare for and

select competitors

for concept

definition

• Develop results chains

• Identify missing

stakeholders

• Enhance understanding

• Explore goals, options, technology, prototypes,

risks

• Negotiate mutually satisfactory objectives

and priorities

• Formulate top-level

operational concept, requirements,

architecture, life cycle

plan, feasibility rational

• Select winning system

integration contract

Hold Life

Cycle

Objective

Review. If feasibility

rationale is

valid,

commit to

proceed

• Prepare for/select

competitors for

component subcontract definition

• Elaborate operational

concept, prototypes,

transition strategy

• Evaluate/determine

COTS, reuse, and

architecture choices

• Develop/ validate critical-infrastructure software

• Prioritize/sequence

desired capabilities,

outstanding risks

• Formulate definitive

operational concept,

requirements, architecture, life cycle

plan, feasibility rationale

• Use to solicit/select

component

subcontractors

Hold Life

Cycle

Architecture

Review. If feasibility

rationale is

valid,

commit to

proceed

• Ensure representative exercise of incremental

capabilities

• Monitor progress and new

operational development

• Monitor project progress,

risk resolution, and new

technology developments

• Identify/work critical project-level actions

• Continuously integrate/ test

growing software

infrastructure and

components

Dynamic, Lower-criticality Module Teams

• Develop compatible component-level prototypes, requirements,

architectures, plans,

critical software, feasibility

rationale

• Classify modules by likely

stability and criticality

Organize

into disciplined and agile

module

teams

Plan, specify, develop stable,

higher-criticality

module

increments

Plan, specify, develop dynamic,

lower-criticality module

increments

• Communicate

proposed

redirections

• Prepare for and execute

acceptance

tests,

installation,

operational

evolution

• Analyze

feedback on

current

version

• Reprioritize

features

• Prepare strategy for

next

increment

• Use risk to determine content of artifacts

NSO and I3-led ProjectRisk Management Team

LCALCO

LUMS 22

Risk Profiles Across Examples


Event Managers SupplyChain.com NISCM

Environmental Risks

E1. Technology uncertainties � ��

E2. Many stakeholders � � ��

E3. Complex system of systems � � ��


A1. Scalability � �� — ��

A2. Use of simple design � �� — ��


A4. Too-frequent releases � � ��

A5. Not enough agile-capable people � � �� — ��





D4. Not enough discipline-capable people � � ��



LUMS 23

Outline

• Tutorial learning objectives– Survey of assumptions about participants

• General software trends and implications– Sources of perplexity about agile, plan-driven methods

• Overview of agile and plan-driven methods– XP, TSP “days in the life”– Comparisons of differences, strengths, weaknesses

• Risk-based balance of agility and discipline– Small, medium, large examples

• Observations and Way Forward

• Conclusions; review of learning objectives

LUMS 24

Summary of Observations

• No silver bullet

• Clear agile and plan-driven home grounds

• Increased demands for agility, velocity, quality

• Balanced methods are emerging

• Build methods up vs. tailor down

• Methods are important; people more so – Values– Communications– Expectations

management

LUMS 25

No Silver Bullet

• Brooks’ werewolf concerns– Complexity, conformity,

changeability, invisibility

• Agile methods – Handle changeability

and invisibility– Miss on complexity and

conformity

• Plan-driven methods– Handle conformity and

invisibility– Miss on changeability

and complexity

LUMS 26

Future Applications Need Both

• Historically– Many small, non-critical,

well-skilled, agile culture, rapidly evolving projects

– Many large, critical, mixed-skill, ordered culture, stable projects

• In the future– Large projects are no

longer stable– Maintenance of extensive

process and product plans will become too expensive

– Complexity and conformity werewolves are waiting for agile projects

– Attributes of both approaches will be needed

LUMS 27

Balanced Methods are Emerging

• Agile methods– Crystal Orange– DSDM– FDD– Lean Development

• Plan-Driven methods– Rational Unified Process– CMMI

• Hybrid– Boehm-Turner Risk-

based– Code Science/Agile Plus

LUMS 28

Build up – Don’t Tailor Down

• Plan-driven methods traditionally– Define heavyweight process guidelines– Advocate tailoring– Treat mass of processes as security blanket

• Agilists traditionally– Begin with the minimum – Add as needed (and justified by cost-benefit)– Start small; extend only where necessary

LUMS 29

Maybe its not the methods!

• Agile movement has echoed a long line of warning calls

• Success of agile may be due as much to people factors as to technology

• Valuing people over processes is the most important factor in the manifesto I know I saw something

about that in the process somewhere…

LUMS 30

People

• Of the people, by the people, for the people

• Separation of concerns is increasingly harmful

• A few quotes…

– The notion of “user” cannot be precisely defined, and therefore it has no place in computer science or software engineering.Dijkstra

– Analysis and allocation of the system requirements is not the responsibility of the software engineering group, but it is a prerequisite for their work. The Capability Maturity Model for Software.

– Software engineering is not project management. Tucker

LUMS 31

Values

• Reconciling values is a critical people-oriented task

• Stakeholders value different things

• Software engineering is usually value-neutral– Every requirement, object, test case, defect equally important

• Process improvement and plan-driven methods are inwardly-focused – Aimed at productivity improvement– Not higher value to customer

• Agilists attention to prioritization and negotiation are promising

LUMS 32

Communications

• Face it, most engineers can’t talk, much less write

• Need to bridge the customer-developer communications gap

• IKIWISI* reigns

• Rapid change increases need for solid communication

• Few sources of guidance– Cockburn’s Agile Software

Development is a good starting place

*I’ll know It When I See It

LUMS 33

Expectations Management

• Differences between successful and troubled projects is often expectations management

• SW developers have problems with EM – Strong desire to please– Little confidence in prediction– Over confidence in abilities

• Most significant common factor in highly effective plan-driven or agile teams

• EM means not setting up unrealistic expectations– Process mastery– Preparation– Courage

Developers seem to like Sisyphean tasks…

LUMS 34

Do Both Assure Success?

• TSP and agile methods are indeed silver bullets and always succeed spectacularly well.

• People tend to report success more than they do failures.

• The pioneer projects are being done by high-capability early adopters of new methods.

• The pioneer projects are experiencing some Hawthorne effects of performing extra well while being in the spotlight.

• The projects are being compared to particularly poorly performing past projects.

• High improvement numbers make options worth exploring

LUMS 35

Organizational Way Forward

• Convene key stakeholders to determine organizational goals and strategies

• Rebalance agility and discipline in this organizational context

LUMS 36

Determining Organizational

Goals and Strategies

• Current software organization’s status and needs

• Key stakeholders’ needs and trends

• How to cope with future trends

• The increased pace of change and need for agility• The increased concern with software dependability and

need for discipline

• Your ability to satisfy your stakeholders’ evolving value propositions and to keep up with your toughest competitors

• The increasing gap between supply and demand for Cockburn Levels 2 and 3 people

• Your ability to cope with existing and emerging challenges– COTS integration, evolving Internet/Web capabilities,

distributed and mobile operations, agent coordination, multimode virtual collaboration, and outsourcing jobs

LUMS 37

Collins’ Good to Great- HarperBusiness, 2001

Bureaucratic Organization

Start-up Organization

Hierarchical Organization

Great Organization

Low

Low

High

High

Ethic of Entrepreneurship

Culture of Discipline

LUMS 38

Rebalancing Agility and Discipline

In agile or plan-driven

home ground

Highly mixed

agile/plan driven

profile

Graph “typical project” on 5D chart

-Now; 5 years ahead

Develop best home

ground continuous process

improvement strategy

Treat anomalies

as risks to be

managed

Develop incremental

mixed strategy. Try

on pilot project

Integrate, execute process, people, technology plans.

Monitor and control using Balanced Scorecard.

In home ground

with some

anomalies

LUMS 39

Conclusions

• Plan-driven and agile methods both aim to– Satisfy customers– Meet cost and schedule parameters

• Home grounds exist, but the need/opportunity for integration is expanding

• We recommend a self-assessment of your organization and business – Locate your place in the home ground space– Identify areas of change and how they might move your location

– Look for ways to integrate methods to better respond to change

• People concerns dominate method concerns

LUMS 40

References

• Barry Boehm and Richard Turner, Balancing Agility and Discipline: Evaluating and Integrating Agile and Plan-Driven Methods, STC 2004 Tutorial, 2004.

• Barry Boehm and Richard Turner, Balancing Agility and Discipline – A Guide for the Perplexed, Addison-Wesley, 2004.

balancing agility and discipline (part 2) -...

Documents