iii conferência cmmi portugal, keynote 1: agile methods and capability maturity: addressing the...
DESCRIPTION
Keynote discussing how agile and CMMI can work together, the challenges each one has and how agile organisations move towards CMMI Level 5.TRANSCRIPT
Portugal
Agile Methods and Capability Maturity:
Addressing the Paradox
Professor Ian Allison
Head of School of Computing Science and Digital Media
Robert Gordon University, Aberdeen, UK
2013-10-18
2 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Introduction
3 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Introduction
4 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Introduction
5 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Introduction
6 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Themes
• Background
• Agile – the maturity paradox?
• Case study of a high maturity organisation
• Good practices to be considered
7 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Agile Development
8 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Agile Processes
9 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Current Status of Agile
84% of organisations using agile
50% started in the last 2 years
35% of companies had distributed agile projects
Main Methodology: SCRUM [54%& 11% SCRUM/XP hybrid]
Key benefits:
• Manage change
• Improve productivity
• Improve project visibility
• Enhanced quality of software
VersionOne, 2013
10 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key techniques
0 20 40 60 80 100
Daily Standup
Iteration Planning
Unit testing
Retrospectives
Release Planning
Burndown/team estimation
Velocity
Coding Standards
VersionOne, 2013
11 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
The Problem With Agile
12 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Leading Causes of Failure
• Culture at odds with agile 12%
• External pressure to follow waterfall 11%
• Broader organisational communication problem 11%
• Lack of experience with agile 9%
• Lack of management support 6%
(18% claimed no failed projects)
VersionOne, 2013
13 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Main Concerns with Agile
• Lack of upfront planning
• Loss of management control
• Management opposition
• Lack of documentation
• Lack of predictability
• Lack of engineering discipline
• Inability to scale
• Regulatory compliance
VersionOne, 2013
14 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
How Agile Addresses the CMMI Practices in
3 Case Studies
CMMI Goal Agile Practices
Requirement management
Stories; product & sprint backlog; sprint planning; sprint reviews; planning games; information radiator; Daily meetings; on-site customer; self-organising teams; planning days; release days;
Estimates Sprint planning; product backlog; Sprint backlog; Planning games; tasks and effort estimations 1-2 weeks ahead; information radiator; planning days
Project Planning Planning games; task on information radiator; product backlog
Commitment to plan
Planning games; self-organising teams; on-site customer; reflection workshops; planning days; task cards on information radiator; release days
Project tracking
Short planning cycles; Burn down charts; project velocity; visual control; daily meetings;
Configuration Management
Collective ownership; small releases; continuous integration;
Paulk, 2001 & Pikkarainen & Mantyniemi, 2006
15 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Specific Goals – Spanish Experience
0
2
4
6
8
10
12
Garzas & Paulk, 2013
16 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
How Agile Companies Move Towards CMMI
Level 5
DTE Energy (Baker, 2005)
• Important role for SEPG
• Create practice groups: Project Management; Core
development; Process Management & Measurement
• PMO; Information office
• Education & Socialisation of the idea of CMMI 5
• Communication – messages to key audiences
Pragmatics (Cohen & Glazer, 2009)
- Metrics programme that adds value
17 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Challenges for High Maturity
• Need for recorded evidence
• Need for project planning and control
• Self-Organizing vs Planned
• Organisational level of standardisation of process
• Project Metrics and performance prediction
• Organisational level data base-lining
18 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
High Maturity Case Study
• A CMMI level 5 Indian organisation working onshore &
offshore development
• Large enterprise with a revenue of $450m
• Study over the last year included site visits that allowed
interviews with 38 practitioners, observations and a review
of confidential agile process documents
19 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Agile History
• Started using in 2002… ad hoc use…
“so small projects here and small projects there, but since
2009 we have seen a huge uptake of agile …some
[projects] because the customer says; some because
there were a lot of problems so we recommended [agile]”
(Corporate Lead Architect)
• Then…
“the size of agile projects really increased. So instead of
having 20 or 10 members on projects, now suddenly we
have 140, you know large offshore development
programs being run in agile” (Corporate Lead Architect)
• Now 600 people … 20% of workforce on agile projects
20 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Recorded Evidence
• Records required in CMMI for:
• Decision rationale
• Project review and evaluation
• It is possible to have evidence in many other forms…
21 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Recorded Evidence
• Records required in CMMI for:
• Decision rationale
• Project review and evaluation
• It is possible to have evidence in many other forms…
22 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Recorded Evidence – Case Study
Practice Head stated that “Sufficient evidence is required so
that “Senior Management Review and Risk Management can
be performed [so] there is no contradiction with agile”
Evidence is typically recorded on agile projects often takes
the form of:
• source code annotations (for lowest level decisions)
• email discussions
• Intranet project content management systems
• Wiki information repositories that organically evolve over
the project lifetime
23 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
24 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
25 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Project Planning and Control
In CMMI it is important to have defined schedules, built from
defined requirements, upfront design and known productivity
rates
In agile approaches, evolution and responding to change
takes precedence over upfront planning.
However, the project plan that is expected in CMMI need not
be “what” but can focus on “how”.
26 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Project Planning and Control – Case Study
It was clear to the organisation that “every project needs to
have a project goal and a defined process … so, preparing
the initial project plan is not a problem” (Practice Head –
SEPG).
Partly this was because even in agile …
“in a distributed environment involving large team size, [then]
some form of upfront design is essential” (RK – Agile Coach,
Feb. 2013)
27 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Self-organising vs Planned
In CMMI:
• a consistent set of processes an organization is believed
to be able to reduce the variability between projects
• rigorous project management and control through a known
set of software engineering activities is the way this is
achieved.
In Agile:
• the emphasis is on self-organization and emergence
• product features emerge as the development progresses.
• release plan is flexible and expected to emerge based on
perceived business value.
28 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Self-organising vs Planned
As the Practice Head – SEPG put it “there is nothing in CMMI
which states that teams cannot be self-organizing.”
So the balance between planned and self-organizing was
struck by emphasizing the common elements of goal-setting
and enabling some upfront design to support the distributed
context.
29 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Organisational Level
Standardisation of Processes
In SCRUM:
• it is commonplace to add engineering practices (e.g. XP)
• set of roles, practices and ceremonies defined is
rather small.
In the CMMI:
• large super-set of processes are defined;
• project teams must select practices from this
large set of process definitions.
30 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Need for Organisational Level
Standardisation of Processes – Case Study
When a project starts “we generally assign a coach to that
project” (Corporate Lead Architect)
To assess process compliance SEPG “will do monthly
reviews of the projects. And they also do audits to see if the
practices are being followed; are they following
continuous integration or are they making daily
builds” (Corporate Lead Architect)
31 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Project Metrics and Performance Prediction
• Metrics used for decision making and software
development process management to compare
between projects, spot trends, and anticipate
problems as well as for managing an individual
project.
• The data needs to be appropriate and part of the
normal process to minimize the effort of capturing
the data.
• CMMI recommends the use of a quantities model
to predict future performance. As long as the model
is specific to the project there is no contradiction.
32 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Project Metrics and Performance Prediction
– Case Study
• The company found “we faced problems in collecting data
required for level 4 KPA’s. The team felt that it was an
overhead.” (Delivery Head)
• “[The] most important aspect of complying with the metrics
needed for CMMI is to understand that traditional metrics
like, productivity and time slippage [are] not applicable …
instead we have been capturing metrics on defect,
resource utilization and on improvement.” (Practice Head –
SEPG)
33 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Organisation level data base-lining
The problem in Agile is the consistency of data between
teams to enable comparisons and trends to be seen.
Traditional data sets are not relevant for agile projects
It takes time to develop a data set from which organisations
can predict future activity
34 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Organisation level data base-lining – Case
Study
The problem is that “some projects use story points and
some projects do not use story points. They use function
points” (Corporate Lead Architect)
The “initial problem with organizational metrics was that we did not
have sufficient data points … with more agile projects getting
executed that is getting resolved.” (Practice Head – SEPG)
SEPG defined an activity based estimation model. “So for example
you are saying this is a four story point for this feature... Okay so we
are trying to define some guidelines on the basis of what you will do
with your story point estimation. But that is against your agile
concept. It should be spontaneous [within any given team]”
(Program Head)
35 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
36 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
37 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
38 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
39 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
• SEPG to use a team approach to define a process map for
agile projects that allows local tailoring, but also to add
elements such as risk management
40 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Key Recommendations
• Establish key forms of record keeping techniques and
ensure they are auditable
• Define a project planning and control process based on
collaborative short-term planning and visual control
• In larger settings, use an architecturally-driven agile
approach for goal setting in large distributed agile projects
balanced with local self-organisation for devolved design
• SEPG to use a team approach to define a process map for
agile projects that allows local tailoring, but also to add
elements such as risk management
• Develop a framework that enables the capture of data
relevant to their agile projects with consistent use of terms
41 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Summary
• Challenges in aligning CMMI and Agile but no major
contradictions
• Agile environments can mature – and this can be achieved
through using an SEPG team approach to agreeing
practices, data, etc
• Agile in an offshore, large-scale distributed context will
naturally deviate from ‘pure’ agile – making CMMI links
more natural
• We have identified some embryonic guidelines
• Just one major case – so need to extend the collection
42 © 2
013 C
MM
I P
ort
ug
al C
onfe
rence S
eri
es –
All
Rig
hts
reserv
ed.
Questions