Measuring the Success of Your Agile TransformationPart 1
June 19, 2014
#AgileLIVE
© 2014 VersionOne, Inc. 2
A few logistics…
#AgileLIVE
© 2014 VersionOne, Inc. 3
Moderator
#AgileLIVE
Aaron SmithEditorial DirectorProjectsAtWork
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Brad SwansonSenior Agile Coach
@bradswanson
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
The Enterprise Transition
Framework™
(ETF) for empirical change
1. Assess2. Strategy
3. Pilot4. Rollouthttp://www.agile42.com/en/agile-transition/etf/
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal PSF NC
Agile Strategy Map™
Define the Goal we want to achieve
Possible Success Factor (PSF) represents a hypothesis about a key factor that would enable us to achieve the Goal
Necessary Condition (NC): what is needed in order to fulfill the Success Factor
http://www.agile42.com/en/agile-transition/agile-strategy-map/
Agile Strategy Map™ - Framework for Strategy
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal
Agile Strategy Map™
PSF#1
PSF#1: we believe that by consolidating all projects in one Portfolio, we will be able to make better business decisions
NC#1.1
NC#1.2
NC#1.1.1
To validate the hypothesis, we define what is needed to run a pilot which will measure the validity of the PSFNC#1.1: we need to list at least 3 projects per product areaNC#1.1.1: we need to choose projects and define comparable dimensions such as: business value, strategic alignment, cost...NC#1.2: we need to define a sample Portfolio funnel, and check that Teams can deliver with the provided information
What conditions are necessary to achieve the success factor (PSF)?
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Success: we can compare the
value of projects easily (using defined KPIs) and
make consistent business decisions.
Failure: we still have endless
discussions about priority and it’s costly to prepare
project info for the portfolio processIndicators (measures):
•< 1 day to prepare 1 project business case
•Allows objective comparison of projects
•< 8 hrs to select priorities each quarter
•No ‘exec overrides’ on priority decisions
How will we measure PSF success and failure?
PSF#1: we believe that by consolidating all projects in one Portfolio, we will be able to make
better business decisions
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Goal CSF#1NC#1.
1
NC#1.2
Agile Strategy Map™
CSF#1: we must consolidate all projects in one Portfolio, so that will be able to make better business decisions
NC#1.1.1
The Pilot was successful...
The Potential Success Factor becomes a Critical Success Factor (CSF). Update the strategy to roll out and institutionalize the change.
Pilot success! Now what?
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Sustainably faster
delivery of solutions that
delight customers
Faster time to market
Alignment & sup-port across the
org
Engaged & joy-ful people
Transparent empirical view
of org performance
Grow and sus-tain agile prac-
tices
Culture of continuous learning & improve-
ment
Org priorities aligned with
Customer value
Agile TransformationExample Goal + Success Factors (PSFs /
CSFs)
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Sustainably faster delivery of solutions
that delight customers
Faster time to market
Balance work-load & capacity:
limit WIP
Alignment & support across
the org
Engaged & joy-ful people
Transparent empirical view of org
performance
Grow and sustain agile
practices
Culture of continuous learn-ing & improve-
ment
Org priorities aligned with
customer valueDedicated Guiding Coalti-
ion
Pilot projects
Clearly communicated vision & strat-
egy
Training & coach-ing
Build internal expertise
Collaborative prioritization
with stakehold-ers
Collaborative cross-functional
teams Automated testing &
deploymentIncremen-tal deliv-
ery
Visible met-rics
Leaders to remove
fearEmpower-ment
Clear, realistic goals
Servant leader-
ship
Action taken
to improve
Foster innova-
tion Celebrate failed experi-ments
Example: Necessary Conditions for the PSFs
Clear career path
GoalSuccess fac-
torsNecessary condi-
tions
Legend:
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
A few metrics (KPIs) for the goalSustainably faster delivery of solutions that delight customers
These are lagging indicators; some leading indicators are described soon…
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Leading & lagging indicators for each PSF / CSF
PSF Leading Indicators Lagging Indicators
• # total concurrent projects• Avg # concurrent
projects/person• Program burn-up chart
• Lead time: idea to 1st delivery• Cycle time: Dev start to 1st
delivery• Deployment frequency post
1st delivery (MVP/V1)
• Frequency & effectiveness of stakeholder collaboration on prioritization
• Standard way to compare project value
• Portfolio priorities highly visible
• All teams working on top portfolio priorities; no ‘side projects’
• Delivered value is measured and meets expectations
• Customer satisfaction survey• Sales and/or usage statistics
1. Faster time to market
2. Org priorities aligned
with cus-
tomer value
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
CSF 1: Time to Market
Lead time: from request to 1st deploy
Cycle time: dev start to 1st de-ploy
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
CSF 1: Time to Market – leading indicator
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 2: Org priorities aligned with customer value –
leading indicators
Churn = % change af-ter commit
point
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Compare actuals
with forecast
PSF 2: Org priorities aligned with customer value –
lagging indicators
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Leading & lagging indicators for each PSFPSF Leading Indicators Lagging Indicators
• Sr. Leadership consensus on strategy
• Frequency of comms on vision & strategy; # of different forums
• Short-term wins visibly celebrated
• # hours spent on Guiding Coalition activities
• Poll: % who agree with strategy
• Poll: % prefer new to old approach
• # of teams asking to ‘go agile’
• # people trained on Agile• # people in coached
projects• # people in the ‘coach the
coach’ program
• Agile team practices maturity level
• # people completing ‘coach the coach’
• # people achieving 3rd party Agile certifications
• Monthly attendance in Agile Community of Practice (CoP) mtgs
4. Grow & sustain
agile prac-tices
3. Align-
ment & support across the org
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 3: Alignment & support
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
PSF 4: Grow & sustain agile practices
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Leading & lagging indicators for each PSF
*Success ratio over 70% may indicate fear of failure
PSF Leading Indicators Lagging Indicators
• Poll: things getting better or worse?
• Career path defined for all roles
• Poll: employee job satisfaction
• Attrition rate• # employee referrals for
hiring
• Avg frequency of team retrospectives
• # of experimental learnings celebrated (fast safe failures)
• Poll: Do you feel safe to experiment & innovate?
• # team-initiated process changes
• # team-initiated product innovations
• Success ratio of experiments*
5. Engaged & joyful people6. Cul-ture of continu-
ouslearning
& Improve-
ment
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Measuring Team Progress on Agile Practices
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Agile Practices Roadmap – Team Level
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Agile Practices Roadmap – Team Level (p.2)
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Team Practices Dashboard
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Team Practices Dashboard
Highest level achieved across all four di-mensions
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Attack of the Killer Metrics!
When metrics go bad…
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Questions to ask about every metric1. What decisions will be made based on this
metric?2. Is this the RIGHT thing to measure or an easy-to-
measure proxy?3. What might be the unintended consequences of
this metric?4. How could this metric be ‘gamed’? 5. What can we do to guard against unintended
consequences and ‘gaming’?
agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
Compensating / Balancing Metrics• What unintended consequences might happen if you measure speed of delivery?
• What is a compensating metric to counteract it?
• Measure speed AND quality!
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.agile42 | The Agile Coaching Company
Management vs. Team Metrics
https://creativecommons.org/licenses/by-nc-sa/2.0/
https://www.flickr.com/photos/joeshlabotnik/ https://www.flickr.com/photos/jeroen020/
www.agile42.com | All rights reserved. Copyright © 2007 - 2014.
agile42 - The Agile Coaching Company
agile42 | The Agile Coaching Company
Thank you!
Brad SwansonSenior Agile Coach
@bradswanson
© 2014 VersionOne, Inc. 32
Q&AJoin us for Part 2, June 25, Noon-1 PM ET
Thank you for joining us today!
Join us for Part 2, June 25, Noon-1 PM ET
#AgileLIVE