estimate value to deliver value: effectively estimate the value of requirements with the experience...

agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013

Estimating Value to Deliver Value

EFFECTIVELY ESTIMATE THE VALUE OF REQUIREMENTS WITH AN EXPERIENCE CANVAS


The Sky is Falling! The Sky is Falling!

Applying the Scientific Method to Software Development

Incorporating Learning into Product Development

A Last Note on Shorter Feedback Loops


“No matter how good the team or how efficient the methodology, if we’re not

solving the right problem, the project fails.”

Woody Williams


http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspx Archives of Ontario, Reference Code: C 190-5-0-0-21

Assembly Line Manufacturing has a high cost of change


• In software development, (presumed) high cost of change leads to long delivery cycles

• Any and every idea has to be captured in the first version of a requirements specification

• This creates waste - bloated documents, unwanted features and entitlement thinking

Inclusive thinkingFeature usage in

enterprise softwareNeverRarelySometimesOftenAlways

Standish Group - CHAOS Report, 1995, 2002


Cost of Change

Detail of Requirements

Hypothesized Requirements

Capture all possible needs


Software Development has commoditized cost of change

Continuous Delivery

Ruby on Rails

Coffeescript

jquery

Object-oriented languages

DevOps

Automated Tests


Cost of Change


Evolving Requirements


Emerging needs as development

progresses



Usability Testing - c.2000


Usability Testing - today

http://welovelean.wordpress.com/2012/01/29/15-essential-tools-every-lean-startup-cant-li/


• If change is cheap, requirements can change continuously

• We can evolve our thinking as we learn more about the product we are building

• Maximizing Return-on-Investment requires validating new features early, before the total investment cost has been spent

Changing paradigms


Cost of Change


Validated Requirements

Evolving Requirements


Lean Startup experiments

Emerging needs as development

progresses



Designed by KISSmetrics


Theory

Prediction

Experiment

Observe

Use the theory to make a prediction

Design an experiment to

test the predictionRun the

experiment

Modify or change your theory

The Scientific Method


The Theory of General Relativity

Einstein’s Theory of General Relativity

Light bends in a gravitational field

Solar Eclipse of 1919

Observations validated General Relativity

Einstein’s prediction

(1907)

Wait for 1919 solar eclipse to observe

whether or not light bends around the sun

Arthur Eddington observed that Light did bend around the sun

Gradual acceptance of General Relativity over Newtonian Mechanics

http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/


User Model

Prediction

Experiment

Validate

Use the model to make a prediction

Design an experiment to

test the predictionRun the

experiment

Modify or change your user model

Applied to an Online Experience


VisionPlanning

RoadmapPlanning

ReleasePlanning

ReleasePlanningItera3onPlanning

DailyPlanning

Five Levels of Planning

Annually Define a vision

Bi-annuallyRank features in order of delivery

Quarterly Plan stories to be delivered during next release cycle

Bi-weeklySprint planning, focussing on delivery of sprint goal

DailyFocus on daily planning of team activities

VisionPlanning

RoadmapPlanning

5 Levels of Agile Planning: From Enterprise Product Vision to Team Stand-up, Hubert Smits (2006)


VisionPlanning

RoadmapPlanning

ReleasePlanning


DailyPlanning


Long-termDefine & refine behavioural models we are testing? Pivots happen here.

Medium-termValidate understanding with Lean (or Customer Experience) Canvas

RegularlyMinimum Viable Product. Validate ideas before fully committing investment

Bi-weeklySprint planning, focussing on delivery of sprint goal

DailyFocus on daily planning of team activities

VisionPlanning

RoadmapPlanning


Learning


Recognize there are gaps in your vision/business model, and look to fill those gaps in as quickly as possible

Known Unknowns

Map of Africa, 1824http://culturalstudieslectures.blogspot.com/2012/01/

lecture-one-on-question-what-is.html


“In a nutshell, the job of the product organization is to make that vision a reality. So we don't work for a few weeks or even a few months and then just declare it's not happening and we should do a pivot. We will typically work hard for many months on a product vision and not even entertain the possibility of giving up.”

Marty Caganhttp://www.infoq.com/news/2013/07/pivoting-product-development


VisionPlanning

RoadmapPlanning

ReleasePlanning


DailyPlanning



Medium-termValidate understanding with User Story Mapping and Lean Canvases


Bi-weeklyProduct Owners, Delivery Team

DailyProduct Owners, Delivery Team

VisionPlanning

RoadmapPlanning


Learning


User Story Mapping is a powerful tool for understanding

your product from the perspective of the user

agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright ©2007-2016.

Photo: Rory Mizen

Highest Paid Person’s Opinion


Validated Learning

• Understand the problem a feature will solve

• Decide how to ask if your customer cares

• Validate before investing complete cost


1.Problemtop3problems

Exis3ngalterna3ves

4.Solu3ontop3features

8.Keymetricse.g.AARRR

3.Uniquevalue

proposi3onClearcompellingmessage.Whymakesyoudifferent?

2.Customersegments

Targetcustomers

Earlyadopters?

9.UnfairadvantageNoteasilycopied

orbought

5.ChannelsPathtocustomers

7.CostStructureCustomeracquisi3on,distribu3on,people,

etc.

6.RevenuestreamsRevenuemodel,life3mevalue,revenue,

grossmargin

Lean Canvas: Ash Maurya

LeanCanvas,fromAshMaurya:RunningLean.LicensedunderCrea3veCommons-A\ribu3on-ShareAlike3.0

Product Market


Customer Experience Canvas™ has been created by agile42 as an extension to the Business Model Canvas of Alex Osterwald and is licensed using Creative Common 3.0 with attribution (by), non commercial usage (nc) and share alike (sa) options. You can reuse and modify the template with the logo as often as you wish.

Customer Experience Name (title)

1. Opportunity 2. Customer SegmentsWhat is the problem to be solved? What type of customers & users will

benefit from this solution?

How is the customer solving the problem right now?

5. Business ReadinessWhat steps are required from the business side to be able to use this capability?

4. BenefitsWhat are the benefits for the customers?

What are the benefits for internal stakeholders?

6. Measuring SuccessWhat metrics will be best measure the success of the feature?

3. Possible SolutionWhat are the key points of a possible solution to the presented problem?

7. Cost of DelayWhich profile better represent the cost of delay (CoD)?

8. Costs StructureHow does the cost structure look like for such a feature? One time, ongoing costs, contractors expenses, development costs?

9. Value to Customer and BusinessWhat are the expected incremental revenue for selling this feature, and what are the strategic and tactical benefit? What are the intangible values (usability, performance, customer knowledge obtained...)

Customer Experience Canvas


time

linear

dela

y co

st

time

must do now

dela

y co

st

time

logarithmic

dela

y co

st

time

intangiblede

lay

cost

Cost of Delay is the cost we incur by delaying delivery of a product, due to lost revenue or unrealized cost savings.

Cost of Delay profiles

time

fixed date

dela

y co

st

Kenny Rubin: Essential Scrum


42 Investment Group: building a luxury resort hotel

Goals: • Luxury

amenities & activities

• Destination hotel

• Repeat customers

• Profitable in 1st year

https://www.flickr.com/photos/28685147@N04/


Portfolio of Projects / Features

• 18 hole golf course• Jacuzzi tubs in guest rooms• Valet Parking• Free internet access • On-site tailor shop• Child care centre• Pet spa• Mini golf

• Bungee jumping tower • Full service spa• Healing mud baths• Swim with dolphins• Concierge service• Live music nightly• Amusement park & roller

coaster• Themed guest rooms


Customer Experience Canvas™ has been created by agile42 as an extension to the Business Model Canvas of Alex Osterwald and is licensed using Creative Common 3.0 with attribution (by), non commercial usage (nc) and share alike (sa) options. You can reuse and modify the template with the logo as often as you wish.

Customer Experience Name (title)

1. Opportunity 2. Customer SegmentsWhat is the problem to be solved? What type of customers & users will

benefit from this solution?

How is the customer solving the problem right now?

5. Business ReadinessWhat steps are required from the business side to be able to use this capability?

4. BenefitsWhat are the benefits for the customers?

What are the benefits for internal stakeholders?

6. Measuring SuccessWhat metrics will be best measure the success of the feature?

3. Possible SolutionWhat are the key points of a possible solution to the presented problem?

7. Cost of DelayWhich profile better represent the cost of delay (CoD)?

8. Costs StructureHow does the cost structure look like for such a feature? One time, ongoing costs, contractors expenses, development costs?

9. Value to Customer and BusinessWhat are the expected incremental revenue for selling this feature, and what are the strategic and tactical benefit? What are the intangible values (usability, performance, customer knowledge obtained...)

Activity 1: Build a Customer Experience Canvas

• Choose a project or two from this list at each table

• In groups of 3-4, write an Experience Canvas, • Problem/opportunity & existing alternatives • Customer segments • Possible solution • Business value • Cost of Delay profile


Estimate the relative value of projects in a portfolio

100 200 500 800 1200 2000 3000


Activity 2: Business Value Game•Play the Business Value Game with 1-2 of the groups around you •Arrange blue cards from smallest to largest value •Arrange canvases as shown in the sample here •Write the business value number on each Canvas

100 200 800 1200 2000 3000

1200

1200 2000200

200


ROI Calculation & Project Sequencing

Project Relative Value Relative Effort (cost) Relative R.O.I. Sequence?

A 1200 3 400

B 800 3 267

C 2000 8 250

D 3000 40 75


Activity 3: Determine How to Sequence Projects

• Write the Cost number on each canvas • Calculate Relative ROI for each canvas. Write it on

each canvas • ROI = Value / Cost

• Sequence projects based on ROI (high to low) • Consider Cost of Delay profile: does this change

the sequence?


ROI calculation & project sequencing

Project Relative Value

Relative Effort (cost)

Relative R.O.I. CoD Sequence?

A 1200 3 400

B 800 3 267

C 2000 8 250

D 3000 40 75


VisionPlanning

RoadmapPlanning

ReleasePlanning


DailyPlanning



Medium-termValidate understanding with User Story Mapping and Lean Canvases


Bi-weeklyProduct Owners, Delivery Team

DailyProduct Owners, Delivery Team

VisionPlanning

RoadmapPlanning


Learning


Zappos famously launched without stock, selling shoes from their local specialist show store


Minimum Viable Product

A Minimum Viable Product may be an entire product or a sub-set of product (such as a feature):

• Product - The canonical MVP strategy for a web application is to create a mock website for the product and purchase online advertising to direct traffic to the site. The mock website may consist of a marketing landing page with a link for more information or purchase. The link is not connected to a purchasing system, instead clicks are recorded and measure customer interest.

• Feature - A link to a new feature in a web application might be shown in a prominent location on the website. The feature is not implemented, rather an apology, mock-up, or marketing page is provided. Clicks of the link are recorded and provide an indication as to the demand for the feature in the customer base.

It is ALWAYS smaller than you think!


• In 2009, Google ran approximately 12,000 experiments, of which about 10% led to business changes

• Linus Pauling famously said, "The best way to have a good idea is to have a lot of ideas."

• Focus on minimizing the incremental cost of testing ideas with your customers/users

How Much Effort on ValidationFeature usage in

enterprise softwareNeverRarelySometimesOftenAlways

Standish Group - CHAOS Report, 1995, 2002


One test is worth a thousand expert

opinions.

Wernher Von Braun

Interaction Model Backlog ValidationDoing Doing Deployed

For <target group> Who <has a specific need> The <product name> Is A <product category> That <key benefit> Unlike <primary competitors> Our Product <additional advantages>

Pathway #1

Pathway #2

•Visualize your User Interaction Model and adjust it as you learn•Come up with ideas for each pathway•Prioritize experimental testing•Spend time on measurement infrastructure (a test without ability to measure = an unvalidated feature)

Level of Detail / Close to done

•Always revisit validation tests for size •The execution board is equivalent to the team’s task board, but at the Idea Card level

Idea #1.1

ExecutionValidated?

Idea

#1.2

Idea

#1.3

Idea

#2.1

Idea

#2.2

To Do Observation

Idea

#2.3

Idea

#2.4

Idea

#1.4

Idea

#1.6

Idea

#1.5

http://www.agile42.com/en/blog/2013/06/17/product-owner-board/


Economists

In November 2007, economists in the Survey of Professional Forecasters — examining some 45,000 economic-data series — foresaw less than a 1-in-500 chance of an economic meltdown as severe as the one that would begin one month later.


In 1940, the chance of an American being killed by lightning was about 1 in 400,000.

Today, it’s 1 in 11 million.

vs Weatherman

http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a-

moron.html


Shifting Paradigms

“I know that half of my advertising

budget is wasted, but I’m not sure

which half” Lord Leverhulme

B.G.|A.G.

http://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditional http://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.php

agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2016.

Estimating Value to Deliver Value Effectively Estimate the Value of Requirements with a Customer Experience Canvas

Thank you @davesharrock [email protected]


compliance international B2B MBA English IPO

agile husband start-up technology

newly-minted Canadian

executive leanstartup outsourcing father enterprise transitions

B2C data analysis kanban seismology PhD

scrum organizational excellence

[email protected] twitter: @davesharrock Certified Scrum Coach (CSC) Certified Scrum Trainer™ (CST)

Dave Sharrock


agile42 The Agile Coaching Company. agile42 is a market leader in providing agile transformation services to companies looking to adopt a more efficient and adaptive way of working. By continually refining our approach, agile42 has developed a unique and proven framework for supporting organizations in realizing the benefits of a agile & lean practices. We provide assessment, strategic consulting, training, and coaching to get you results. Our customers regularly confirm that they achieve: • Accelerated revenue generation

• Faster Return-on-Investment

• Improved technical quality

• Rapid delivery to market

• Increased team motivation


Reference Section: The process steps for sequencing projects


Process for Sequencing Projects

1. Stakeholders and Product Owners prepare canvases. Optional: limit number per stakeholder

2. Review & validate canvases for completeness & understanding - ahead of time

3. Select appropriate group of voting stakeholders and a neutral facilitator

4. Brief “sales pitch” for each project, based on the canvas


Process for Sequencing Projects (2)

6. Choose a baseline project/canvas, assign it a mid-range point value. Can shift up/down later if necessary.

7. Business Value GameTM,1; assign the relative value of each project, without considering cost/effort

1agile42 2Innovation Games®

Alternative to Biz Value Game: Buy a Feature / Prune the Product Tree 2


Process for Sequencing Projects (3)

8. Delivery teams estimate cost/effort.

9. Calculate relative ROI: value / cost

10.Sequence projects, and/or allocate investments:

1. ROI

2. cost of delay

3. people and/or resource constraints