productcamp vancouver 2013
DESCRIPTION
Product management as a discipline is undergoing a transition towards data-driven decision making. As the cost of testing product feature ideas falls, the role of a product manager becomes one based on testing many feature ideas, and rolling out proven features. In this presentation we touch on some of the trends behind this paradigm shift, and consider some tools the product manager can use.TRANSCRIPT
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Lean Product ManagementThe Value of Experiments
Photo: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space
@DaveSharrockProductCampVancouver 2013
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
http://www.archives.gov.on.ca/english/on-line-exhibits/d-day/big/big_03_airline_assembly.aspxArchives of Ontario, Reference Code: C 190-5-0-0-21
Assembly Line Manufacturing has a high cost of change
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
• In software development high cost of change leads to inclusive thinking
• Any and every idea has to be captured in the first version of a requirements specification
• Creates waste - bloated documents, unwanted features and entitlement thinking
Inclusive thinkingFeature usage in
enterprise software
45%
19%
16%
13%
7%
NeverRarelySometimesOftenAlways
Standish Group - CHAOS Report, 1995, 2002
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Cost of Change
Detail of Requirements
Hypothesized Requirements
Capture all possible needs
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Software Development has commoditized cost of change
Continuous Delivery
Ruby on Rails
Coffeescript
jquery
Object-oriented languages
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Cost of Change
Detail of Requirements
Evolving Requirements
Hypothesized Requirements
Emerging needs as development
progresses
Capture all possible needs
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
• If change is cheap, requirements can change continuously
• We can evolve our thinking as we learn more about the product we are building
• Maximizing Return-on-Investment means validating new features early, before the total investment cost has been spent
Changing paradigms
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Cost of Change
Detail of Requirements
Validated Requirements
Evolving Requirements
Hypothesized Requirements
Lean Startup experiments
Emerging needs as development
progresses
Capture all possible needs
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Shifting Paradigms
“I know that half of my advertising
budget is wasted, but I’m not sure
which half”Lord Leverhulme
B.G.|A.G.
http://www.marketingprofs.com/charts/2009/3161/more-smbs-use-online-media-than-traditionalhttp://marketing.blogs.ie.edu/archives/2011/01/trends-in-online-advertising-in-2011-in-the-united-states.php
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Validated Learning
• Understand the problem a feature will solve
• Decide how to ask if your customer cares
• Validate before investing complete cost
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Theory
Prediction
Experiment
Observe
Use the theory to make a prediction
Design an experiment to
test the predictionRun the
experiment
Modify or change your theory
The Scientific Method
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
The Theory of General Relativity
Einstein’s Theory of General Relativity
Light bends in a gravitational field
Solar Eclipse of 1919
Observations validated General Relativity
Einstein’s prediction
(1907)
Wait for 1919 solar eclipse to observe
whether or not light bends around the sun
Arthur Eddington observed that Light did bend around the sun
Gradual acceptance of General Relativity over Newtonian Mechanics
http://thethoughtstash.wordpress.com/2011/01/03/how-eddington-demonstrated-that-einstein-was-right/
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
User Model
Prediction
Experiment
Validate
Use the model to make a prediction
Design an experiment to
test the predictionRun the
experiment
Modify or change your user model
Applied to an Online Experience
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Designed by KISSmetrics
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
User Story (Journey) Mapping
Story Mapping © Jeff Patton, all rights reserved, www.AgileProductDesign.com
Elements of Customer Journey
Diff
eren
t Cus
tom
er
Exp
erie
nces
Your model for customer interaction covers each step of the customer
journeyEach row matches a different
customer experience
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Vanity vs. Actionable Metrics
Vanity metrics (like web hits or number of downloads) which only serve to document the current state of the product but offer no insight into how we got here or what to do next.
An actionable metric is one that ties specific and repeatable actions to observed results.
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
http://500hats.typepad.com/500blogs/2007/09/startup-metrics.html
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Depersonalize Decision Making
• “HiPPO’s” rule the business world • It is never about you, it can’t be
about you • Benchmarking is awesome • Leverage competitive analysis • Experimentation / testing rock
• Execution strategy: • Transparency, standardization,
looking outside in • Be a slave to customer
centricity (its about your customers (internal & external))
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
“I can’t say it any better, HiPPO’s rule the world, they over rule your data, they impose their opinions on you and your company customers, they think they know best (sometimes they do), their mere presence in a meeting prevents ideas from coming up. The solution to this problem is to depersonalize decision making, simply don’t make it about you or what you think. Go outside, get context from other places. Include external or internal benchmarks in your analysis. Get competitive data (we are at x% of zz metric and our competition is at x+9% of zz metric).Be incessantly focussed on your company customers and dragging their voice to the table (for example via experimentation and testing or via open ended survey questions). Very few people, HiPPO’s included, can argue with a customer’s voice, the customer afterall is the queen / king! : )”
Avinesh Kaushik, kaushik.net
Interaction Model Backlog ValidationDoing Doing Deployed
For <target group>Who <has a specific need>The <product name>Is A <product category>That <key benefit>Unlike <primary competitors>Our Product <additional advantages>
Pathway#1
Pathway#2
•Visualize your User Interaction Model and adjust it as you learn•Come up with ideas for each pathway•Prioritize experimental testing•Spend time on measurement infrastructure (a test without ability to measure = an unvalidated feature)
Level of Detail / Close to done
•Always revisit validation tests for size •The execution board is equivalent to the team’s task board, but at the Idea Card level
Idea#1.1
ExecutionValidated?
Idea
#1.2
Idea
#1.3
Idea
#2.1
Idea
#2.2
To Do Observation
Idea
#2.3
Idea
#2.4
Idea
#1.4
Idea
#1.6
Idea
#1.5
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
• In 2009, Google ran approximately 12,000 experiments, of which about 10% led to business changes
• Linus Pauling famously said, "The best way to have a good idea is to have a lot of ideas."
• Focus on minimizing the incremental cost of testing ideas with your customers/users
How Much Effort on ValidationFeature usage in
enterprise software
45%
19%
16%
13%
7%
NeverRarelySometimesOftenAlways
Standish Group - CHAOS Report, 1995, 2002
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Feedback loops make all the difference
In November 2007, economists in the Survey of Professional Forecasters — examining some 45,000 economic-data series — foresaw less than a 1-in-500 chance of an economic meltdown as severe as the one that would begin one month later.
http://www.nytimes.com/2012/09/09/magazine/the-weatherman-is-not-a-
moron.html
In 1940, the chance of an American being killed by lightning was about 1 in 400,000.
Today it’s 1 in 11 million.
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright ©2007-2013
Lean Product ManagementThe Value of Experiments
• Validate requirements trump everything• Incremental cost of validation is falling• Feedback means nothing without a model
Photo: ESAhttp://www.nbcnews.com/id/50677062/ns/technology_and_science-space
The race doesn’t go to the one with the best idea, but to the one with the most tested ideas
agile42 | We advise, train and coach companies building software www.agile42.com | All rights reserved. Copyright © 2007 - 2009.
thank you
[email protected]: dave.sharrock
twitter: @davesharrockslides: slideshare.net/davesharrock
“Coming together is a beginning. Keeping together is progress. Working together is success.”
Henry Ford