John Cutler | Senior Product Manager | Pendo.io | @johncutlefish
Not An Afterthought: Interweaving Measurement Into User Stories
where stories go to die …where we get things donewhere we shuffle things constantly
Backlog The SprintUser Story Black Hole
I wanted to build stuff people use … that they value. But then I just checked out
Sell. Sell. Ship. Ship. Guess. Guess. New Project. Guess
It’s easy to start phoning it in …
Test Coverage
Defects over time
Budget vs. actual cost
Test pass/fail over time
Iteration burdown
Velocity
Planned vs. actual release dates
Individual hours per iteration/week
Cumulative flow chart
Work-in-Process (WIP)
Apparently, I’m being measured. Does the
Product Manager have a “passing test”?
Roadmap ReviewMy idea! My idea!
My idea!
My idea! My idea!My idea!
As a product manager, I can see the [Feature Name] feature finished in two weeks, so that I can approve itSadly, sometimes this happens in the real world
Idea
Validate Pain
Talk To Them
Sell
Sell
Hurry
BuildScale
Execute
Whoops
Boom!
Another Miss
UX Debt
SilosFeature Soup
Growth Slows
Hurry
And um, product, can you innovate? Optimize? And do it on budget?
Align
Intuition Tech Debt
Restructure
Measurement Debt Rocket ShipConsultantsLinkedIn
Learning Debt
Time to be data-driven
Too littleToo late
Learning
ExperimentIterate
TestExploreTinker
Mission
vsShipPlan
CommitAccountability
VelocityFeature
Consider the power of words.
DATALEARNING CONTROLIMPACT
INNOVATIONPREDICTABILITY
BLAME
Trust?
VALIDATIONITERATION CONFIRMATION
Y N
CONVERGE
DIVERGE
EXPLOITEXPLORE
Cast a wide net
Validate learnings
Try various solutions
Converge on solution
Conversations
StoriesDetails
For Perspective
To Explore
Arc
Drama
Actors
User Story
Pictures
Safety
Diversity
Converge
Diverge
Time
Sequence
Place
PainEnemies
Desires
not a requirement
Given [Context] When [Event] Then [Outcome]
Dan North and Chris Matts as part of Behavior-Driven Development (BDD).
Example: GIVEN that people are
perpetually in need of social
gratification, AND THAT they enjoy
seeing news from their friends,
WHEN they encounter a “feed” of
friend related news-snippets, THEN
they will spend hours online, and
may even click on ads.
As a user a [who], I want [outcome], so
that [some reason] …
Add this to dashboard
Add tracking to the click of this on that
Verify that we are logging refreshes
I need to pull this graph once a week for my meeting
Um. Hey Team. When you’re done with the story, I might have some
So I’ve got this story …
Happier Relationship
Less Stressed
Higher Quality Sleep
Exercise
Mattress
More Exercise
Better Diet
More Sleep
Less Sugar
More Water
Bit More Intensity
Work
Work Less
More Breaks
More Us Time
Renewals
NPS
Divorce?
Break T/day
Mg/meal
Kg Per Lift
Calories Hours
Time w/o Kids
Satisfaction Rating
L/day
Food Log
VO2max
Blood Pressure
Growth in 7d avg sleep
Morning Check In
NPS
wk/wk decrease
Lagging LeadingWhy What
Outcome Experiment
Plan Build Build Build Ship
Plan Build Build Ship
We will “listen” for feedback
Plan
Let’s get “ahead” of the next project
Feature A
Feature B
Feature C
Measure
A Powerful Mission
Measure
Learn Learn Learn
Build
Ship Ship Ship Ship Ship
Learn
PlanPlanPlanPlanPlanPlan
Um …Measure
Where’s Feature A?
Can I see?
Measurement and learning happens continuously. Not sporadically
Learn
Learning is siloed, or doesn’t happen at all …
comparativeunderstandable
ratio or a ratechange how you behave
You can iterate on this. You won’t get it right the first time …
DATALEARNING CONTROLIMPACT
INNOVATIONPREDICTABILITY
BLAME
Trust?
VALIDATIONITERATION CONFIRMATION
Y N