![Page 1: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/1.jpg)
lean research will set you free*
#leandaylondon @farrahbostic
* but first it will piss you off.
![Page 2: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/2.jpg)
happy birthday
![Page 3: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/3.jpg)
Hi, I’m @farrahbostic
!
!
I run...
![Page 4: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/4.jpg)
i come from advertising.
![Page 5: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/5.jpg)
we bet only on holes-in-one
![Page 6: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/6.jpg)
this guy.
![Page 7: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/7.jpg)
but…
![Page 8: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/8.jpg)
$30,000,000,000+
![Page 9: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/9.jpg)
information request
research RFP
study design
recruitment
fielding
Analysis
Brand/product team
research & insights
research company
recruiter/panel
moderators
research company
![Page 10: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/10.jpg)
why?
![Page 11: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/11.jpg)
because marketers want roi, but avoid defining what that is.
You’d think it might be cost per acquisition, or customer lifetime value, or revenue.
But often, KPIs are less about business results, and more about media performance.
![Page 12: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/12.jpg)
exhibit A: “passion” is a kpi.
http://www.cmo.com/content/cmo-com/home/articles/2013/9/26/KPIs_CWTK.html
![Page 13: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/13.jpg)
we’re not designing for data.
![Page 14: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/14.jpg)
“those numbers won’t count themselves.”
![Page 15: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/15.jpg)
what are we using the data for?
![Page 16: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/16.jpg)
how (a lot of) marketing people deal with data
![Page 17: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/17.jpg)
data v. prestige
Who cares about affluent baby boomers & the advertisers who love them?
I want the 18-24 demo!!
![Page 18: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/18.jpg)
there’s a fear of small numbers
Marketers worry about the signal-to-noise ratio.
The smaller the sample, the more they worry about noise (wrong people, inaccuracies or untruths, groupthink, etc.)
So they compensate with bigger sample sizes.
But more numbers often make more noise.
![Page 19: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/19.jpg)
20,000 people surveyed in three countries.
100 shop-alongs and at-home interviews.
Post-hoc analytics.
More?
leading to… analysis paralysis
![Page 20: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/20.jpg)
how marketers ‘get out of the building’
![Page 21: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/21.jpg)
“why do we even do focus groups anymore? it’s so
weird.”
![Page 22: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/22.jpg)
because, tradition.
![Page 23: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/23.jpg)
because, tradition. but also, because, bias.
![Page 24: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/24.jpg)
“statistically significant” “rigorous sample method”
“information with authority”
![Page 25: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/25.jpg)
“The data will tell us what to do.”
![Page 26: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/26.jpg)
market research isn’t as sciencey as it pretends to be.
![Page 27: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/27.jpg)
research ≠ designResearch is the process by which we understand problems.
Design is the process by which we solve problems.
![Page 28: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/28.jpg)
The CEO’s Kids
“I keep a lot of kids around me and ask them questions and peek around the corner when they think I'm not listening. My master plan was I had a lot of kids. I have six kids.
Three girls that are six, a 15-year-old, 18-year-old and 22-year-old. I have the perfect research team in my house."
http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/
![Page 29: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/29.jpg)
It's really hard to design products by focus groups. A lot of times,
people don't know what they want until you show it to them.
- Steve Jobs
![Page 30: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/30.jpg)
![Page 31: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/31.jpg)
you ≠ steve jobs.
![Page 32: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/32.jpg)
over targeting leads us astray
“[M]ost companies with a practiced discipline of listening to their best customers and identifying new products that promise greater profitability and growth are rarely able to build a case for investing in disruptive technologies until it is too late.”
- Clay Christensen, The Innovator’s Dilemma
![Page 33: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/33.jpg)
research ignores switchers, rejectors, fringe & new users
“Generally, disruptive technologies underperform established products in mainstream markets. But they have other features that a few fringe (and generally new) customers value.”
- Clay Christensen, The Innovator’s Dilemma
![Page 34: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/34.jpg)
it’s complicated.
![Page 35: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/35.jpg)
“people lie.”
![Page 36: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/36.jpg)
it’s not these people’s fault focus groups are artificial.
![Page 37: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/37.jpg)
people are great.
![Page 38: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/38.jpg)
1. we ask them to.
![Page 39: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/39.jpg)
2. but often, they show us how we’re lying to ourselves.
![Page 40: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/40.jpg)
we’ve sacrificed the truth for ‘science’.
![Page 41: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/41.jpg)
how lean can help fix research.
![Page 42: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/42.jpg)
what is a lean company for?
1. To specify customer value.
2. To identify & implement the actions that create value.
3. To remove anything that doesn’t create value.
4. To analyze the results (and repeat).
![Page 43: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/43.jpg)
why does feedback & customer pull matter?
Feedback and testing (a.k.a., research) help teams settle questions and make decisions based on evidence not preferences.
Continuous feedback helps teams see progress.
![Page 44: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/44.jpg)
people are data, too. (and vice versa)
![Page 45: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/45.jpg)
cool story, bro.
![Page 46: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/46.jpg)
we need to understand people and behavior so we can make better
predictions and measure outcomes more effectively.
![Page 47: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/47.jpg)
theory v. information“Who needs theory when you have so much information? But this is categorically the wrong attitude to take toward forecasting, especially… where the data is so noisy. Statistical inferences are much stronger when backed up by theory or at least some deeper thinking about their root causes.”
![Page 48: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/48.jpg)
we should always be checking our bias
![Page 49: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/49.jpg)
and we should use data to trigger behavior
![Page 50: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/50.jpg)
validation
![Page 51: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/51.jpg)
scripts
![Page 52: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/52.jpg)
statistical significance
![Page 53: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/53.jpg)
stop stockpiling data
![Page 54: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/54.jpg)
stop trapping insight in powerpoint decks
![Page 55: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/55.jpg)
empower the research manager
We have to transform the research manager from a project manager into a Scrum master.
Their role is to remove obstacles to progress, and to facilitate communication.
Their responsibility to the team is to facilitate hypothesis testing, and to create opportunities to challenge assumptions.
![Page 56: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/56.jpg)
Do less, more oftenLean research is about continuous
customer, partner & team feedback.
![Page 57: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/57.jpg)
Use right-sized tools that are closest to hand
![Page 58: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/58.jpg)
Go and see.This is the new intimacy.
![Page 59: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/59.jpg)
the whole team can do this.
http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/
![Page 60: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/60.jpg)
forget the horse’s mouth
Don’t count on users to make your decisions for you - that’s not their job.
Don’t let the report be the end of your learning process - it’s just a way to memorialize what you’ve learned and provide a reference when you encounter similar issues again.
![Page 61: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/61.jpg)
To make research meaningful to the whole team, the whole team needs:
Investment
Access
Accountability
nothing beats a conversation.
![Page 62: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/62.jpg)
![Page 63: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/63.jpg)
make information reusable, not disposable.
![Page 64: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/64.jpg)
data should make us more empathetic
If we design for data that helps us make decisions, that helps us create value for our customers, and that helps us develop empathy for people so deep that we can anticipate solutions to problems they can’t yet express…
That will enable the kind of analysis sufficiently advanced to be indistinguishable from magic.
![Page 65: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/65.jpg)
be nice, and listen.
![Page 66: Lean Research Will Set You Free - Lean Day London 2014](https://reader034.vdocument.in/reader034/viewer/2022050614/54244b068d7f72442b8b460d/html5/thumbnails/66.jpg)
what’s next?#leandaylondon @farrahbostic