michael sosnowski advances in quant - 2011
TRANSCRIPT
A Presenta*on from The NewMR “Advances in Quan*ta*ve Research” Event
19 September, 2012
Event sponsored by Affinnova All copyright owned by The Future Place and the presenters of the material
For more informa=on about Affinnova visit www.affinnova.com For more informa=on about NewMR events visit newmr.org
Doing More with Less: GeFng Greater Value from Mobile Quant Michael Sosnowski, TRC Market Research
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Doing more with less: GeFng greater value from mobile quant
Michael Sosnowski Execu=ve Vice President TRC Market Research
hNp://newmr.org/videos-‐slides/advances-‐in-‐quant-‐sept-‐2012/steve-‐nutall-‐and-‐adrian-‐morgan/
hNp://newmr.org/videos-‐slides/advances-‐in-‐quant-‐sept-‐2012/steve-‐nutall-‐and-‐adrian-‐morgan/
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Primary researchers in the pressure cooker.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Is mobile a way to do more with less?
• The most interes=ng and value-‐genera=ng work we do requires some degree of complexity.
q Conjoint
q Priori=za=on techniques
q “What-‐if” simula=ons
• A powerful way to get at otherwise hard-‐to-‐reach popula=ons.
• An invita=on to gather data at =mes and places not previously possible.
• Less informa=on; higher frequency.
• Increasingly consistent with how consumers want to give feedback.
… just not too much less. Yes...
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Is mobile a way to do more with less?
Embrace the mobile impera=ve but…
…don’t marginalize yourself with low involvement or low value contribu=ons.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Short and sweet won’t be enough.
Much of our current focus:
• 5 to 10 ques=ons.
• Simple response dynamics.
• Assume short aNen=on span.
• Maximize experience at expense of informa=on value.
• All good things of course, but we need to draw upon our other strengths – design and analysis.
Consumer-‐ and mobile-‐friendly surveys
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Short and sweet won’t be enough.
More with less in a mobile environment
Consumer-‐ and mobile-‐friendly surveys
Don’t maximize. Op=mize!
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
“You can’t do advanced choice-‐based research on a handheld device.”
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
Is that true? Let’s ask ourselves some fundamental ques=ons:
Why do we do advanced choice-‐based studies? • To understand preferences and build beNer products.
• Because we believe trade-‐offs are superior to ra=ngs.
Why don’t we do such studies on mobile devices? • They require too many choices.
• It’s difficult to render mul=-‐op=on choices on the small screen.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
An es*ma*on challenge: What’s the right number of choices? Case study: Understanding preferences for HDTVs
• 7 unique aNributes – brand, price, specifica=ons. • 331 par=cipants. • 3 HDTV profiles (plus “none”) per choice task • 13 tasks (cards).
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
53
61 65
68
7.1 4.7 3.3 2.9 0
10
20
30
40
50
60
70
80
3 6 9 13
Hit Rate
Error
Perhaps we don’t need so many choice tasks?
Maximum hit rate
Minimum error
Case study: Understanding preferences for HDTVs
# of choice tasks
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
A design challenge: Can choices be simplified? Case study:
Stemming aKriLon among mobile subscribers
• 9 unique aNributes. • 1,208 par=cipants. • Take it or leave it -‐ 1 mobile (plus “none”) per choice.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
89%
3.7 0
50
100
Hit rate Error
Stark “yes or no” choices can yield robust models of choice.
Case study: Stemming a:ri;on among mobile subscribers
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & challenge conven*onal wisdom.
“Fewer tasks? Simplified choices?
That might just work!”
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Op*mize & do more powerful quan*ta*ve studies via the mobile, today.
Tournament-‐style prioriLzaLon… good for online; GREAT for the mobile plaWorm • What makes it good for lengthy feature sets makes it a great poten=al fit for mobile MR.
q Fewer ques=ons q More relevant ques=ons q Handheld-‐friendly formaFng requirements q Simple response mechanics
Consumer-‐ and mobile-‐ friendly surveys
More with less in a mobile environment
A potent applicaLon that combines innovaLve design and analysis with a simple and engaging task
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile “more with less” in ac*on.
Want to catch a movie in a theater?
How do you decide what to see?
An everyday priori=za=on problem.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile “more with less” in ac*on.
• So many factors to consider.
• We stopped at 18.
• Max-‐Diff = 14 choice sets.
• Tournament-‐style approach: 9.
• “That’s more mobile-‐appropriate,” we thought. “Let’s test it out.”
A tournament style approach
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Mobile “more with less” in ac*on.
• 414 U.S. adults. • 20-‐ques=on survey (6-‐7 minutes)
• All accessed and completed the survey using a mobile device.
• Main tasks – review and priori=ze across our 18 “go to the movies” choice factors.
• Simple, appropriately-‐formaNed tasks presented on-‐device.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Results confirm our ability to tackle difficult quant work via mobile.
a. You like the plot or story b. It is your favorite movie genre c. It is adapted from something you are familiar with d. It received good reviews from movie cri=cs e. Your friends and family liked it f. The movie has your favorite stars g. The theater is conveniently located h. It is playing at a convenient =me at the theater i. It's a sequel to a movie you've already seen j. It is shown in 3D k. The theater has good quality and variety of food l. You have a loyalty card or get points or rewards m. It's targeted towards people like you n. You think it will be a popular movie o. It has your favorite director p. It is a long movie q. It has an R ra=ng r. It is animated
A rank-‐ordering of importance
Most important decision factor
Least important decision factor
Case study: What’s important to you when choosing a movie?
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Results confirm our ability to tackle difficult quant work via mobile. Case study: What’s important to you when choosing a movie?
0
5
10
15
20
25
30
35
40
45
50
a b c d e f g h i j k l m n o p q r
Mobile Tradi=onal
Bayesian esLmaLon of uLliLes helps idenLfy Lers of
importance.
Results match nicely to those obtained using tradiLonal online data collecLon.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
“More with less” mobile research: Beyond short & sweet. More than fast & cheap.
• Don’t marginalize the role of custom, quan*ta*ve research in an increasingly mobile environment.
• Remember we have more tools in our belt than programming and crea*ve forma\ng.
• This is relevant right now.
• “More with less” mobile can help improve all methodologies.
• Be an op*mizer, not a maximizer.
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Thank you. Michael Sosnowski
Execu=ve Vice President TRC Market Research
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Q & A
Sue York NewMR
Michael Sosnowski TRC Market Research
Speaker: Michael Sosnowski, TRC Market Research NewMR Advances in Quantitative Research Event, 19 September 2012, Session 1
Michael Sosnowski TRC Market Research
Visit TRC’s site at hNp://trchome.com Read Michael and his colleagues’ blog posts @ hNp://trchome.com/trc-‐blog.html Follow Michael @ hNp://twiNer.com/MRSosnowski (@MRSosnowski) Connect with Michael on LinkedIn @ hNp://linkedin.com/in/michaelrsosnowski Contact Michael at 215.641.2258 or at [email protected]
A Presenta*on from The NewMR “Advances in Quan*ta*ve Research” Event
19 September, 2012
Event sponsored by Affinnova All copyright owned by The Future Place and the presenters of the material
For more informa=on about Affinnova visit www.affinnova.com For more informa=on about NewMR events visit newmr.org
Doing More with Less: GeFng Greater Value from Mobile Quant Michael Sosnowski, TRC Market Research