A  Presenta*on  from  The  Fes*val  of  NewMR  –  Training  Day  

3  December  2012  

All  copyright  owned  by  The  Future  Place  and  the  presenters  of  the  material  For  more  informa:on  about  NewMR  events  visit  NewMR.org  

Sponsored  by:  

See    the  eXhib:on  for  booths  from  media  partners  &  supporters  

Best  Prac*ce  In  Mobile  Research   Lachlan  Stokoe,  Vision  Cri:cal      

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

BEST PRACTICE IN MOBILE RESEARCH  FESTIVAL OF NEWMR TRAINING DAY 3RD DECEMBER 2012 LACHLAN STOKOE VISION CRITICAL, AUSTRALIA

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

A BIT OF BACKGROUND...

Vision Critical has been Investigating mobile research since 2008, both in terms of technical design and research on research Launched mobile-web optimised surveys during 2011 Conducted hundreds of multi-modal and mobile only studies globally My background is in quantitative research

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

MANY  PEOPLE  LIVE  A  ‘MOBILE-­‐CENTRIC’  LIFE...  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

Many operating systems

Unique browser applications

Different input modalities

Screen size/resolution

Lack of Adobe Flash/Javascript

   

Over

6,500 web-enabled

devices

MOBILE  WEB-­‐BASED   APP-­‐BASED  SMS/WAP-­‐BASED  

BUT  IT’S  STILL  A  JUNGLE  OUT  THERE...  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

MOBILE  RESEARCH  IS  AN  EVOLVING  FIELD:    Some  examples  of  current  and  poten*al  future  applica*ons...  

The  ability  to  seamlessly  connect  mul*ple  elements  of  these  together  in  an  ongoing  dialogue  with  your  respondents  (while  keeping  each  ac*vity  short  and  engaging)  

QUANTITATIVE  In-­‐situ  research  &  geoloca:on  (capturing  consumers/  shoppers  in  the  moment)    Event  based  research  (at  the  event  or  as  soon  as  possible  aNerwards)    Post-­‐experience  sa:sfac:on  Diary  studies  Website  evalua:on  via  pop-­‐ups    

QUALITATIVE  Mobile  ethnography    Photo/video  upload  and  share    Event  experience    Integra:on  with  social  media  (Facebook  app)    

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

There are additional considerations when conducting mobile research You should consider these in the research design process, NOT get to the end of that process and tack on mobile as just an additional deployment option Think of it is an opportunity to challenge the status quo and approach things differently

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

IDENTIFY AND OPTIMISE

BE AWARE OF QUESTION TYPES & BIASES

KEEP THE LENGTH UNDER 10 MINUTES (LESS IF NOT INCENTIVISED)

MAXIMISE AVAILABLE SCREEN SPACE

CONTROL FOR SCROLLING

TARGET MULTIMEDIA FOR IMPACT

QA FOR ALL DEVICES

DEPLOY INTELLIGENTLY

BEST  PRACTICES  FOR  MOBILE…  

     

             

                         

                             

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

IDENTIFY  AND  OPTIMISE…  

Understand what questions work and how they will display in different operating systems and design questionnaire accordingly

Question types that aren’t supported may be skipped by the software

…FOR THE DEVICES SUPPORTED BY YOUR SOFTWARE

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

UNDERSTAND  YOUR  MARKET…  

A bias may exist in terms of smartphone owners generally and/or particular types/brands

…AND BE AWARE OF POTENTIAL BIASES

In Australia, we found that there is an inherent bias in Smartphone ownership, but no difference between different brands

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

KEEP IT SHORT… AND SIMPLE

KISS…  

The same survey will take 50% longer on a Smartphone compared to a desktop

Respondents are willing to give some leeway (within reason) – up to 10 minutes is a fair estimate for an incentivised survey (which is a 6-7 minute standard online survey)

Consider breaking longer surveys up into chunks and running as a series of shorter surveys – this obviously introduces other challenges

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

REAL  ESTATE  IS  AT  A  PREMIUM…  

Remove superfluous words from question and response code text to keep length short

Only use images in response buttons when there are a small number of options or it is absolutely necessary for question comprehension

Keep branding elements simple - avoid using banners and logos

…SO LOOK TO MAXIMISE YOUR AVAILABLE SCREEN SPACE  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

EXCESSIVE  SCROLLING  HURTS…  

Where it makes logical sense, you may be able group 3-4 short related questions on the one screen e.g. age, gender, marital status

Keep answer categories short (try to keep to 6 or less, particularly when multiple response or scale ratings)

If you have more than 5 items in a grid you should consider splitting over multiple screens. NB: Different software will display grids in different ways so make sure you understand the layout/number of screen implications for your survey

Minimise scrolling as much as possible. Where it is unavoidable you should try to keep it to ‘one-way’.

…SO CONTROL FOR LIMITED AND LOGICAL SCROLLING    

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

USE  MULTIMEDIA  SPARINGLY…  

Keep in mind that data usage limitations and screen load times can be an issue when using multimedia – introduces drop out risk and respondent backlash

Excessive use of multimedia can dilute its impact

…FOR MINIMUM PAIN AND MAXIMUM IMPACT  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

…QA ON AS MANY DIFFERENT DEVICES AS POSSIBLE

ALLOW  TIME  TO  ‘CHECK  IT  AGAIN  SAM’…  

If doing a multi-modal study make sure you check both completion methods thoroughly.

Just to reiterate – there are many operating systems, browsers, screen sizes, and other quirks to mobile devices. Know what ones your software covers and test on as many different platforms as you are able to.

If possible, also test on an un-supported platform to see what this experience is like. Can a respondent see a non-optimised version of the survey or are they precluded from taking part?

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

GIVE  PEOPLE  A  CHOICE  TO  RESPOND  WHEN,  WHERE  AND  HOW  THEY  PREFER…  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

…TO MAXIMISE CHANCE OF RAPID RESPONSE

Be aware of the sample implications of making a study mobile only – even when advised it is mobile only many will try to complete on a desktop and drop out when that fails

Be clear in your communication to respondent about the choice or requirement of completing via mobile

The response window for mobile is tighter than for PC – Need to deploy at times of the day people are most likely to be able to complete straight away on their mobile – morning/evening commutes/lunchtime/after 8pm (likely to vary by target group)

Special additional challenges/considerations apply to weekend deployments

DEPLOY  INTELLIGENTLY…  

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

IDENTIFY AND OPTIMISE

BE AWARE OF QUESTION TYPES & BIASES

KEEP THE LENGTH UNDER 10 MINUTES( LESS IF NOT INCENTIVISED)

MAXIMISE SCREEN SPACE

CONTROL FOR SCROLLING

TARGET MULTIMEDIA FOR IMPACT

QA FOR ALL DEVICES

DEPLOY INTELLIGENTLY

RECAP:  BEST  PRACTICES  FOR  MOBILE…  

     

             

                         

                             

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1

IDENTIFY AND OPTIMISE

BE AWARE OF QUESTION TYPES & BIASES

KEEP THE LENGTH UNDER 10 MINUTES( LESS IF NOT INCENTIVISED)

MAXIMISE SCREEN SPACE

CONTROL FOR SCROLLING

TARGET MULTIMEDIA FOR IMPACT

QA FOR BOTH MOBILE AND DESKTOP

DEPLOY INTELLIGENTLY

AND  REMEMBER…  

     

             

                         

                             

DESIGN FOR MOBILE DON’T JUST TACK IT

ON

Lachlan Stokoe, Vision Critical, Australia Festival of NewMR 2012 – Training Day – Session 1