designing engaging questionnaires to deliver high response rates

52
Summer Camp Designing engaging questionnaires to deliver high response rates Presented by Steve Bax Managing Director Bax Interaction Limited

Upload: audra

Post on 10-Jan-2016

33 views

Category:

Documents


2 download

DESCRIPTION

Designing engaging questionnaires to deliver high response rates. Presented by Steve Bax Managing Director Bax Interaction Limited. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Designing engaging questionnaires to deliver high response rates

Summer Camp

Designing engaging questionnaires to deliver high response rates

Presented by

Steve Bax

Managing Director

Bax Interaction Limited

Page 2: Designing engaging questionnaires to deliver high response rates

Summer Camp

Overview

This webinar will explore the key principles behind effective questionnaire design. It will include current thinking on how best to engage potential respondents and thereby improve completion rates.

Page 3: Designing engaging questionnaires to deliver high response rates

Summer Camp

Content

1. Introduction

2. Defining the research process

3. Questionnaire design and piloting

4. Engaging the respondents

5. Research design checklist

6. Questions

Page 4: Designing engaging questionnaires to deliver high response rates

Summer Camp

Research phases

Analysis, reporting and

reviewphaseR

eso

urc

es r

equ

ired

DefiningPhase

Piloting and DesignPhase

Engagement including

pre-qualification

Reminders

Start Time Finish

Page 5: Designing engaging questionnaires to deliver high response rates

Summer Camp

Why use questionnaires?

Primarily to acquire quantitative information from respondents

Also provide qualitative insights

Reasons include:-

Identifying, evaluating and sizing existing and new markets

Testing reactions to new products and services

Measuring performance

Satisfaction surveys

And more

Page 6: Designing engaging questionnaires to deliver high response rates

Summer Camp

Defining

Page 7: Designing engaging questionnaires to deliver high response rates

Summer Camp

Defining

Purpose

Research objectives The ‘shopping list’

Data to be collected Research vs.. volunteered

Target audience(s)

Delivery mechanisms

Previous experience

Page 8: Designing engaging questionnaires to deliver high response rates

Summer Camp

Research objectivesThe objectives are key and must generate the topics for investigation.

The wording can imply a certain method

E.g. to measure = quantitative

E.g. to explore = qualitative

Consider other terms: goal, aim, purpose and hypothesis

Page 9: Designing engaging questionnaires to deliver high response rates

Summer Camp

Stop and think

What is my ‘shopping list’?

What data do we need at the end of the process?

Is it pure marketing research or volunteered data?

Who do we need to gather data from?

What are the implications on method and design?

What have we learnt from previous or others’ research successes or failures?

Page 10: Designing engaging questionnaires to deliver high response rates

Summer Camp

Selection of delivery mechanisms

Page 11: Designing engaging questionnaires to deliver high response rates

Summer Camp

Online surveys

Advantages

Low cost

Fast

International coverage

Creative opportunities

Effective for qualitative and quantitative data collection

Disadvantages

Declining response rates

Incomplete cover

Biased responses possible

Page 12: Designing engaging questionnaires to deliver high response rates

Summer Camp

Written and postal surveys

Advantages

Geographical reach

Control over the process

Quantity of data that can be gathered

Respondent perceptions

Disadvantages

Cost due to postage rates and printing vs. online

Slow to administer

Response rates

Self-selecting samples - bias

Page 13: Designing engaging questionnaires to deliver high response rates

Summer Camp

Telephone based questionnaires

Advantages

Low cost vs. face to face

Fast

International coverage

Control over process

Understanding checking possible

Disadvantages

Refusal rates

Sugging and frugging perceptions

Limited duration

Page 14: Designing engaging questionnaires to deliver high response rates

Summer Camp

Multi mode approachesSnap Surveys and Silver Dialogue findings with National Trust visitors

Page 15: Designing engaging questionnaires to deliver high response rates

Summer Camp

MMR ‘DiaLogs’ APP

Page 16: Designing engaging questionnaires to deliver high response rates

Summer Camp

Design and piloting

Page 17: Designing engaging questionnaires to deliver high response rates

Summer Camp

Design

Fundamentals

Relevance

Usability

Layout and appearance

Question and response construction

Methodology

Qualitative vs.. quantitative building blocks

Multi-mode approaches

Sampling

Open and closed question mix

Piloting

Page 18: Designing engaging questionnaires to deliver high response rates

Summer Camp

Key design principles

1. .. the rigour, integrity and quality of data. Neither interviewers nor respondents should be confused or bored by any questionnaire.

2. The guiding rule can be expressed as ‘doing justice to how people would normally talk about things’.

Source: MRS – Questionnaire design guidelines

Page 19: Designing engaging questionnaires to deliver high response rates

Summer Camp

‘Funnel’ sequence

As a rule – progress from the general to the specific

1. Gain understanding of broad context of opinions held

2. THEN the nature and strength of opinion in a given area

3. Step by step to the detail underpinning these

ALWAYS PILOT TEST

Page 20: Designing engaging questionnaires to deliver high response rates

Summer Camp

Questionnaire construction

Target audience

Develop question topics

Select question and response formats

Select wording

Determine sequence

Design layout and appearance

Clear instructions

MRS/DPA considerations

Page 21: Designing engaging questionnaires to deliver high response rates

Summer Camp

Layout and appearance

Introduction

Spacing

Numbered headings and sections

Numbered questions

Length

Quality of production – on and offline

Use of sequencing, routing, piping and a variety of response mechanisms

Clear instructions

Use of colours to indicate changes in areas being questioned

Font and text size

Page 22: Designing engaging questionnaires to deliver high response rates

Summer Camp

Types of question

Behavioural - market size, market share, usage rate, awareness

Attitudinal - image and attitude surveys, brand mapping, help build market share

Classification - all surveys

Page 23: Designing engaging questionnaires to deliver high response rates

Summer Camp

Behavioural questions

Have you ever ……?

Who do you know …………?

How many ………….?

When did you last ………?

Do you have ………?

Who does …………?

Page 24: Designing engaging questionnaires to deliver high response rates

Summer Camp

Attitudinal questions

What do you think of ……?

Why do you …………?

Do you agree or disagree ………?

How do you rate ………?

Which is best (or worst) for ………?

Page 25: Designing engaging questionnaires to deliver high response rates

Summer Camp

Question / response formats

Closed questions

Open-ended questions

Scales

Variety is key

Page 26: Designing engaging questionnaires to deliver high response rates

Summer Camp

Closed questions

Dichotomous

Two possible answers

Multiple choice

Three or more answers! Collectively exhaustive Mutually exclusive

Pros and cons

Page 27: Designing engaging questionnaires to deliver high response rates

Summer Camp

Open-ended questions

Advantages

Full answer

Exploratory

Disadvantages

Misinterpretation

Speed

Coding / analysis

Completely unstructured - interviewer or respondent writes answer

Three key open-ended questions are useful:

1. What is good? 2. What is bad? 3. What improvements could be made?

Page 28: Designing engaging questionnaires to deliver high response rates

Summer Camp

Classification questions Examples

Age

Gender

Household status

Marital status

Social class

Education level

Industrial occupation

Number of employees

Location

Neighbourhood

Page 29: Designing engaging questionnaires to deliver high response rates

Summer Camp

Use of scales

Forced vs.. non forced

Forced Doesn’t allow ‘middle ground’

Non forced Allows neutral views e.g. ‘neither expensive or inexpensive’

Balanced vs.. unbalanced

Most balanced e.g. equal number + as –

Unbalanced used where ‘end piling’ is expected e.g. Car safety features

Page 30: Designing engaging questionnaires to deliver high response rates

Summer Camp

Question wording

Ambiguous questions

Double barrelled

Leading / loaded questions

Checking questions

Choice of statements to measure

Implicit assumptions

Language

Tone

Page 31: Designing engaging questionnaires to deliver high response rates

Summer Camp

Design variations

Qual stage

Final report

Quant stage

A B C

D E

F

Desk research

Page 32: Designing engaging questionnaires to deliver high response rates

Summer Camp

Ensuring relevance to the audience

Screening

Sampling

Research currency

Timeliness

Clear explanation of reason target chosen and mutual benefit

Page 33: Designing engaging questionnaires to deliver high response rates

Summer Camp

Online design options

Progress indicators

Forward and back buttons

Self checking

Filtering

Routing

Piping

Radio buttons

Sliders

Page 34: Designing engaging questionnaires to deliver high response rates

Summer Camp

Online design options

Video / visuals / documents as stimuli and intuitive response mechanisms

Audio comments

‘Click and drag’

Select and deselect in a visual context

Voting on visual contexts

Hotspot mapping

Timed responses

Page 35: Designing engaging questionnaires to deliver high response rates

Summer Camp

The power of pilot testing

Defining parameters

Improving design aspects

Adding key questioning areas missed

Assessment of mix of closed vs.. open questioning

Ensuring understanding and logic

Checking completion timing

Checking perceived response likelihoods

Page 36: Designing engaging questionnaires to deliver high response rates

Summer Camp

Engagement

Page 37: Designing engaging questionnaires to deliver high response rates

Summer Camp

Engagement

Pre-qualification

Landing timing

Reminders’ timing and impact

Incentivisation

Current environment

Page 38: Designing engaging questionnaires to deliver high response rates

Summer Camp

Pre-qualification – “What’s in it for me?”

Why should the target respond?

Transparency and consent

Justification Relationship building Mutual benefit Sampling should be used to reduce data fatigue

Relevance critical

Incentive?

Pre-contact

Multi-mode approaches

Attention grabbing headlines and titles needed

Page 39: Designing engaging questionnaires to deliver high response rates

Summer Camp

Response rate trends

39

Page 40: Designing engaging questionnaires to deliver high response rates

Summer Camp

Timing

Initial survey invitation – online

Monday pm optimal*

Reminders

Avoid over-use of reminders

2 reminders appear to generate optimal additional returns currently** Typically 50% of initial response following reminder 1 and… 50% of response to reminder 1 following reminder 2

Sources: *Lightspeed Research survey timing tests / **Bax Interaction further education surveys results 2012/2013

Page 41: Designing engaging questionnaires to deliver high response rates

Summer Camp

Incentivisation

Consider target audience(s)

Perceived value

MRS rules

Include in pilot testing

Previous experience

Comparable surveys

Page 42: Designing engaging questionnaires to deliver high response rates

Summer Camp

Current environment

Time poor audiences

Data fatigue – many are over researched

Instant gratification culture

Low attention spans

Desire for intuitive design

Desire for simplicity

Base of ‘tech savvy’ respondents growing

Digital environment changing attitudes to research

Page 43: Designing engaging questionnaires to deliver high response rates

Summer Camp

Getting it wrong? Ofsted example

(Please tick) strongly agree

agree disagree strongly disagree

unable to comment

1. My child enjoys school.

2. My child is making good progress because the teaching is good at school.

3. I feel that my child is safe and well cared for at school.

4. Behaviour in school is good.

5. The school is well led and managed.

6. The school takes account of children’s views.

7. The school seeks the views of parents/ cares and takes account of their suggestions and concerns.

8. Additional comments:

Page 44: Designing engaging questionnaires to deliver high response rates

Summer Camp

Design checklist- getting it right

Engage your targets to gain optimal response rates

Page 45: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 1

1. Methodology

Relevance to the target audience?

Delivery mechanisms – right for the target audience?

Expected response rates to be measured?

Has previous experience been applied?

Page 46: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 2

2. Questions

Number of and logical order?

Relevance and appropriateness to the audience and the required outcomes?

Tone / language?

Ease of understanding?

Clarity of instructions?

Classification questions at the end?

Page 47: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 3

3. Responses

Variety and use of scales and other mechanisms for scoring responses?

4. Testing

Has a pilot test to identify possible improvements including length, layout, question order and whether content is meeting the objectives been done?

Page 48: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 4

5. Introduction

Timing for completion?

Explanation of purpose?

Gaining consent?

Explanation of how respondents’ data will be used?

Incentives being offered?

Meeting regulatory requirements?

Page 49: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 5

6. Layout

Attractiveness to the respondents?

Colour, fonts, branding, visual appeal?

Clear routing instructions?

Effective use of filtering and piping?

Use of numbered questions and sections?

Page 50: Designing engaging questionnaires to deliver high response rates

Summer Camp

Checklist - 6

7. Structure

Is there the right mix of open and closed questions?

8. Post survey planning

How will the results be analysed, findings presented and followed up?

Page 51: Designing engaging questionnaires to deliver high response rates

Summer Camp

Q and A?

Page 52: Designing engaging questionnaires to deliver high response rates

Summer Camp

Contact detailsSteve Bax DMS MCIM AMRS FCMC Chartered Marketer

Bax Interaction Limited

Research - Strategy - Training Consultancy

Cambridge

www.baxinteraction.co.uk

T . 01223 864011

M. 07778 407676