SEBASTIEN J ANINI Group Client Service Director TNS CONSUMER Sebastien has worked on the agency side for twelve years, with a career spanning various geographies, research practice areas and client sectors. At TNS UK, Sebastien is a Group Client Service Director for Consumer. His main role is to look at strategies for commercial growth, and to support the development of new structures to deliver commercial momentum. Sebastien has also direct responsibility for some of the largest UK Consumer accounts. Sebastien has been involved in promoting and supporting the development of Online research across the whole UK business, and, as such, has first hand experience of clients needs as well as Online projects execution. Having spearheaded the deployment of Online methodologies across the group for several years now he is thus is a key witness of how the growth of Online is changing the face of research.

To advance the presentation slides use the space bar. To exit use Esc

Engaging Respondentsto drive the Quality of Online Research

15 March 2007Sebastien Janini

3

Survey experience and data quality

Towards a new survey interface

Building a relationship with the forgotten stakeholders

Delivering the value exchange promise

Engaging Respondentsto drive the Quality of Online Research

What’s new?

USA: 68

67

433

56

75

50

51

63

51

3530

5424

26

4764

42

24

36

38

46

26

14

2

15

20

76

5219

40

20

15

82

33

USA: 68

67

433

56

75

50

51

63

51

3530

5424

26

4764

42

24

36

38

46

26

14

2

15

20

76

5219

40

20

15

82

33

Internet penetration is ever growing

but is it the only gauge of quality?

Source: Internetworldstats.com & TNS – 2005 Users in %

6

A nomenclature of error types

Total Error

Random Sampling Error Non Sampling Error

Non Response Error Response Error

Researcher Errors Interviewer Errors Respondent Errors

• SurrogateInformation Error

• Measurement Error• Population Definition Error

• Sampling Frame Error• Data Analysis Error

• Questioning Error• Recording Error• Cheating Error

• Inability Error• Unwillingness Error

• Cheating Error (embellishment, numbing down)

Source: Naresh K. Malhotra, Marketing Research

Can the survey execution impact respondent experience and data quality?

Motivating respondents, Motivating respondents, generating curiosity and interestgenerating curiosity and interest

Dynamic interface to establish Dynamic interface to establish connectivity with respondentsconnectivity with respondents

The ‘practitioner’ point of view

experienceand data quality

survey

Online survey design optimisationDoes this look right?

Appearance and formatting of Interactive surveys …

Drop Down Box vs

Radio Buttons

… will affect response rate and response values

Respondents tend to opt in favour of going ‘higher’ in the drop down box response format than they did when using radio buttons

In some regions Language itself can be In some regions Language itself can be an issuean issue

“ 火 "

““Popular”Popular”

oror

““Hot”Hot”

ChinaChina

““Angry”Angry”

TaiwanTaiwan

Version A

BrandA

BrandB

BrandC

BrandD

BrandE

Version B

BrandA

BrandB

BrandC

BrandD

BrandE

BrandA

BrandB

BrandC

BrandD

BrandE

BrandA

BrandB

BrandC

BrandD

BrandE

Full grid versus split grid

Brand A

service

Version A: Full grid

16%22% 25%

11% 11%

Creative A dynamic brand Offer fashionable andinnovative products

Offer promotion or cashdiscount frequently

It’s my ideal brand

16% 15% 12% 13%21%

Reliable and trustworthy Produce high-quality cars Offer value-for-moneyproducts

High-tech. equipment Products are widely favored

13% 14% 14% 16% 16%

Engineering better Offer high-quality after-sales Has vision TV ad is reliable Customer oriented

Version B: Split gridSignificantly different

20%28% 27%

24% 24%

37%29%

17%24%

22%

19% 17% 15% 22%33%

Full grid versus split grid

Large vs Small text boxes

Large text boxes

Print Ad

Small text boxes

Print Ad

Q. What if anything do you like / dislike about this product?

Means: 10.46 8.51 10.85 11.13

Significantly different

Likes Dislikes

16% 16%27% 32%

56%68%

51%49%

8% 6% 4% 4%11%14%11%

21%

Large box Small box Large box Small box

21+ characters

11-20 characters

1-10 characters

Don't know / No answer

Large vs Small text boxes

Representing brands as text, logos or pack shots

18

Prompted Awareness by prompt type

0 20 40 60 80 100

Colgate

Aquafresh

Sensodyne

Macleans

Crest

Signal

Mentadent P

Oral B

Thera-med

Euthymol

Aim

Verbal

Logo

Pack

Little difference seen for major

brands

Lack of clarity of

pack shots reduces

awarenessDisastrously so!

But logo and pack can increase

awareness for smaller brands

19

The effect of different prompt types cannot be ignored.

Colgate Sensodyne Signal Euthymol Aim

Recommended by dentistsHigh quality

For sensitive teeth

Fresh tasting

Suitable for every day

Has a minty taste

Keeps teeth white

Keeps breath fresh

Product attributes

No significant difference between text logo and pack shot

At least one significant difference between text logo and pack shot

20

The effect of different prompt types cannot be ignored.

Colgate Sensodyne Signal Euthymol Aim

Modern and up to date

Good value for money

An upmarket brand

Easily recognisable

Attractive packaging

Caring

Eco-friendly

Used by younger people

Used by older people

Image attributes

No significant difference between text logo and pack shot

At least one significant difference between text logo and pack shot

21

Sensodyne

00.10.20.30.40.5

0.60.70.80.9

1

Rec

omm

ende

d by

dentis

ts

Hig

h qual

ity

For s

ensi

tive

teet

h

Fre

sh T

astin

g

Suita

ble

for e

very

day

Has

a m

inty

tast

e

Kee

ps te

eth w

hite

Kee

ps bre

ath fr

esh

Verbal Logo Pack

Product qualities

The pack detracts from product attributes

The pack detracts from product attributes

22

Aim

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Rec

omm

ended

by d

entis

ts

Hig

h qual

ity

For s

ensitiv

e te

eth

Fre

sh T

astin

g

Suita

ble fo

r eve

ryday

Has

a m

inty

tast

e

Kee

ps te

eth w

hite

Kee

ps b

reat

h fres

h

Verbal Logo Pack

Product qualities

The pack provides product cues.

The pack provides product cues.

Prompted Awareness is 3%!

23

You will get differences in awareness and image measurements with different forms of prompt

Packs and logos are designed to convey positionings and images and respondents do pick up on these in the interview

The effect can be to reduce association with claims that are not propounded in the logo or pack, particularly with bigger brands. Or it can be to reinforce specific product/image positions that are suggested by the logo or pack, particularly for lesser known brands

Respondents will answer based on the logo and, particularly, the pack alone, despite having no knowledge of the brand.

There is no right or wrong form of prompt, but different forms of prompt measure different things

A text prompt relies on what is in the consumer’s mind alone – the equivalent of writing the shopping list.

A logo prompt provides some further product or image cues

The pack provides (usually) even more specific cues – equivalent to standing in front of the facing in the supermarket.

If you change the form of prompt in a tracking study, you will lose continuity.

Representing brands as text, logos or pack shots

Benefits of best practices

The Internet and the Online data collection medium have made it possible for a wider range of businesses to have instant access to a huge amount of information at low cost

Finding information is NOT what Market Research is about

It is all about knowing who to survey and what the sample represents, how to ask the questions and what the biases in the method are, what the data collected mean and what it does not mean, how to analyse that data and how far it can be extrapolated

Discipline, structure and method in the research design are critical

Otherwise the researcher only ends up ‘finding stuff’

Training and Research on Research are key to develop a culture of best practices

Providing structure to surveys doesn’t necessarily mean designing ‘boring’ surveys.

We still need to push for a better look and feel, and new interface solutions

a new survey interface

Towards

Drag & Drop

Page Turner

29

Enhancing the survey process …

Rich stimulus

Rich data

Deeper consumer insights

Busy retail

environment …

30

… by using the visual language of the respondents

Stimulate

respondents

Engage

consumers

31

The convergence of digital media

32

33

A new interface

Bring in entertainment …

Movement away from conventional research to greater involvement / dialogue

A hybrid quantitative and qualitative approach

Scale brings rational intelligence

Open chat brings emotional intelligence

Mass marketing near an end, same dynamics for research

Clients looking for greater 'connection' or ‘engagement’ with their target

market/ audience

Significant trend and opportunity for insight communities / co-creation

'Death of distance‘ – or moribund

New software tools provide ability to Engage, Stimulate and Capture

consumer preferences Faster and more Accurately

with the forgotten stakeholders

Building a relationship

Access panel managementSuperior Panel Management

Response

Rates

Contact Controls

Usage Controls

Positive Panellist

Experience

37

Each survey is the face of the panel/research company

What does the survey invite say about you?

How do you treat respondents when they providewrong / incomplete response? Are we stillthinking ‘permission’ rather than ‘interruption’?

How motivating is the incentive scheme? What does it say about how you value your respondent?

Are you honest with survey length? Is there a progress bar?

Do you ensure respondent-friendly download times?

Are you working toward a more enjoyable & less intrusive interviewing experience? A respondent-friendly survey for better respondent engagement?

Building a relationship

38

Panel management rules tell a lot about how you value your panellists

What are reasonable recontact rules?

What do you do to maximise cooperation rates, to promote loyalty and tenure?

Is the look and feel of your surveys consistent? Are you actually building a brand panellists can recognise and engage with?

Are you maximising efficiencies (and minimising respondents frustration) via targeted sampling?

Are there any control for over-usage of hard to reach target groups?

Is there a policy of minimum fieldwork period length?

Building a relationship

39

Panel/research company need to build a relationship with panellists outside project work

An ‘Active’ management of panellists, outside project work, is critical to maximise engagement. It is about building a relationship (continual dialogue, feedback process, celebration of birthday dates)

What do you do to build and maintain trust with respondents?

Is there anything we can learn from how stores manage their loyalty card holders?

How about running employee satisfaction studies with the panellists? Tracking the results and setting targets for ‘commitment levels’

Building a relationship

the value exchange

promise

delivering

41

What do respondents want?

Passing some ‘value’ back to the

respondent

Concerns over…

- Growing individualism - Rejection of ‘your opinion counts’…what’s in it for me? - Privacy protection - Time - Over-contacting

Respondent co-operation rates are low and falling in many

markets…

42

Respondent engagement is a two way stream

Enhancing dialogue with respondents (less intrusive, more enjoyable)

Closer connection with respondents outside projects

Passing value back (incentives, feedback)

Asking respondents input into the panel management rules

Building a Brand

Developing ‘employee’ satisfaction management programmes

Quality controls embedded in the survey

Checking interview duration

Sense checking responses

Systematically picking up on patterned responding

Inserting checks/trap questions

Purging of non-responsive panelists

Online responses are faster, deeper, and more considered

Proven reliability of panel results Test retest Validation studies

People do respond differently online more considered responses More candid and honest Sensitive subjects More critical Richer open ended responses

Online

Better Research

Outcomes

Page 44

Thank you!