bill blyth global methods director, tns changing respondent pathways in the eurobarometer landscape...

Bill Blyth

Global Methods Director, TNS

Changing Respondent Pathways in the Eurobarometer LandscapeBetter EB for better understanding the citizens’ needs

2

• Too often, international studies are regarded as the sum of national studies conducted in different countries.

• The key principles of our philosophy:

Method and mode evaluative criteria

ConsistencyReplicability

Reliability TimelinessCost

effectiveness

Validity

The Quality Triangle

Quant. alone has its limits

Fitness

Process User

Design

Eurobarometer Surveys Mode Assessment

• Need explicit feasibility assumptions

• Consistent coverage definition

• Regular coverage benchmarking and updating

• Country by country evaluation

• Transparent ‘Fitness for Purpose’criteria

• Agreed decision-making process

CAPI availability- drivers and constraints

• Respondent independent• Internet dependent• Very high capital investment• Sophisticated IT infrastructure

need• Multi-country inter-operable

capabilities

The EB gold standard

CATI feasibility- drivers and constraints

• Respondent dependent

• Fixed line diminishing

• Mobile only growing rapidly in some countries

• Response declining

• Legislative issues (do not call etc…)

% Individuals with a phone of any sort

Source: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CY DK LU EL FI FR UK AT BE DE IE IT SI CZ ES EE HU SK LV PT LT PL MT TR MK BG RO

2005 2008

% Individuals with a mobile phone

Source: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FI LU CZ DK SI IE EE IT UK BE AT SK CY ES FR DE LT LV PT HU EL PL MT TR BG MK RO

2005 2008

% Individuals with mobile and no fixed line

Source: Eurobarometer 2005 & 2008

-20%

0%

20%

40%

60%

80%

100%

SE NL LU DE CY UK EL SI DK FR IE PL BE ES AT IT HU EE PT SK LV CZ FI LT

2008

2005

Individuals with a mobile but no fixed line

Penetration within ageSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

15-24 25-39 40-54 55+IT CZ ITITIT ELELELEL PTPTPTPT CZCZCZ

Mobile only challenges

• Inadequate/non-existent sampling frames

• Legislative/ethical barriers to RDD - e.g. cost to respondents

• Multiple chances of selection

• Questionnaire length restrictions

• Poor response levels

CAWI feasibility- drivers and constraints

• Respondent dependent

• Little defined best practice

• Coverage variability

• Sample frame availability

• Access panel approach the norm

• Results are design and process sensitive

% Individuals with internet access at homeSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

EL HU SK LT PT LV IT PL CY CZ ES FR IE AT EE DE SI UK BE LU FI DK SE NL

2008

2005

Individuals with internet at home

Penetration within ageSource: Eurobarometer 2005 & 2008

0%

10%

20%

30%

40%

50%

60%

70%

80%

15-24 25-39 40-54 55+ITITITIT ELELELEL PTPTPTPT CZCZCZCZ

Summary

• Fixed-line below 70% in a number of EU countries• Rate of mobile-only coverage growth can invalidate

national trend comparisons for fixed-line designs• Internet penetration under 60% in around 50% of EU

countries• Internet penetration is highly skewed• Internet skew by education suggests a minority will never

access mode• Fixed-line and Internet coverage gaps are different• Questionnaire effects compound coverage and frame

bias

A way forward?

• Fitness for purpose model provides optimal approach

• Re-visit and clearly articulate priorities for Flash- for example speed, validity and consistency

• Evaluate mobile-only coverage gap effects against priorities, by re-analysis of standard EB data

• Assess alternative single mode designs• Expert group to advise on mixed-mode design

implications

A way forward?

• Choose uni-mode or mixed-mode model• Expert group to produce detailed specification

and consensus view on budget requirement• Specification to include ongoing validation and

detailed quality control metrics supported by audit

• Tender evaluation using a ceiling price accompanied by quality base contract award as opposed to current price/quality hybrid

• Expert group provides independent ongoing performance and validation monitoring

18