tracking research of public information campaigns in the ... · pdf filequestionnaires...

Tracking research of publicinformation campaigns in the

Netherlands

Marc Arnold, Jaap Veenstra

Presentation setup• The Netherlands Government Information Service• Public information campaigns• Why tracking research?• Production process before use of SAS

• Tracking Data Warehouse• Production process with the use of SAS

• Questions

Department of General Affairs

The Netherlands GovernmentInformation Service/DTC

Division of communication research

Public information campaigns

Organisation• 25 - 30 campaigns per yearCharacteristics• Use of massmedia, advertising and marketing

techniques• Own brand: Postbus 51• Accountably to the parliament (meta-evaluation)

• Individual campaigns:– problems with former research design

• New media, and more old media ($ 30 million)• Shift in attention for cost efficiency• Standardised research focused on:

– effect off campaigns– efficiency off media use

• More users off information:– campaign managers– media managers– policy makers– government cost efficiency

Why tracking research?

• For each campaign– pre measurement (n=600)– weekly survey of media reach (n=1200)– post measurement (n=600)

• Continuously– media use– appreciation Postbus 51 brand– Political issues

Research design tracking

• High standards– telephonic screening: stratified random sampling– face-to-face interviews– use of multi media laptops (CAPI/CASI)

Questionnaire

Media consumption

Political issues

Media reach & appreciation

Background variables

Process dataOther

Awareness

Attitudes

Knowledge

Behaviour (intention)

Remembered reach

Reach specific media

Appreciation campaign

Functioning campaign

Campaign-specific4 or 5

DTC-intake

questionnaires campaign material

Screeningsurvey

Preparation field work

executionField work

data filesDTC

Process tracking research

Primary process

Secondary process

Primary production process:reporting

Voor-meting

Bereiks-meting

Effect-meting

Recognition

0

10

20

30

40

50

60

70

80

90

wk 01 wk 02 wk 03 wk 04 wk 05 wk 06 wk 07 wk 08

%

campaign recognition

TV spot

radio one

radio two

outdoor

ad's in newspaper

SPSS code SPSS output

To what extent are you interested in......n ot tt

in ter es ted n eu tr al in ter es ted% % % befor e after

total 19 47 34 63 .5 4 .3*

gen der **men 17 47 36 3 .8 4 .4 *women 30 52 18 43 .4 3 .7 *

age **18-24 34 53 13 3.2 3.325-34 22 48 30 03 .9 4 .1 *35-49 27 45 28 64 .0 4 .3 *50> 19 40 41 44 .1 4 .4 *

r epon den t is a **smoker 42 34 24 43.9 4.1non-smoker 18 48 34 54 .6 5 .0 *

r each ed **reached 19 43 38 0- 4.5not reached 32 53 15 5- 3.9

aver age on7-poin t s cale

metaevaluation

Analysingtrends

BrandawarenessPostbus 51

Secondary production process:reporting

Political issues week 1 - week 33

3,5

4,0

4,5

5,0

5,5

6,0

6,5

7,0

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

safety/criminality

education

environment

Dutch economy

taxes

Building expertise

• What can we expect off a communicationcampaign in terms off effects?

• Is there a relation between media pressure andeffectiveness off campaigns– What’s the most efficient media mix

• in general / specific issues• What’s most important:

– a good concept / use off media• Building simulation models

Old

pro

duct

ion

proc

ess

DTC-intake +questionnaire

questionnaires + show material

screeningsample survey

Preparation field work

executionField work

data filesDTC

Process tracking research

Primary process

Secondary process

DTC-intake +questionnaire

questionnaires + show material

screeningsample survey

Preparation field work

executionField work

data filesDTC

Process tracking research

Reporting

new

pro

duct

ion

proc

ess

Database

Reporting

DTC-intake +questionnaire

questionnaires + show material

screeningsample survey

Preparation field work

executionField work

data filesDTC

Process tracking research

Database

Consultancy

•New Name

•New Skills & Competencies

•New Profile and Image

•New Company

eXplore your potential to grow

Tracking Data Warehouse

• Goals– Improve efficiency standard analyses en reporting– Facilitate secondary analyses

• Requirements– Flexibility in analyses– High demands on layout reports

• Approach– Data model– Extraction, organization and exploitation module– User Interface

Former analysis process

SPSS files

SPSS output

Excel tables

1 SPSS file

n ot totalin ter es ted n eu tr al in ter es ted n

% % % bef or e aftertotal 19 47 34 6813 .5 4 .3 *

gen der **men 17 47 36 3333 .8 4 .4 *women 30 52 18 3483 .4 3 .7 *

age **18-24 34 53 13 373.2 3.325-34 22 48 30 1803 .9 4 .1 *35-49 27 45 28 2164 .0 4 .3 *50> 19 40 41 2474 .1 4 .4 *

r epon den t is a **smoker 42 34 24 4183.9 4.1non-smoker 18 48 34 2594 .6 5 .0 *

r each ed **reached 19 43 38 309- 4.5not reached 32 53 15 358- 3.9

av er age on7-p o in t s cale

-Manual data processing-Visual validation-Unpractical categorization

-Cut and paste of statistical output

Building the Data Warehouse

SPSS files

Excel tables

n ot totalin ter es ted n eu tr al in ter es ted n

% % % befor e aftertotal 19 47 34 6813 .5 4 .3 *

gen der **men 17 47 36 3333 .8 4 .4 *women 30 52 18 3483 .4 3 .7 *

ag e **18-24 34 53 13 373.2 3.325-34 22 48 30 1803 .9 4 .1 *35-49 27 45 28 2164 .0 4 .3 *50> 19 40 41 2474 .1 4 .4 *

r epon den t is a **smoker 42 34 24 4183.9 4.1non-smoker 18 48 34 2594 .6 5 .0 *

r each ed **reached 19 43 38 309- 4.5not reached 32 53 15 358- 3.9

av er age on7-p o in t s cale

Database

EXPLOITATIONEXPLOITATION

EXTRACTIONEXTRACTION

ORGANIZATIONORGANIZATION

Metadata

Metadata

Maintenance

Extraction

background survey answers

-Respondent id-Date of birth-Gender...

Exploitation: example of analysis

• Campaign: smoking prevention• To what extent are you interested in ….

• Calculate frequencies en averages• Analyse by gender, age and education• Extra analysis by smoker/non-smoker• Test for differences in pre and post measurement• Test for differences within groups

To what extent are you interested in ….

To what extent are you interested in ….

Secondary analysis: example• Analysis: political issues for wk1-wk33• Explore pattern for issue Education

3,5

4,0

4,5

5,0

5,5

6,0

6,5

7,0

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

safety/criminality

education

environment

Dutch economy

taxes

Conclusions• Time-saving primary process:

– no manual data processing– no cut and paste of statistical output

• Improved quality– automatic data validation

• Meet requirements for– flexibility– demands on reports

• Categorized data suitable for secondary process

⇒ Improved utilization of Tracking Research

Visit our stand for demo and moreinformation