oppgavegiveren: show cards implications for survey response

1

1

An experimental study on show cards in social surveys

Nordic Statistical Meeting in Bergen 14th-16th of August 2013

Karianne Lund, Frode Berglund og Kjersti Stabell Wiggen Division for Data Collection Methods, Statistics Norway

Division for analyses, Directorate of Labour and Welfare

An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour and Welfare

Background

• Investigate the use of show card in computer assisted telephone interviews (CATI).

• The show card is distributed by mail prior to the interview, together with an information letter about the survey.

• The purpose of the show card is to reduce the response burden and to improve data quality.

2


3

Research questions

o How do show cards affect respondents’ preconception of the survey? To what extent can show cards be interpreted as effecting response rate?

o Do the respondents use the show card?

o How do the use of show cards affect the response distribution and - more importantly- the data quality?


4

Test design

Experiment I Experiment II Experiment III

Show card on alcohol unit (instructive)

Show card on alcohol unit (instructive)

Show card on chronic diseases (retrieval)

N/A Survey on Alcohol, Tobacco and Drugs

(SATD)

Survey on Living conditions and health

(LKH)

N/A CATI CATI

Controlled experiment Embedded experiment Embedded experiment

N=260 N=1947 N=9601

Response rate, preconceived response

burden

Mapping of use, response

distribution/prevalence

Mapping of use, response rate,

response distribution/prevalence


5

Show cards effect on respondents willingness to participate in a survey (Data source: LKH)

Response rates by presence of show card and education


Education

No

show card

With

show card

Difference

Low 53 47 6

Middle 66 60 6

High 71 72 -1

Total 64 60 4

6

Respondents use of show card – if and who (Data

source: SATD )


Low Middle High

Say they did not

get show card

31,6

25,7

24,8

Did not use 29,3 29,4 30,3

Used 39,1 44,8 44,9

Total (= 100 %) 1401 2358 2193

The distribution of the use of show card in the survey.

7

Show cards effect on response distribution and (more importantly) the data quality? (Data source:

SATD)


Show card Mean N Std. dev.

Not received 20,4 329 19,9

Active user 19,4 245 20,9

Knows content

20,3 495 19,7

Refuse to use

18,9 234 17,8

Total 19,9 1303 19,6

Alcohol consumption of alcohol units related to use of show card

8

Show cards effect on response distribution and (more importantly) the data quality? (Data source: LKH)


Show card Mean N Std. dev.

Did not receive 1,26 834 1,74

Say they did not

get

0,99 1302 1,52

Did not use 1,05 1437 1,51

Used 1,45 2088 1,78

Total 1,21 5661 1,66

Average number of reported diseases by use of show card.

9

Conclusions from the study

• Show cards had a negative effect on the response rate, especially among people with low education, and thus increased bias for this group.

• The presence of show card did not have an effect on response distribution, and we can assume it did not improve the data quality.

• Not a large share of the respondents used the show card


10

Thanks for your attention!

An experimental study on show cards in social surveys. Lund, Berglund and Wiggen, Statistics Norway/Directorate of Labour

and Welfare

oppgavegiveren: show cards implications for survey response

Business