show me the money? effects of preincentives, differential ......show me the money? effects of...
TRANSCRIPT
![Page 1: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/1.jpg)
SHOW Me the Money?Effects of Preincentives, Differential Incentives,
and Envelope Messaging in an ABS Mail Survey
Kristen Cyffka1,2, Jennifer Dykema1, John Stevenson1, Kelly Elver1, Karen Jaques1
University of Wisconsin Survey Center
© 2012. Materials may not be reproduced without permission of the author.
1University of Wisconsin Survey Center2Department of Statistics, University of Wisconsin-Madison
AAPOR
May 18, 2012
![Page 2: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/2.jpg)
Background
• RDD telephone surveys are no longer a viable mode for many studies
• Decline in last 2 decades in response rates
• Inadequate coverage due to cell phones
• Researchers are increasingly looking to alternative designs
• Return to mail surveys due to the DSF and ABS
University of Wisconsin Survey Center
• Return to mail surveys due to the DSF and ABS
• Effective ways to increase response rates in mail surveys (Edwards et al 2002, Church 1993):
• More interesting vs. less interesting questionnaire
• Recorded delivery vs. standard
• Monetary incentive vs. no incentive
2
![Page 3: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/3.jpg)
Research Questions
• How much of an increase will a $5 pre-incentive yield over a $2 pre-incentive?
• Can we attract initial nonresponders with a second incentive?
University of Wisconsin Survey Center
• What effect will messages on the front of the envelope have on response rates?
• What is the most cost-effective incentive package we can give to increase response rates?
3
![Page 4: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/4.jpg)
Study: Survey of the Health of Wisconsin (SHOW)
SHOW 1 SHOW 2
Year 2011 2012
Contacts Initial packetReminder postcardSecond packet
Initial packetReminder postcardSecond packet
Sampled cases 2,608 2,616
University of Wisconsin Survey Center 4
Completed surveys 1,566 1,440
Response rate (RR1) 66.9% 66.8%
Experimental conditions
Pre-incentive
Second incentive
Envelope message
![Page 5: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/5.jpg)
Study: Survey of the Health of Wisconsin (SHOW)
SHOW 1 SHOW 2
Experimental conditions
Pre-incentive - $2- $5
- $2
Second incentive - $0- $2
- $0- $5
Envelope message - No message - No message
University of Wisconsin Survey Center 5
Envelope message - No message- Monetary message
- No message- Monetary message- Health message
![Page 6: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/6.jpg)
Envelope Message: SHOW 1
University of Wisconsin Survey Center 6
![Page 7: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/7.jpg)
Response Rates: Envelope Messaging
57% 57%
11%9%
50%
60%
70%
Resp
on
se R
ate
SHOW 1
After 2nd SAQ
67% 66%
University of Wisconsin Survey Center 7
0%
10%
20%
30%
40%
No Message Money
Resp
on
se R
ate
After 2nd SAQ
After 1st SAQ
p=0.600
![Page 8: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/8.jpg)
Envelope Message: SHOW 2
University of Wisconsin Survey Center 8
![Page 9: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/9.jpg)
Response Rates: Envelope Messaging
54%52% 51%
14%14% 15%
50%
60%
70%
Resp
on
se R
ate
SHOW 2
69% 66%66%
57% 57%
11%9%
50%
60%
70%
Resp
on
se R
ate
SHOW 1
67% 66%
University of Wisconsin Survey Center 9
0%
10%
20%
30%
40%
No Message Health Money
Resp
on
se R
ate
p=0.4370%
10%
20%
30%
40%
No Message Money
Resp
on
se R
ate
p=0.600
![Page 10: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/10.jpg)
Response Rates: Pre-Incentive
53%
61%11%
9%
50%
60%
70%
Re
sp
on
se
Ra
teSHOW 1
64%70%
University of Wisconsin Survey Center 10
0%
10%
20%
30%
40%
$2 $5
Re
sp
on
se
Ra
te
p=0.005
![Page 11: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/11.jpg)
Response Rates: After All Incentives
53%51%
11%
18%
50%
60%
70%
Resp
on
se R
ate
SHOW 2
64% 69%
54%52%
60%62%
11%10%
9%9%
40%
50%
60%
70%
Resp
on
se R
ate
SHOW 1
66%63%
69% 70%
University of Wisconsin Survey Center 11
0%
10%
20%
30%
40%
Resp
on
se R
ate
p=0.015
$2$0
$2$5
Pre:2nd:
0%
10%
20%
30%
40%
Resp
on
se R
ate
p2nd=0.734
$2$0
$2$2
$5$2
$5$0
Pre:2nd:
pAll =0.030
![Page 12: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/12.jpg)
Response Rates: Message and Incentive
67%70%
62%
70%
64%68%
50%
60%
70%
SHOW 2
p=0.141
University of Wisconsin Survey Center 12
0%
10%
20%
30%
40%
50%
No Message
Health
Money
$2$0
$2$5
Pre:2nd:
![Page 13: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/13.jpg)
Costs per Complete by Incentive
SHOW 1 Pre:2nd:
$2$0
$2$2
$5$0
$5$2
Total Number of Completes 383 371 403 423
Response Rates 65.7% 63.1% 69.0% 70.6%
Cost Per Complete $23.31 $26.05 $26.67 $26.28
University of Wisconsin Survey Center 13
SHOW 2 Pre:2nd:
$2$0
$2$5
Total Number of Completes 669 741
Response Rates 64.4% 69.3%
Cost Per Complete $25.49 $28.37
![Page 14: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/14.jpg)
Other Analyses in SHOW 1
Data quality
• Item-missing data: no difference between treatments
Differential response
• No difference between treatments in:
University of Wisconsin Survey Center
• No difference between treatments in:• Gender
• Age
• Race
• Education
• Employment
• County
14
![Page 15: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/15.jpg)
Discussion
• Envelope messaging
• Effects on response rates
• Future study
• Incentives
University of Wisconsin Survey Center
• Incentives
• Effects on response rates
• Pre-incentives
• Second incentives
• Implications on cost
15
![Page 16: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/16.jpg)
Thank You!
For copies of this presentation or more information, contact:
Kristen Cyffka
University of Wisconsin Survey Center
Please visit us at:www.uwsc.wisc.edu
16
![Page 17: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/17.jpg)
Response Rates: After All Treatments
53%56%
61% 59%
53% 51%
62% 61%10%
13% 6%11%
10% 10%
9% 8%
40%
50%
60%
70%
80%
SHOW 1
University of Wisconsin Survey Center 17
0%
10%
20%
30%
40%
Message None Message None Message None Message None
$2 $2 $5 $5 $2 $2 $5 $5
$0 $0 $0 $0 $2 $2 $2 $2
![Page 18: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/18.jpg)
Response Rates: After All Treatments
55%50%
54% 53% 54%
47%
12%12% 10%
17% 17%20%
40%
50%
60%
70%
80%
SHOW 2
University of Wisconsin Survey Center 18
0%
10%
20%
30%
40%
None Health Money None Health Money
$2 $2 $2 $2 $2 $2
$0 $0 $0 $5 $5 $5
![Page 19: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/19.jpg)
Response in Initial Nonresponders
25%
30%
35%
40%R
esp
on
se R
ate
SHOW 2: Second Incentive
23%
38%
University of Wisconsin Survey Center 19
0%
5%
10%
15%
20%
$0 $5
Resp
on
se R
ate
p=0.000
![Page 20: SHOW Me the Money? Effects of Preincentives, Differential ......SHOW Me the Money? Effects of Preincentives, Differential Incentives, and Envelope Messaging in an ABS Mail Survey Kristen](https://reader036.vdocument.in/reader036/viewer/2022070821/5f1f96ccbf8ab064bd6ec557/html5/thumbnails/20.jpg)
Cost analysis – What we included
We included:
• Mailings
• Postage, printing of all materials
• Stuffing
• Incentives
• Data entry
University of Wisconsin Survey Center
• Data entry
We did not include:
• Survey development and layout of SAQ
• Programming of data entry instrument
• Project management
• Sample management
• Data delivery
20