Comparing Social Media and Traditional Recruitment Methods: Which is Most Effective?

Jennifer Anderson, Jocelyn Newsome, and Kerry Levin, WestatApril Oh, NCI

American Association of Public Opinion Research, Hollywood, FL May 16, 2015

2

Social Media is Everywhere

Method

3

• Compare traditional and social media methods of recruitment of adult women for a focus group research project

• Measure cost and effectiveness of recruiting across the two methods

Purpose of Recruitment Study

• Recruited for eight focus groups about health with women ages 20-50

– 2 groups with physically inactive African-American women;

– 2 groups with physically inactive Latina women;

– 2 groups with physically inactive white women; and,

– 2 groups with women (mix of race/ethnicity) who are physically active.

• $30 for 100-minute evening focus group

• No food offered

*The research project was approved by both NCI’s and Westat’s Institutional Review Board.

5

NCI Focus Group Research Project

• Flyers, Newspaper Ads, Craigslist Ads

6

Traditional Recruiting

• Facebook Ads, Google Ads, LinkedIn Ads, Blogs/Forums

7

Social Media Recruiting

● Historically effective

● Inexpensive

● Subject to “professional respondents”

● Unable to easily target who sees the ad

8

● Ability to easily target respondents

● Ability to quickly adapt recruiting strategies

● New approach so there is little research

Traditional

Recruiting

Social Media

Recruiting

9

Recruitment Study Design

Respondents saw ad

Respondents called or emailed

Respondents screened over

telephone

Respondents selected

Respondents scheduled

Focus Group

• Two-month recruitment

• Utilized two equally experienced recruiters who shared responsibility

• Tracked time and money spent for each method separately

Recruitment Process

Effectiveness of Recruiting

– Responses to ad

– Screening rate

– Eligibility rate

– Selection rate

– Show rate

Cost

– Labor hours

– Advertisements

– Cost per eligible

– Cost per show

Measures

10

Results

12

Recruitment Effectiveness

668 responses

315 screened

235 eligible

66 invited

25 showed

408 responses

238 screened

170 eligible

24 invited

16 showed

Social MediaTraditional

Traditional

Recruiting

Social Media

Recruiting

Responses

to ad408 people 668 people

More people responded to

social media ads

Screening

Rate58.3% 47.2%

Significantly higher screening

rate for traditional adsp <.05

Eligibility

Rate71.4% 74.6% No significant difference

p <.05

Selection

Rate14.1% 28.1% No significant difference

p <.05

Show Rate 66.7% 37.9% No significant differencep <.05

13

Recruiting Effectiveness

Recruitment Sources

61.0%

39.0%

Percentage of Focus Group Participants by Source

Social media(n=25)

Traditional(n=16)

53.7%

2.4%4.9%

Percentage of Focus Group Participants by Source

Facebook (n=22)

LinkedIn (n=0)

Google (n=1)

Blogs/Forums (n=2)

15

Recruitment Sources

16

Cost in Labor Hours

0

50

100

150

200

250

300

350

Traditional Recruiting Social Media Recruiting

Recruiter

Management ofRecruitment

236

hours

350

hours

$471.00

$4,907.56

0

1000

2000

3000

4000

5000

6000

Traditional Social Media

17

Cost of Ads

Traditional170 eligible

$163

Social Media235 eligible

$192

Cost per Eligible(including both labor & ads)

18

Traditional16 shows

$1,729

Social Media25 shows

$1,806

Cost per Participant

19

Traditional

• More success at contacting respondents to screen for eligibility

• Lower advertising and labor costs

• Lower cost per eligible

• Comparable cost per participant

Social Media

• More responses, leading to ultimate higher participation

• Higher advertising and labor costs

• Higher cost per eligible

• Comparable cost per participant

Summary of Results

20

Discussion

21

• Challenging recruit(specific demographics and physical activity levels, low incentive)

• Conducting screening by phone may have reduced screening rate for individuals responding to social media ads

22

Things to Consider

• Experiment with less challenging recruit(less specific recruitment criteria, better incentive)

• Experiment with online screener to perhaps increase screening rate for social media

• Experiment with other social media platforms

– Mturk, Twitter, TrymyUI, Instagram, “next new thing”

23

Next Steps

Jennifer Anderson

jenniferanderson@westat.com

24

Antoun, C., Zhang, C., Conrad, F., Schober., M. (2013). Comparisons of Online Recruitment Strategies: Cragislist, Google Ads and Amazon’s Mechanical Turk.

Chu, J. L., & Snider, C. E. (2013). Use of a social networking web site for recruiting Canadian youth for medical research. Journal of Adolescent Health, 52(6), 792-

794.

Frandsen, M., Walters, J., & Ferguson, S. G. (2013). Exploring the Viability of Using online Social Media Advertising as a recruitment Method for Smoking Cessation

Clinical trials. nicotine & tobacco research, ntt157.

Graham, A. L., Fang, Y., Moreno, J. L., Streiff, S. L., Villegas, J., Muñoz, R. F., ... & Vallone, D. M. (2012). Online advertising to reach and recruit Latino smokers to

an internet cessation program: impact and costs. Journal of medical Internet research, 14(4).

Head, B. F., Dean, E. F., Keating, M. D., Swicegood, J.E., Powell, R., & Sage, A. J. (2012, October). Recruiting virtual world users for cognitive interviews: A

comparison of Facebook and Craigslist.com advertisements. Presented at 2012 Annual Southern Association for Public Opinion Research Conference,

Raleigh, NC.

Hill, C.A., Dean, E., Murphy, J. (2014). Social Media, Sociality, and Survey Research. John Wiley & Sons.

Kapp, J. M., Peters, C., & Oliver, D. P. (2013). Research recruitment using Facebook advertising: big potential, big challenges. Journal of Cancer Education, 28(1),

134-137.

Murphy, J., Keating, M., Edgar, J. (2013, November). Crowdsourcing in the Cognitive Interview Process. Presented at 2013 Federal Committee on Statistical

Methodology Research Conference, Washington, DC.

Fenner, Y., Garland, S. M., Moore, E. E., Jayasinghe, Y., Fletcher, A., Tabrizi, S. N., & Wark, J. D. (2012). Web-based recruiting for health research using a social

networking site: an exploratory study. Journal of medical Internet research, 14(1), e20.

Ramo, D. E., Hall, S. M., & Prochaska, J. J. (2010). Reaching young adult smokers through the internet: comparison of three recruitment mechanisms. Nicotine &

Tobacco Research, ntq086.

Quach, S., Pereira, J. A., Russell, M. L., Wormsbecker, A. E., Ramsay, H., Crowe, L., & Kwong, J. (2013). The good, bad, and ugly of online recruitment of parents

for health-related focus groups: lessons learned. Journal of medical Internet research, 15(11).

25

References