Evaluating Active (Opt-In) and Passive (Opt-Out) Consent Bias in the Transfer of Federal

Contact Data to a Third-Party Survey Agency

Joseph W. Sakshaug (corresponding author)Department of Social Statistics, University of Manchester

Humanities Bridgeford Street-G12Manchester M13 9PL

Phone: +44 161 275 0271Fax: +44 161 275 4722

Email: joe.sakshaug@manchester.ac.uk

Alexandra SchmuckerResearch Data Center, German Institute for Employment Research

104 Regensburger Str.Nuremberg 90461

Phone: +49 911 179 1762Email: alexandra.Schmucker@iab.de

Frauke KreuterDepartment of Sociology, University of Mannheim

Department of Statistical Methods, German Institute for Employment ResearchJoint Program in Survey Methodology, University of Maryland

1218Q LeFrak HallCollege Park, MD 20742Phone: +1 301 405 0935

Email: fkreuter@umd.edu

Mick P. CouperJoint Program in Survey Methodology, University of Maryland

Institute for Social Research, University of MichiganPO Box 1248

Ann Arbor, MI 48106Phone: +1 734 647 3577

Email: mcouper@umich.edu

Eleanor SingerInstitute for Social Research, University of Michigan

PO Box 1248Ann Arbor, MI 48106

Phone: +1 734 647 4599Email: elsinger@umich.edu

Tables: 8

Figures: 0

References: 42

Word count: 6,737

ABSTRACT

Obtaining informed consent from individuals to participate in voluntary research studies is

widely considered to be an ethical research practice. However, there is considerable debate over

how consent should be obtained from subjects. Many researchers argue that active (opt-in)

consent is the only type of consent that accurately reflects the true wishes of the subject, and is

closer to the informed consent ideal than passive (opt-out) consent procedures. Opponents of

active consent procedures argue that such procedures harm study participation rates and increase

the risk of self-selection bias to a greater extent than passive consent procedures. Empirical

evaluations of these claims are rare given the lack of studies that experimentally assign subjects

to different consent procedures and utilize a control group (in which no consent is sought) to

facilitate comparison. We report on an experiment that overcomes these issues in an application

of consent to transfer contact data from a federal register to a third-party data collector for

purposes of carrying out a telephone survey. Specifically, we evaluate the impact of requiring

consent on survey participation rates, self-selection bias, and the resulting survey estimates. We

find that the passive consent procedure does a better job of minimizing self-selection bias and

maximizing the validity of the survey estimates (relative to the control group) compared to the

active consent procedure. However, neither procedure is ideal: both consent procedures increase

the total self-selection bias and reduce the sample size. We conclude with a general discussion of

the main findings and their practical implications.

Key words: self-selection bias, nonresponse, informed consent, data privacy, total survey error

1. Introduction

Obtaining informed consent from individuals to participate in voluntary research is a central

tenet of ethical research practice. Legal frameworks and regulations often require that

researchers obtain informed consent from human subjects prior to their participation in voluntary

research studies. For example, the U.S. Privacy Act of 1974 and the U.S. Department of Health

& Human Services put forth regulations mandating that research subjects be properly informed

about the risks and benefits associated with their participation, and outlined several required

elements of informed consent, including an explanation of the study procedures and their

purpose, a description of reasonably foreseeable risks and discomforts, an explanation of who to

contact for answers to pertinent questions, and a statement that participation is voluntary and can

be withheld at any time, among other stipulations (U.S. Privacy Act of 1974; U.S. Department of

Health & Human Services). While the intended purpose of the informed consent process is clear,

there is a long-standing concern among researchers that obtaining informed consent reduces

study participation and undermines the validity and generalizability of the study results (Singer,

1978; Kearney, Hopkins, Mauss, and Weisheit, 1983). Hence, a central question for researchers

is to what extent the informed consent process affects the accuracy of inferences drawn from

sample surveys, and what can be done to minimize this source of self-selection error.

Consent to participate in voluntary surveys has declined steeply over many years, especially

for telephone surveys (Curtin, Presser, and Singer, 2005), raising concerns about the quality of

the resulting survey estimates. Sometimes further consent requests are administered to survey

participants after they have already consented to take part in the survey. For example, consent

may be needed to link respondents’ survey answers with administrative data sources, collect

physical or biological measurements, and collect process-oriented information (or paradata),

among others (e.g., GPS, accelerometry measures). Studies investigating the impact of obtaining

consent to these requests often find that individuals who consent are systematically different

from those who do not, suggesting that survey estimates can be affected by different sources of

consent bias (Sakshaug, Couper, and Ofstedal, 2010; Couper and Singer, 2013; Sala, Knies, and

Burton, 2014; Skender, Schrotz-King, Böhm, Abbenhardt, Gigic et al., 2015).

Another survey-related function, one in which informed consent may be needed, is the

transfer of contact information for study subjects from government organizations to a third-party

data collection agency. This type of data transfer is often performed because government

organizations that sponsor data collection activities are usually the only ones with access to

databases containing relevant contact information suitable for sample selection and recruitment.

Privacy laws generally prohibit government agencies from disclosing information about

individuals contained in these databases. However, there are some legal exceptions. For example,

in the United States, federal agencies may disclose record information to the U.S. Census Bureau

without consent from the individual to whom the record pertains “for purposes of planning or

carrying out a census or survey or related activity pursuant to the provisions of Title 13” (U.S.

Privacy Act of 1974, PL 93-579, December 31, 1974; 5 U.S.C.-552a). An analogous situation is

the transfer of contact information from medical records without patient authorization which is

generally prohibited under the HIPAA Privacy Rule unless (contingent on IRB or Privacy Board

approval) the information will be used to conduct research that is “designed to develop or

contribute to generalizable knowledge” (The HIPAA Privacy Act, 45 C.F.R. 164.501).

In Germany, similar legislation exists that allows, under limited circumstances, the transfer of

name, address, and telephone information to a third-party agency without the consent of the

individual for purposes of conducting interviews for scientific research (Section 75, Tenth Book

of the German Social Code). This legislation is frequently drawn on at the Institute for

Employment Research (German abbreviation: IAB) in Nuremberg, the independent research arm

of the Federal Employment Agency (BA). The IAB commissions several household and

establishment surveys on behalf of the BA to conduct labor market research and evaluate active

labor market programs. These studies are used to supplement the vast amounts of administrative

employment information collected by the BA. One source of this contact information is the BA

administrative register, which covers over 80 million individuals in Germany. The primary

purpose of the register is to collect notification processes for the German social security system

and public service processes administered by the BA (see vom Berge, Burghardt, and Trenkle,

2013, for more details about the BA register). As of December 2012, the register contained

information on about 89 percent1 of the German civilian labor force between the ages of 15 and

64. Given its broad population coverage, the BA register is commonly used as a sampling frame

for surveys commissioned by the IAB, including the Panel Study of “Labour Market and Social

Security” (PASS) and the “Further Training as Part of Lifelong Learning” (WeLL), as well as for

surveys that are commissioned by other research institutions. The IAB does not maintain its own

data collection staff and thus contracts with a third-party vendor to carry out all data collection

activities. Names, addresses, and telephone numbers (when available) corresponding to

individuals sampled from the BA register are then forwarded to the contractor immediately prior

to the start of the data collection period.

1 Sources: Integrated Employment Biographies Sample (http://fdz.iab.de/en/FDZ_Individual_Data/Integrated_Employment_Biographies.aspx), own calculations; Bundesagentur für Arbeit, Statistik: Dokumentation “Bezugsgröße 2012” (http://statistik.arbeitsagentur.de/Statischer-Content/Grundlagen/Berechnung-Arbeitslosenquote/Dokumentation/Generische-Publikationen/Dokumentation-der-Bezugsgroesse-2012.pdf); Statistisches Bundesamt (2015): Bevölkerung: Deutschland, Stichtag, Altersjahre, Wiesbaden 2015, own calculations.

As previously mentioned, the sharing of BA contact data for survey purposes is exempt from

data protection laws that normally require the consent of the individual whose information is

being shared. However, recently this exemption came under increased scrutiny by

representatives of the German Federal Data Protection Agency, which is charged with ensuring

that strict data privacy standards are met across Germany’s federal statistical system. A key point

raised was that an individual’s contact data contained within the BA register is confidential

information and should not be revealed to any third-party without the consent of the individual.

The counterargument expressed was that requiring consent would likely lower participation rates

and increase self-selection bias in IAB surveys, which already suffer from low response rates.

This discussion prompted the IAB to experiment with alternative procedures for obtaining

consent from individuals regarding the transfer of their contact data, and assess the potential

impact of this consent step on the sample size and resulting self-selection bias in the subsequent

survey. Two consent procedures were tested: active and passive consent. Under the active

consent (or opt-in) condition, subjects were asked to sign and return a consent form documenting

their authorization to the transfer of their contact data from the IAB to an unnamed third-party

data collection agency. Under the passive consent (or opt-out) condition, authorization was

granted if no action was taken, i.e., a signed refusal form was not returned. A control sample

reflecting the status quo (no consent requirement) was also utilized for comparison purposes.

Further details of the experiment are presented in subsequent sections of this paper, which is

structured as follows. In Section 2 we review the advantages and disadvantages of active and

passive consent procedures by drawing on the empirical literature. In Section 3 we describe the

methodology used in designing and deploying the experiment. In Section 4 we summarize the

main findings of the experiment. Specifically, we examine the extent of sample loss and consent

bias under each consent procedure and determine what effect (if any) the consent procedures

have on the resulting survey estimates. In Section 5 we expand on the study’s main findings and

discuss their practical implications.

2. Background

The choice between active and passive consent procedures has generated considerable debate

among social scientists who conduct voluntary observational studies that pose low risks to

subjects relative to clinical research. Advocates of active consent argue that the procedure,

specifically, the deliberate action of documenting one’s consent with a signature, fulfills the legal

requirements of obtaining informed consent and provides additional protection to researchers

should any legal repercussions arise due to the subject’s participation in the study. They criticize

passive consent procedures on ethical grounds, arguing that subjects who are actually opposed to

participation may be included simply because they did not read or return the opt-out form, or

because they never received the recruitment materials (Ellickson, 1989; Ellickson and Hawes,

1989; Hollman and McNamara, 1999). Writing in the context of research with children, for

which parental consent is sought, Range, Embry, and MacLeod (2001; p. 28) argue that, for these

reasons, “active consent is closer to informed consent than passive consent.”

On the other side of the debate, opponents of active consent argue that the legal and ethical

benefits of this procedure are outweighed by low participation rates, increased risk of bias in the

study variables, and costly implementation relative to passive consent procedures. It has also

been argued that there are many other opportunities to withdraw from survey research (unlike

clinical trials or experiments), as evidenced by breakoff rates in surveys and panel attrition.

Furthermore, active consent opponents argue that a failure to return a signed consent form does

not necessarily reflect a deliberate refusal to participate. Indeed, there is evidence that passive

consent procedures yield higher participation rates, reduce the likelihood of self-selection bias,

and decrease implementation costs relative to active consent procedures (Ellickson and Hawes,

1989; Anderman, Cheadle, Curry, Diehr, Shultz et al., 1995; Schuster, Bell, Berry, and Kanouse

et al., 1998; Johnson, Bryant, Rockwell, Moore, Straub et al., 1999; Range, Embry, and

MacLeod, 2001; Junghans, Feder, Hemingway, Timmis, and Jones, 2005). Even outside of

survey research, passive consent procedures can have strong participation effects when consent

is the default option. For example, the percentage of organ donors is considerably higher in

countries that employ passive (as opposed to active) consent procedures for organ donor

registration (Johnson and Goldstein, 2003).

A key question on both sides of the debate is to what extent each procedure accurately

captures the true wishes of the subject. Some empirical evidence sheds light on this issue. In a

follow-up interview of parents who had passively consented to their child’s participation in a

research study by not returning an opt-out form, Ellickson and Hawes (1989) found that the

overwhelming majority (96 percent) of parents indeed intended to allow their child to participate.

This is in contrast to the active consent group, in which about 87 percent of those who initially

failed to mail back a signed form indicating consent eventually did so after further prompts.

Thus, for the majority, failure to return the signed form did not appear to be a deliberate attempt

to withhold consent. A similar result was found by Singer (1978, 2003), who noted that some

respondents are willing to participate in research but are not willing to sign a consent form. The

requirement for the documentation of informed consent may serve to increase the (perceived)

risk of disclosure for subjects.

The aforementioned research indicates that active and passive consent procedures can

differentially impact the consent rate. However, what is also important to investigators is the

extent to which each procedure introduces bias in the survey estimates. There are generally two

strategies that researchers have adopted to assess consent bias. The first strategy is to randomize

subjects to an active or passive consent condition and evaluate whether systematic differences

exist between consenters in each group. Usually the passive consent group is treated as the

reference group against which the active consent group is compared. Thus, any systematic

differences that result are attributed to increased bias in the active consent procedure. The second

strategy for assessing consent bias, commonly used in observational studies employing a single

consent procedure, is to simply compare consenting and non-consenting subjects on variables

available for both groups (whether from the frame, or from survey questions asked prior to the

consent request). Both of these bias evaluation strategies have been used in the context of

consent to study participation (Kearney et al., 1983; Severson and Ary, 1983; Anderman et al.,

1995; Dent, Sussman, and Stacy, 1997; Pokorney, Jason, Schoeny, Townsend, and Curie, 2001;

Henry, Smith, and Hopkins, 2002; White, Hill, and Effendi, 2004; Junghans et al., 2005; Spence,

White, Adamson, and Matthews, 2014), and record linkage consent (Jenkins, Cappellari, Lynn,

Jäckle, and Sala, 2006; Sakshaug, Couper, Ofstedal, and Weir, 2012; Sala, Burton, and Knies,

2012; Knies and Burton, 2014).

However, a common (and understandable) limitation of consent bias studies is the lack of a

control group in which consent is not requested. This omission limits the analysis of bias in two

important ways. First, in the context of assessing the biasing effects of multiple consent

procedures, assuming that one procedure is bias-free relative to the other may be inappropriate,

even when the “gold standard” reference procedure yields higher rates of consent. The survey

participation literature has shown that nonresponse rates and nonresponse biases are only weakly

correlated, with high participation rates providing no guarantee of less bias (e.g., see Groves,

2006; Groves and Peytcheva, 2008) and, in some cases, increased bias results when efforts are

undertaken to improve the response rate (e.g., Keeter, Miller, Kohut, Groves, and Presser, 2000).

A second consequence of omitting a control group in consent studies is that it precludes any

analysis of how the consent request may influence the answers that respondents give to the

substantive questions asked in the study. That is, in order to fully capture the extent of bias

attributed to each consent procedure it is necessary to consider not only differences between

subjects who consent and those who do not, but also the effect that exposure to the consent

procedure has on their answers to the study questions.

There are virtually no studies that have examined the effects of nonconsent using a control

group for comparison. One exception is the study by Das and Couper (2014). Using the

Longitudinal Internet Studies for the Social Sciences in the Netherlands, a random selection of

respondents were sent an advance letter detailing plans to link their survey information with

administrative records maintained by Statistics Netherlands. A passive consent procedure was

adopted which gave respondents the ability to “opt-out” if they objected to the linkage. The

mode of delivery (e-mail versus postal mail) and the amount of text (short versus long) were

experimentally manipulated in a 2x2 factorial design. Overall, the consent rate was about 95

percent and there was no strong evidence that the mode and content manipulations influenced the

consent rate. A follow-up survey was then conducted with these respondents as well as a set of

respondents who did not receive the advance letter (control group). The survey contained several

attitude questions related to privacy, confidentiality, and trust in survey organizations, among

other items. In general, Das and Couper found that concerns about privacy issues were largely

unaffected by the consent statement; however, respondents who received the consent statement

did report higher levels of trust in government agencies and agencies that study public opinion.

To sum up, studies examining the effect of bias due to active and passive consent procedures

are sparse and generally limited to participation in school-based studies and record linkage

among survey respondents. To our knowledge, no studies have simultaneously examined the

biasing effects of these different consent procedures on individuals’ authorization to transfer

their federal contact data to a third-party survey agency prior to receiving an invitation to

participate in a subsequent survey interview. We sought to address this knowledge gap by

investigating 1) the extent to which active and passive consent procedures affect consent rates to

the transfer of contact data; 2) the impact of the consent procedures on the bias due to

nonconsent, nonresponse, and total self-selection into the survey; and 3) the extent to which each

consent procedure affects the resulting survey estimates relative to the control group.

Our expectations are outlined as follows. In accordance with the consent literature, we expect

that the group receiving the active consent procedure will consent at a lower rate than the group

receiving the passive consent procedure. However, subsequent survey participation is likely to be

higher among those who actively consent to contact data transfer compared to those who

passively consent, because the former have already explicitly expressed their interest in

participating in the survey by returning the consent form. Therefore, the overall survey yield (i.e.

the percentage of the total sample that completed interviews), calculated as the product of the

consent and response rates, may be similar under both consent procedures. Given the likely

higher consent rate under the passive consent condition, we expect the magnitude of consent bias

(and total self-selection bias) to be smaller for this group compared to the group assigned to the

active consent condition. Furthermore, we expect that the total self-selection bias (due to consent

and nonresponse) will be larger in both consent groups than in the control group, which is only

affected by nonresponse bias.

3. Methodology

3.1 Consent Experiment

We utilize the BA register for this experiment. As previously mentioned, a distinct advantage of

this register is that it comprises information on the majority of the labor force in Germany.

Hence the register can be used to draw large and diverse samples of the German population. A

second advantage is that it contains detailed administrative information about individuals’

employment history that facilitates bias analysis. These data have been collected continuously on

a daily basis since 1975 and include about 260 million employment history episodes for about 86

million individuals. However, because there is a significant time lag between the availability of

these data and its provision to researchers, the version of the register used for sampling in this

study comprises data only through December 31st, 2012. Only persons with at least one

observation at this reference date were eligible for simple random selection into the sample.

After removing foreign and invalid addresses, the total eligible sample consisted of 16,012

individuals randomly assigned across three groups: 4,460 in the active consent group, 4,551 in

the passive consent group, and 7,001 in the control group.2 In the beginning of February 2014,

letters were mailed to the 9,011 individuals assigned to the consent groups; no mailings or

requests for consent were sent to the control group, reflecting the status quo. The two consent

groups each received different versions of the letter. Both letters requested permission to transfer

the person’s contact details to an unspecified data collection agency that would contact them in 2 The sample size of the control group was initially intended to be roughly the same as the active and passive consent groups. However, due to the lower-than-expected yield in the active consent group, the control group sample size had to be increased in order to achieve the target number of 1,200 completed interviews.

the future to conduct an IAB-sponsored telephone survey under the theme “Challenges in the

German Labor Market 2014.” All the usual confidentialities were ensured. The only difference in

the letters pertained to the instructions for expressing consent. For the active consent group,

permission to release contact data was granted only if the individual signed and mailed back the

enclosed consent form. For the passive consent group, individuals were asked to sign and mail

back the form only if they did not want their contact data shared. Copies of both active and

passive consent letters (translated into English) can be found in the Appendix. A deadline of

March 7th was given to both groups to return the signed form. A postage-paid envelope was

enclosed in each mailing to facilitate return. Because the initial response to the active consent

letters was very low the decision was made to send a reminder to all persons in this group who

did not respond by the deadline. Hence, about 4,300 reminders were sent in March with a new

deadline of April 13th. Reminders were not sent to the passive consent group as this might have

created the impression that the IAB was encouraging refusal of the data transfer. Most responses

were received before the extended deadline but delayed responses were also accepted until June

2014.

A total of 2,213 signed forms (out of a possible 9,011) were returned, for an overall return

rate of 24.6 percent. For the active consent group, a total of 550 signed active consent forms (out

of a possible 4,460) were received from individuals who authorized the contact data transfer, a

return (and authorization) rate of 12.3 percent. For the passive consent group, a total of 1,663

(out of 4,551) signed consent forms were received from individuals who objected to the data

transfer (see Table 1), a return rate of 36.5 percent who actively opted out of the data transfer,

leaving the remaining 63.5 percent who passively consented.

Table 1. Consent to Contact Data Transfer by Survey Outcomes and Experimental Groups.

Experimental Groups

Active Consent Passive Consent Control TotalN % N % N % N %

Initial sample size 4,460 -- 4,551 -- 7,001 -- 16,012 --Consent experiment Returned form No responseTotal

5503,9104,460

12.3387.67100.00

1,6632,8884,551

36.5463.46100.00

-- -- 2,2134,1989,011

24.5646.59100.00

Consent to data transfer 550 12.33 2,888 63.46 -- -- 3,438 38.15Transferred to vendor 550 -- 2,500 -- 7,001 -- 10,051 --

Survey outcome Interview Refusal No contact Phone number avail. No phone number avail.Total

174121

116139550

31.6422.00

21.0925.27100.00

357767

858518

2,500

14.2830.38

34.3220.72100.00

6772,074

1,8622,3887,001

9.6729.62

26.6034.11100.00

1,2082,962

2,8363,04510,051

12.0229.47

28.2230.30100.00

Other survey rates Cooperation 58.98 31.76 24.61 28.97 Overall survey yield -- 3.90 -- 9.06 -- 9.67 -- --

Note: Column percentages do not always sum to 100 due to rounding.

3.2 Survey Data Collection

Upon completion of the consent experiment, a telephone survey was conducted by the contracted

data collection agency. Complete names, addresses, and telephone numbers (when available)

were transferred from the BA register to the third-party vendor for persons who provided active

consent (n=550), a random sample of persons who passively consented (n=2,500), and all

persons assigned to the control group (n=7,001). Telephone numbers (when available) were

transferred for 6,072 cases. For the remaining 3,979 cases, the vendor attempted to retrieve a

telephone number by matching addresses to a commercial database. This process yielded

telephone numbers for an additional 934 cases (a match rate of 23.5 percent). From an inferential

perspective, the non-telephone cases would be considered non-coverage, but since the study

started as a household address sample we keep them in the eligible sample and treat them as

noncontacts in the subsequent analyses.

After a pre-test from 17th-22nd September 2014, the actual fieldwork took place between 9th

October and 19th November 2014. In total, 1,208 interviews were realized, yielding a response

rate of 12.0 percent (see Table 1). With reference to the experimental groups, the response rates

differed widely. As expected, the response rate of the active consent group was much higher

(31.6 percent) than that of the passive consent (14.3 percent) and control (9.7 percent) groups

(AAPOR RR1; AAPOR, 2016). A similar pattern was observed for the cooperation rate,

computed as the ratio of completed interviews to the sum of all contacted cases (interviews plus

refusals), which was 59.0, 31.8, and 24.6 percent for the active consent, passive consent, and

control group, respectively. However, even with the higher response rate, the overall survey

yield – computed as the number of completed interviews over the total sample – of the active

consent group (3.9 percent of the original sample) was still much lower than that of the passive

consent (9.0 percent) and control (9.7 percent) groups.

The survey questionnaire covered topics related to employment and job seeking activities, the

use of online social media, and attitudes towards data sharing and privacy, among others. To

simplify the comparison of survey estimates across the consent and control groups, we focus on a

set of 20 categorical variables and allocate them into the following summary categories:

household composition, employment/volunteering, internet/social networks, privacy, data

sharing, and linkage/panel consent. The variables and their groupings can be found in Appendix

Table A1.

3.3 BA Register Data

To evaluate the impact of bias due to the different consent procedures and nonresponse to the

subsequent telephone survey, we use administrative data from the aforementioned BA register.

In our analysis of consent and nonresponse bias, we focus on 16 categorical administrative

variables that are commonly utilized in labor market research. We group the variables into five

broad categories: demographics, employment, wages, job seeking measures, and benefit receipt.

The individual variables and groupings are shown in Appendix Table A2.

3.4 Evaluation Criteria

To facilitate the comparison of consent procedures across the different administrative variable

groups, we utilize two summary measures of bias: average absolute bias and largest absolute

bias. Both bias measures have been utilized in other survey-based comparison studies (Yeager,

Krosnick, Chang, Javitz, Levendusky et al., 2011; Erens, Burkill, Couper, Conrad, Clifton et al.,

2014). We use these summary measures to estimate the effects of active and passive consent

bias, nonresponse bias, and total self-selection bias. We acknowledge that our estimates of bias

implicitly assume that other sources of bias (e.g., measurement, coverage) are constant across the

different experimental conditions. That is, we are looking at relative bias across conditions, and

assume differences in other (unmeasured) bias sources are zero. Thus, we are likely

underestimating the true bias for all conditions.

Average Absolute Bias offers an indication of the extent to which each consent procedure

impacts the bias of the estimates, on average. It is calculated in two steps; shown here for

consent. First, estimates of consent bias are computed for the k th (¿1,2,…, K ) administrative

variable estimate, Y , within a specific variable group i:

Consent Biasi , k ( Y c )=Biasi , k ( Y c )=Y c−Y n (1)

where subscript c denotes the consenters and n denotes the full sample size of the consent

condition. The bias is calculated as the difference between the administrative variable estimate

based on the consenters, Y c, and the corresponding full-sample estimate, Y n. In the second step,

the bias estimates are converted into absolute values and averaged across all Kestimates within

the ith variable group:

Consent Avg .|.|Biasi (Y c )=∑k=1

K

|Biasi ,k (Y c )|K

(2)

Thus, a single measure of average absolute bias is produced for each of the ith variable groups.

To calculate the average absolute bias due to nonresponse, equations (1) and (2) are modified by

replacing Y c with Y r, the respondent-based administrative estimate, and replacing Y n withY c, the

administrative estimate derived from the consenting sample. To calculate the average absolute

bias due to the combined effects of consent and nonresponse (i.e. total bias), equations (1) and

(2) are modified by replacing Y c with Y r. Table 2 shows explicit equations for each bias source.

Table 2. Equations for Consent Bias, Nonresponse Bias, and Total Bias.

Consent Bias=Y c−Y n

Nonresponse Bias=Y r−Y c

Total Bias=Y r−Y n

Notes: Y c corresponds to the estimate based on consenters;Y r is the estimate based on survey respondents; and Y n is the full sample estimate. Total bias refersto the combined estimate of consent and nonresponse bias.

The second summary measure of bias, largest absolute bias, is used to assess the maximum

impact of bias among the individual estimates within a particular variable group and consent

procedure. It is simply computed as the maximum estimate of bias within variable group i:

Largest|.|Biasi=max (|Biasi , k (Y c)|) (3)

To evaluate the extent to which the resulting survey estimates are affected by the consent

procedures, the same summary measures are used. The only difference is that bias is computed

as the difference between the survey estimate produced under each consent procedure and the

corresponding survey estimate produced under the control group.

4. Results

4.1 Consent Bias in Active and Passive Consent Procedures

The first set of analyses examines the impact of bias due to differential consent to contact data

transfer under the active and passive consent procedures. Administrative characteristics of

consenters under each procedure are presented in Appendix Table A3. In general, the table

shows that both consent procedures yield statistically significant consent biases for several

administrative variables.

Summary measures of consent bias (average absolute and largest absolute bias) are presented

in Table 3 for each administrative variable group. Overall, the average absolute bias for the

active consent group is about twice as large as the passive consent group (active: 2.39; passive:

1.15). The average absolute bias for the active consent group exceeds that of the passive consent

group for most variable groups. The largest discrepancy occurs for the demographics group,

where the average absolute bias for the active consent group exceeds that of the passive consent

group by a factor of about 3.7 (active: 5.27; passive: 1.44). The smallest discrepancy occurs for

the employment variable group, which yields a slightly smaller average absolute bias under the

active consent condition (active: 1.36; passive: 1.42). The only other variable group indicating a

smaller average absolute bias in the active consent group is job seeking measures, which is about

half the size of the passive consent group (active: 0.56; passive: 1.06).

The second summary measure, largest absolute bias, produces similar conclusions, with the

passive consent procedure yielding smaller maximum biases compared to the active consent

procedure for most variable groups. The largest overall absolute bias occurs for the

demographics group in the active consent condition (12.60), which is about three times as large

as the next largest absolute bias. This unusually large absolute bias is driven by a

disproportionately high rate of active consent among older persons (see Appendix Table A3).

Table 3. Summary Measures of Consent Bias by Experimental Conditions and Administrative Variable Groups. Parenthetical Entries Represent Number of Estimates Included in Each Variable Group.

Experimental ConditionsSummary Bias Measure Active Consent Passive ConsentAvg. Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)Overall (41)

5.271.361.810.561.242.39

1.441.420.891.060.681.15

Largest Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)

12.604.273.430.663.22

4.174.161.691.781.51

4.2 Nonresponse Bias Conditional on Active and Passive Consent Procedures

The previous analyses showed that the passive consent procedure produces less consent bias, on

average, compared to the active consent procedure for most variable groups. To assess which

consent procedure is optimal for minimizing bias it is also important to evaluate the impact of

nonresponse conditional on active or passive consent. This is the purpose of the next analyses,

which examine how the telephone survey respondents differ from their respective samples under

the consent and control conditions. Full details of respondent administrative characteristics and

differences with their respective samples can be found in Appendix Table A4.

Table 4 provides summary measures of absolute nonresponse bias for each consent procedure

and the control group. The overall pattern of nonresponse bias is similar to the pattern of consent

bias presented earlier. That is, the overall average absolute bias for the active consent group is

larger than the passive consent group (active: 2.61; passive: 2.05; control: 2.03); interestingly,

the passive consent group is nearly identical to that of the control group. For most variable

groups, the passive consent and control conditions yield smaller average absolute nonresponse

bias than the active consent condition with two exceptions: the demographics and job seeking

measures groups both yield smaller average absolute nonresponse biases under the active consent

condition compared to the passive consent and control conditions. However, on the whole,

discrepancies between the two consent procedures in terms of average absolute bias are rather

modest and do not exceed one unit in most cases. Similar patterns are revealed by examining the

largest absolute bias summary measure, which shows that maximum nonresponse biases are

modestly larger under the active consent condition than the passive consent and control

conditions for most variable groups.

Table 4. Summary Measures of Nonresponse Bias by Experimental Conditions and Administrative Variable Groups. Parenthetical Entries Represent Number of Estimates Included in Each Variable Group.

Experimental ConditionsSummary Bias Measure Active Consent Passive Consent ControlAvg. Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)Overall (41)

2.951.832.621.173.372.61

3.241.481.772.151.602.05

3.491.061.852.081.692.03

Largest Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)

4.884.664.091.706.32

4.973.032.972.573.23

5.372.052.643.363.32

4.3 The Joint Impact of Consent and Nonresponse Bias on Total Self-Selection Bias

The analyses above suggest that for most variable groups biases due to differential consent and

nonresponse tend to be smaller under the passive consent procedure than the active consent

procedure. However, what remains unclear is how consent and nonresponse bias jointly impact

the total self-selection bias of the estimates under these two procedures. The joint effect of these

biases is likely to be cumulative if consent and nonresponse biases shift estimates in the same

direction, but an offsetting effect may occur if both biases shift estimates in opposite directions.

Taking the joint impact of both biases into account is important when determining which consent

procedure is optimal from a total bias perspective.

Individual estimates of consent bias, nonresponse bias, and total bias are presented for the two

consent procedures and the control group in Table 5. In short, there is evidence of both

cumulative and offsetting effects of consent and nonresponse bias. Out of 41 total estimates,

offsetting biases occur for 17 and 18 estimates under the active and passive consent conditions,

respectively. However, only two estimates experience offsetting biases under both procedures

(No. of benefit receipts: 2-3; No. of employer changes since 2008: 3+), suggesting that

cumulative and offsetting effects occur differently under the two consent procedures.

Table 5. Estimates of Consent and Nonresponse Bias (and Standard Errors) by Experimental Conditions and Administrative Characteristics. Bold Font Denotes Statistically Significant Bias Estimate at the 0.05 Level.

Active Consent Condition Passive Consent Condition Control ConditionAdministrative characteristics

ConsentBias

NonresponseBias

TotalBias

ConsentBias

NonresponseBias

TotalBias

NonresponseBias

Male -1.57(2.00) 3.53(3.13) 1.96(3.71) 1.97(0.56) 1.46(2.45) 3.61(2.53) -1.21(1.83)

Age 20-33 34-45 46-54 55+

-10.26(1.44)

-4.87(1.61)

2.53(1.78)

12.60(1.92)

-1.93(2.11)

-3.73(2.40)

2.07(2.85)

3.59(3.08)

-12.19(2.47)

-8.60(2.76)

4.60(3.40)

16.19(3.66)

4.17(0.47)

0.56(0.49)

-1.78(0.50)

-2.95(0.50)

-4.83(2.13)

-3.67(2.05)

4.89(2.19)

3.61(2.10)

-0.99(2.19)

-2.94(2.11)

3.28(2.29)

0.65(2.20)

-3.50(1.52)

-4.06(1.51)

2.63(1.64)

4.94(1.64)

Foreign citizenship -3.22(0.94) -4.88(1.02) -8.1(0.70) 1.57(0.28) -3.84(1.19) -2.42(1.19) -5.22(0.72)

Education Secondary/intermediate Upper secondary College/University Missing

-9.97(1.96)

3.37(1.48)

6.75(1.63)

-0.15(0.79)

-4.04(3.13)

2.63(2.43)

3.11(2.68)

-1.70(1.08)

-14.01(3.72)

6.00(2.95)

9.86(3.23)

-1.85(3.15)

-1.12(0.52)

0.27(0.39)

0.23(0.38)

0.63(0.21)

-3.89(2.36)

0.89(1.78)

4.97(1.87)

-1.96(0.86)

-5.13(2.45)

1.28(1.85)

5.22(1.96)

-1.35(0.86)

-4.17(1.76)

0.62(1.26)

5.11(1.45)

-1.55(0.62)

Living in former East Germany 2.61(1.66) 1.20(2.65) 3.81(3.16) -0.55(0.46) 1.61(2.02) 1.04(2.10) 5.37(1.59)

Currently employed -1.21(1.72) -0.56(2.73) -1.77(3.24) -0.79(0.48) 0.99(2.09) 0.51(2.16) 0.80(1.56)

Marginally employed 0.59(1.49) -4.66(2.19) -4.07(2.44) -0.88(0.44) -3.03(1.75) -4.08(1.80) -2.05(1.32)

% of days employed since 2008 0 1-33 34-66 67-99 100

-1.19(0.90)

-1.49(1.26)

-0.31(1.22)

-1.28(1.68)

4.27(2.00)

1.05(1.47)

-3.44(1.80)

-0.59(1.89)

1.98(2.67)

0.99(3.14)

-0.14(1.82)

-4.93(1.98)

-0.90(2.22)

0.70(3.21)

5.26(3.72)

0.74(0.23)

1.82(0.35)

1.19(0.36)

0.41(0.49)

-4.16(0.56)

1.08(1.22)

-0.23(1.69)

-0.23(1.65)

1.65(2.19)

-2.26(2.39)

1.71(1.26)

1.42(1.74)

0.85(1.70)

2.17(2.27)

-6.15(2.48)

-0.76(0.85)

0.38(1.23)

-1.01(1.17)

0.41(1.58)

0.98(1.83)

# of employer changes since 2008 0 1 2 3+ Missing

0.91(2.00)

1.52(1.63)

0.57(1.25)

-1.69(1.10)

-1.32(0.90)

-0.91(3.13)

-3.45(2.48)

1.55(2.04)

1.77(1.79)

1.05(1.47)

0.00(3.72)

-1.93(2.86)

2.12(2.47)

0.08(2.22)

-0.27(1.82)

-3.55(0.56)

0.33(0.44)

1.02(0.35)

1.44(0.33)

0.75(0.24)

-2.30(2.45)

2.93(2.06)

-0.03(1.63)

-1.84(1.52)

1.24(1.23)

-6.29(2.54)

3.74(2.14)

1.03(1.68)

-0.39(1.63)

1.90(1.28)

-1.45(1.83)

1.35(1.49)

-1.02(1.05)

1.81(1.23)

-0.70(0.86)

Average daily wage 0-33 34-70

-1.59(1.61)

0.66(1.64)

-4.09(2.40)

-0.01(2.57)

-5.68(2.75)

0.65(3.05)

-1.02(0.48)

-0.21(0.47)

-2.59(1.93)

-1.83(2.00)

-3.82(2.00)

-1.64(2.07)

-2.64(1.45)

1.51(1.54)

71-108 109+ Missing

-2.95(1.57)

3.43(1.77)

0.44(1.34)

-2.46(2.37)

3.95(2.85)

2.61(2.20)

-5.41(2.75)

7.38(3.44)

3.05(2.69)

0.53(0.46)

-0.99(0.47)

1.69(0.34)

0.33(2.08)

2.97(2.05)

1.13(1.69)

1.31(2.16)

1.55(2.14)

2.61(1.75)

-2.51(1.46)

1.19(1.55)

1.42(1.28)

Enrolled in job seeker program 0.66(1.50) 1.70(2.42) 2.36(2.92) 1.78(0.39) 2.57(1.95) 4.22(2.02) 3.36(1.46)

Participates in active labor market program -0.46(0.42) 0.63(0.75) 0.17(0.97) 0.33(0.12) 1.72(0.88) 2.17(0.93) 0.79(0.54)

Receiving unemployment benefit -1.26(1.36) 0.70(2.14) -0.56(2.57) 1.51(0.37) 0.45(1.77) 1.79(1.83) 1.66(1.35)

Receiving income assistance -1.50(1.22) -0.41(1.89) -1.91(2.22) 1.32(0.33) -0.67(1.55) 0.55(1.60) 0.74(1.22)

No. of benefit receipts 0 1 2-3 4+

-0.53(1.95)

-0.09(1.60)

-2.61(1.54)

3.22(1.67)

-6.15(3.00)

2.81(2.59)

6.32(2.55)

-2.97(2.58)

-6.68(3.51)

2.72(3.13)

3.71(3.18)

0.25(2.99)

-0.07(0.55)

0.13(0.45)

-0.75(0.47)

0.69(0.45)

-3.23(2.34)

1.69(2.00)

0.81(2.02)

0.73(2.03)

-3.46(2.42)

1.89(2.08)

0.01(2.10)

1.56(2.11)

-3.32(1.74)

-0.75(1.42)

2.35(1.57)

1.72(1.52)

# days with benefit receipt 0 1-200 201-550 551-1550 1551 or more

-0.53(1.95)

-1.28(1.42)

0.01(1.39)

-0.41(1.42)

2.21(1.51)

-6.15(3.00)

1.36(2.27)

3.24(2.28)

4.24(2.35)

-2.70(2.31)

-6.68(3.51)

0.08(2.74)

3.25(2.81)

3.83(2.81)

-0.49(2.66)

-0.07(0.55)

-0.65(0.41)

-0.80(0.41)

0.25(0.41)

1.27(0.39)

-3.23(2.34)

0.25(1.77)

2.89(1.83)

1.89(1.88)

-1.79(1.75)

-3.46(2.42)

-0.13(1.83)

1.95(1.91)

2.15(1.96)

-0.50(1.80)

-3.32(1.74)

-0.72(1.31)

2.55(1.36)

1.16(1.35)

0.32(1.35)

Note: All administrative variables are measured on the reference date of 31st December 2012.

Next, we use the summary bias measures to assess the net effect of consent and nonresponse

bias on the total self-selection bias across the administrative variable groups (Table 6).

Consistent with previous analyses, the overall average absolute bias tends to be larger under the

active consent procedure than the passive consent procedure and the control group (active: 4.01;

passive: 2.27; control: 2.03). The fact that the average absolute bias under the passive consent

procedure is only slightly larger than that of the control group is remarkable given that the

control group is only affected by one source of bias (due to nonresponse). The active consent

procedure yields larger average absolute bias than the passive consent procedure for three (out of

five) variable groups—the exceptions being employment and job seeking measures. The same

conclusions generally hold based on the largest absolute bias summary measure. The wages and

demographics variable groups produce the largest absolute biases, which are respectively two

and three times greater in the active consent group than in the passive consent group. Even in the

presence of offsetting biases, the control condition yields smaller summary measures of absolute

bias compared to one (and sometimes both) of the consent procedures for all variable groups.

Table 6. Summary Measures of Total Bias (Consent and Nonresponse) by Experimental Conditions and Administrative Variable Groups. Parenthetical Entries Represent Number of Estimates Included in Each Variable Group.

Experimental ConditionsSummary Bias Measure Active Consent Passive Consent ControlAvg. Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)Overall (41)

7.931.854.431.272.744.01

2.542.522.193.201.592.27

3.491.061.852.081.692.03

Largest Abs. Bias Demographics (11) Employment (12) Wages (5) Job seeking measures (2) Benefit receipt (11)

16.195.267.382.366.68

5.226.293.824.223.46

5.372.052.643.363.32

4.4 Comparison of Survey Estimates Conditional on Active and Passive Consent

The final set of analyses evaluates the impact of each consent procedure on the resulting survey

estimates relative to the control group. Estimates of all 20 survey variables are presented in Table

7. For most survey estimates, respondents in the active and passive consent groups are

comparable to those in the control group. Exceptions include social media use, which is slightly

higher among passive consent than control respondents, and the use of social media for reading

and being notified of job vacancies – the latter is less common and the former more common

among passive consenters compared to the control group. The largest impact of the consent

procedures occurs for record linkage and panel consent. Both active and passive consent

respondents agree to record linkage and panel participation at a higher rate than control

respondents, with the active consent group yielding the highest rate of consent for both requests.

We interpret this finding as an indication that the active and passive consent conditions yield

respondents who are more trustful of government agencies (in this case, the BA and IAB) and,

therefore, more willing to share personal information with them compared to respondents who

are not exposed to the consent conditions.

Table 7. Survey Estimates in Percentages (and Standard Errors) and Bias Estimates (Relative to Control Condition), by Consent Condition. Bold Font Denotes Statistically Significant Bias Estimate at the 0.05 Level.

Active Consent Condition Passive Consent Condition Control Condition

Survey characteristicsSurvey

EstimateBias

EstimateSurvey

EstimateBias

EstimateSurvey

EstimateHousehold size: 3 or more 47.98(3.80) 2.22(4.26) 47.31(2.66) 1.56(3.28) 45.75(1.92)

Minors in household 32.76(3.56) 0.42(3.99) 33.43(2.51) 1.09(3.09) 32.34(1.81)

Marital status: never married 24.86(3.29) -6.26(3.74) 33.33(2.51) 2.22(3.08) 31.11(1.78)

Currently employed 73.56(3.34) -4.69(3.70) 78.43(2.18) 0.18(2.69) 78.25(1.59)

Occupation: white collar 76.56(3.74) 1.47(4.19) 78.42(2.47) 3.32(3.11) 75.10(1.89)

Volunteered in last 12 months 36.99(3.67) 5.64(4.08) 32.87(2.49) 1.51(3.07) 31.35(1.79)

Internet access 87.36(2.52) 0.80(2.84) 87.64(1.74) 1.08(2.18) 86.56(1.31)

Uses social media 61.18(3.95) 0.09(4.44) 68.59(2.63) 7.50(3.31) 61.09(2.01)

Used social media website in last 4 weeks (check all that apply) Facebook Whatsapp Other

59.14(5.10)

72.04(4.65)

39.79(5.08)

-3.71(5.70)

-5.33(2.21)

-4.35(5.71)

66.36(3.23)

83.18(2.56)

45.79(3.41)

3.51(4.12)

5.80(3.38)

1.66(4.30)

62.85(2.55)

77.37(2.21)

44.13(2.62)

Read job vacancies on social media 24.73(4.47) -6.00(5.09) 22.43(2.85) -8.30(3.75) 30.73(2.44)

Notified of job vacancies on social media 10.75(3.21) 2.91(3.51) 13.62(2.35) 5.77(2.75) 7.84(1.42)

Has privacy concerns when using social media 72.04(4.65) 6.12(5.29) 65.57(3.26) -0.36(4.11) 65.92(2.51)

Regularly takes action to protect online security (check all that apply) Change passwords Delete cookies

56.85(4.10)

67.11(3.85)

-6.24(4.57)

0.38(4.33)

63.04(2.77)

68.11(2.69)

-0.06(3.43)

1.38(3.34)

63.09(2.02)

66.73(1.99)

Importance of privacy in general: very important 48.86(5.33) 3.33(5.96) 44.07(3.73) -1.47(4.59) 45.53(2.67)

In favor of telephone providers registering telecommunications data to improve their services 14.71(2.72) -1.58(3.07) 16.67(2.00) 0.38(2.46) 16.29(1.43)

In favor of telephone providers

passing stored data to the authorities to fight crime 49.71(3.82) 6.68(4.28) 46.15(2.71) 3.12(3.33) 43.03(1.93)

Consents to business partners re-using personal data for advertising purposes: never 36.78(3.66) -4.29(4.12) 42.25(2.62) 1.18(3.24) 41.07(1.90)

Rather likely to link survey answers with records (check all that apply) Income tax records Employment records Health insurance records Social benefit records School records/certificates

28.32(3.43)

70.52(3.47)

18.97(2.97)

39.31(3.71)

47.40(3.80)

1.57(3.83)

8.88(3.94)

-2.58(3.37)

4.67(4.15)

4.03(4.25)

28.81(2.41)

66.09(2.54)

23.45(2.25)

38.98(2.59)

40.11(2.61)

2.06(2.95)

4.45(3.16)

1.90(2.75)

4.35(3.18)

-3.26(3.23)

26.76(1.71)

61.64(1.88)

21.55(1.59)

34.63(1.84)

43.37(1.91)

Consent to record linkage 93.68(1.85) 11.51(2.36) 88.20(1.71) 6.03(2.26) 82.17(1.48)

Consent to panel participation 93.10(1.92) 6.25(2.32) 90.48(1.55) 3.62(2.03) 86.85(1.30)

Next, we examine the summary bias measures to assess the accuracy of the survey estimates

(relative to the control group) under each consent procedure (Table 8). Overall, the average

absolute bias is larger under the active consent procedure than the passive consent procedure by

about 31 percent (active: 4.15; passive: 2.86). The same pattern holds for nearly all individual

variable groups, which indicates that the passive consent procedure produces survey estimates

that are closer to the control group estimates than does the active consent procedure. The lone

exception is the internet/social media variable group, which yields a larger average absolute bias

under the passive consent than the active consent condition (active: 3.31; passive: 4.80). The

same conclusions can be drawn from examining the largest absolute bias summary measure,

which is higher under the active consent than the passive consent procedure for all variable

groups except internet/social media (active: 6.00; passive: 8.30). The same conclusions hold after

applying a survey weighting adjustment that controlled for age, sex, nationality, and region

(results not shown). In general, the effect of the survey weights was minimal. The average

absolute bias differed between the weighted and unweighted estimates by less than 0.70

percentage points for every survey variable group. The effect of the weights was inconsistent

across variable groups; that is, some variable groups experienced an increase in bias while others

a decrease.

Table 8. Summary Measures of Bias for Survey Estimates (Relative to the Control Condition) by Experimental Condition and Survey Variable Group. Parenthetical Entries Represent Number of Estimates Included in Each Variable Group.

Experimental ConditionsSummary Bias Measure Active Consent Passive ConsentAvg. Abs. Bias Household composition (3) Employment/volunteering (3) Internet/social media (7) Privacy (4) Data sharing (3) Linkage/panel consent (7)Overall (27)

2.973.933.314.024.185.644.15

1.621.674.800.821.563.672.86

Largest Abs. Bias Household composition (3) Employment/volunteering (3) Internet/social media (7) Privacy (4) Data sharing (3) Linkage/panel consent (7)

6.265.646.006.246.6811.51

2.223.328.301.473.126.03

5. Discussion

The present study examined a unique, yet realistic, scenario in which individuals were asked for

consent to transfer their contact data from federal employment records to a third-party data

collection agency for the purpose of conducting a subsequent telephone survey. Active and

passive consent procedures were administered in a randomized experiment and detailed

administrative data were used to determine which procedure is optimal for minimizing bias due

to consent, nonresponse, and total self-selection. Furthermore, a control group reflecting the

status quo of not requiring consent was utilized as a benchmark against which the resulting

survey estimates derived from each consent procedure were compared.

The study can be summarized by five main findings. First, as expected, consent to the transfer

of contact data was substantially lower when active consent instead of passive consent was

requested. However, the response rate to the subsequent telephone survey (conditional on

consent) was significantly higher among the active than passive consenters; both consenters

yielded a higher response rate than the control group. Despite the higher response rate among the

active consenters, the overall survey yield was still considerably smaller than that of the passive

consent condition. Second, while both consent procedures yielded significant consent biases, the

passive consent procedure tended to yield smaller overall consent bias than the active consent

procedure.

Third, the passive consent procedure yielded modestly smaller nonresponse bias, on average,

compared to the active consent procedure. Fourth, despite some evidence of offsetting biases

under each of the two consent conditions, both conditions yielded larger total self-selection bias

than the control group, on average. The passive consent procedure produced less total bias than

the active consent procedure for most variable groups. Lastly, bias in the survey estimates was

modestly smaller under the passive than the active consent condition. It is worth noting that the

same study findings hold when the joint impact of bias and increased variance due to sample loss

is taken into account in a mean squared error framework (see Appendix Tables A5-A8).

Overall, we conclude that the passive consent procedure does a better job of minimizing self-

selection bias and maximizing the validity of the survey estimates (relative to the control group)

than the active consent procedure. However, neither procedure is ideal: both consent procedures

increase total self-selection bias, reduce sample size, and increase the total mean squared error of

the estimates – the passive consent procedure just to a lesser extent.

The finding that active and passive consenters are more likely to consent to record linkage

and panel survey participation is in line with the consent literature which has shown that

respondents who consent to previous requests are more likely to consent to subsequent requests

(Sala, Knies, and Burton 2014). It is also consistent with Das and Couper (2014) who found that

respondents who received the consent treatment (record linkage) had higher levels of trust in

government agencies than respondents who did not receive the consent request. While the higher

rates of record linkage and panel consent are likely to be viewed as a positive side effect of

requiring consent to transfer contact data, one should bear in mind that these respondents are

highly selective and are unlikely to be representative of the target population.

The findings drawn from this study should be interpreted in light of its research design. The

study was based on a sample of individuals drawn from a single federal employment register.

While this register covers the majority of the labor force population in Germany, there are certain

sectors of the workforce that are underrepresented (e.g., civil servants, self-employed), which

may limit broader generalizations of the study’s findings. Furthermore, the study sponsors, the

Federal Employment Agency and the Institute for Employment Research, are well-known

entities in Germany and likely influenced people’s decision to consent to the contact data transfer

and subsequent survey participation. It is unclear whether a less salient entity would yield similar

results under the same experimental design. The role of the survey sponsor in influencing the

consent decision is a notable topic for future research.

Another topic for future research is individuals’ understanding of the active/passive consent

request. Respondents’ understanding of consent is a topic often omitted in the literature under the

implicit assumption that what is meant by consent is perfectly communicated to research

subjects. However, there is some empirical evidence that runs counter to this assumption,

suggesting that survey respondents do not always understand what they are consenting to (Das

and Couper, 2014). The fact that the passive consent condition yielded relatively similar results

to the control condition may speak to the attention (or lack thereof) that respondents paid to the

consent letter and/or to the extent they truly understood what they were (passively) consenting

to.

Another knowledge gap relates to the impact of consent on data collection costs. In our study,

the overall survey yield under the active consent procedure was substantially lower than the

passive consent procedure, thus active consent may be more costly for the same case count as

well as producing greater bias. However, if the telephone effort is significantly cheaper because

these respondents have already consented, then the active consent approach might lead to

reduced costs for the same case count. Understanding the relationship between the survey yield

and data collection costs under each consent procedure is a worthwhile topic for future

exploration.

In conclusion, we find that passive consent procedures produce less self-selection bias than

active consent procedures when consent is requested to transfer federal contact data to a third-

party data collector. Although such transfers are sometimes exempt from federal privacy laws,

these exemptions may be overturned in the future. We hope this study is informative about the

potential impact of such an additional consent step on the validity of the study results.

6. References

The American Association for Public Opinion Research (AAPOR; 2016), Standard Definitions:

Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th Edition. AAPOR.

Anderman, C., A. Cheadle, S. Curry, P. Diehr, L. Shultz, and E. Wagner (1995), “Selection Bias

related to Parental Consent in School-Based Survey Research,” Evaluation Review, 19(6),

663-674.

Bates, N. (2005), “Development and Testing of Informed Consent Questions to Link Survey

Data with Administrative Records,” Paper Presented at the Joint Statistical Meetings of the

American Statistical Association, Minneapolis, MN, August.

<http://www.amstat.org/committees/ethics/linksdir/Jsm2005Bates.pdf>

Couper, M. P., and E. Singer (2013), “Informed Consent for Web Paradata Use,” Survey

Research Methods, 7(1), 57-67.

Curtin, R., S. Presser, and E. Singer (2005), “Changes in Telephone Survey Nonresponse Over

the Past Quarter Century,” Public Opinion Quarterly, 69(1), 87-98.

Das, M., and M. P. Couper (2014), “Optimizing Opt-Out Consent for Record Linkage,” Journal

of Official Statistics, 30(3), 479-497.

Dent, C. W., S. Y. Sussman, A. W. Stacy (1997), “The Impact of a Written Parental Consent

Policy on Estimates from a School-Based Drug Use Survey,” Evaluation Review, 21(6), 698-

712.

Ellickson, P. L. (1989), “Limiting Nonresponse in Longitudinal Research: Three Strategies for

School-Based Studies,” Santa Monica, CA: RAND Corporation, 1989.

<http://www.rand.org/pubs/notes/N2912.html>

Ellickson, P. L., and J. A. Hawes (1989), “An Assessment of Active Versus Passive Methods for

Obtaining Parental Consent,” Evaluation Review, 13(1), 45-55.

Erens, B., S. Burkill, M. P. Couper, F. Conrad, S. Clifton, C. Tanton, A. Phelps, J. Datta, C. H.

Mercer, P. Sonnenberg, P. Prah, K. R. Mitchell, K. Wellings, A. M. Johnson, and A. J. Copas

(2014), “Nonprobability Web Surveys to Measure Sexual Behaviors and Attitudes in the

General Population: A Comparison With a Probability Sample Interview Survey,” Journal of

Medical Internet Research, 16(12):e276.

Groves, R. M. (2006), “Nonresponse Rates and Nonresponse Bias in Household Surveys,”

Public Opinion Quarterly, 70(5), 646-675.

Groves, R. M., and E. Peytcheva (2008), “The Impact of Nonresponse Rates on Nonresponse

Bias: A Meta-Analysis,” Public Opinion Quarterly, 72(2), 167-189.

Henry, K. L., E. A. Smith, and A. M. Hopkins (2002), “The Effect of Active Parental Consent on

the Ability to Generalize the Results of an Alcohol, Tobacco, and Other Drug Prevention

Trial to Rural Adolescents,” Evaluation Review, 26(6), 645-655.

Hollmann, C. M., and J. R. McNamara (1999), “Considerations in the Use of Active and Passive

Parental Consent Procedures,” The Journal of Psychology: Interdisciplinary and Applied,

133(2), 141-156.

Jenkins, S. P., L. Cappellari, P. Lynn, A. Jäckle, and E. Sala (2006), “Patterns of Consent:

Evidence from a General Household Survey,” Journal of the Royal Statistical Society: Series

A (Statistics in Society), 169(4), 701-722.

Johnson, K., D. Bryant, E. Rockwell, M. Moore, B. W. Straub, P. Cummings, and C. Wilson

(1999), “Obtaining Active Parental Consent for Evaluation Research: A Case Study,”

American Journal of Evaluation, 20(2), 239-249.

Johnson, E. J., D. G. Goldstein (2003), “Do Defaults Save Lives?” Science, 302, 1338-1339.

Junghans, C., G. Feder, H. Hemingway, A. Timmis, M. Jones (2005), “Recruiting Patients to

Medical Research: Double Blind Randomised Trial of ‘Opt-In’ versus ‘Opt-Out’ Strategies,”

BMJ, 331(7522):940.

Kearney, K. A., R. H. Hopkins, A. L. Mauss, and R. A. Weisheit (1983), “Sample Bias Resulting

from a Requirement for Written Parental Consent,” Public Opinion Quarterly, 47(1), 96-102.

Keeter, S., C. Miller, A. Kohut, R. M. Groves, and S. Presser (2000), “Consequences of

Reducing Nonresponse in a National Telephone Survey,” Public Opinion Quarterly, 64(2),

125-148.

Knies, G., and J. Burton (2014), “Analysis of Four Studies in a Comparative Framework

Reveals: Health Linkage Consent Rates on British Cohort Studies Higher than on UK

Household Panel Surveys,” BMC Medical Research Methodology, 14:125.

Pascale, J. (2011), “Requesting Consent to Link Survey Data to Administrative Records: Results

from a Split-Ballot Experiment in the Survey of Health Insurance and Program Participation

(SHIPP),” U.S. Census Bureau Research Report Series (Survey Methodology), 2011-03.

Pokorney, S. B., L. A. Jason, M. E. Schoeny, S. M. Townsend, C. J. Curie (2001), “Do

Participation Rates Change When Active Consent Procedures Replace Passive Consent,”

Evaluation Review, 25(5), 567-580.

Range, L. M., T. Embry, and T. MacLeod (2001), “Active and Passive Consent: A Comparison

of Actual Research with Children,” Ethical Human Sciences and Services, 3(1), 23-31.

Sakshaug, J. W., M. P. Couper, and M. B. Ofstedal (2010), “Characteristics of Physical

Measurement Consent in a Population-Based Survey of Older Adults,” Medical Care, 48(1),

64-71.

Sakshaug, J. W., M. P. Couper, M. B. Ofstedal, and D. R. Weir (2012), “Linking Survey and

Administrative Records: Mechanisms of Consent,” Sociological Methods and Research,

41(4), 535-569.

Sala, E., J. Burton, and G. Knies (2012), “Correlates of Obtaining Informed Consent to Data

Linkage: Respondent, Interview, and Interviewer Characteristics,” Sociological Methods and

Research, 41(3), 414-439.

Sala, E., G. Knies, and J. Burton (2014), “Propensity to Consent to Data Linkage: Experimental

Evidence on the Role of Three Survey Design Features in a UK Longitudinal Panel,”

International Journal of Social Research Methodology, 17(5), 455-473.

Schuster, M. A., R. M. Bell, S. H. Berry, and D. E. Kanouse (1998), “Impact of a High School

Condom Availability Program on Sexual Attitudes and Behaviors,” Family Planning

Perspectives, 30(2), 67-72.

Severson, H. H., D. V. Ary (1983), “Sampling Bias Due to Consent Procedures with

Adolescents,” Addictive Behaviors, 8, 433-437.

Singer, E. (1978), “Informed Consent: Consequences for Response Rate and Response Quality in

Social Surveys,” American Sociological Review, 43(2), 144-162.

Singer, E. (2003), “Exploring the Meaning of Consent: Participation in Research and Beliefs

about Risks and Benefits,” Journal of Official Statistics, 19(3), 273-285.

Skender, S., P. Schrotz-King, J. Böhm, C. Abbenhardt, B. Gigic, J. Chang-Claude, E. M. Siegel,

K. Steindorf, and C. M. Ulrich (2015), “Repeat Physical Activity Measurement by

Accelerometry among Colorectal Cancer Patients – Feasibility and Minimal Number of Days

of Monitoring,” BMC Research Notes, 8:222.

Spence, S., M. White, A. J. Adamson, J. N. S. Matthews (2014), “Does the Use of Passive or

Active Consent Affect Consent or Completion Rates, or Dietary Data Quality? Repeat Cross-

Sectional Survey Among School Children Aged 11-12 Years,” BMJ Open, 4:e006457.

Tenth Book of the German Social Code – Social Administrative Procedures and Social Privacy.

Section 75: Erhebung, Verarbeitung oder Nutzung von Sozialdaten im Auftrag. <

http://www.gesetze-im-internet.de/sgb_10/BJNR114690980.html>

The HIPAA Privacy Act. Title 45 Code of Federal Regulations 164.501.

U.S. Department of Health & Human Services. Title 45: Code of Federal Regulations: General

Requirements for Informed Consent 46.116.

U.S. Privacy Act of 1974, PL 93-579, December 31, 1974; 5 U.S.C.-552a.

vom Berge, P., A. Burghardt, and S. Trenkle (2013), “Sample-of-Integrated-Labour-Market-

Biographies Regional-File 1975-2010 (SIAB-R 7510),” FDZ-Datenreport, No. 9.

White, V. M., D. J. Hill, and Y. Effendi (2004), “How Does Active Parental Consent Influence

the Findings of Drug-Use Surveys in Schools?” Evaluation Review, 28(3), 246-260.

Yeager, D. S., J. A. Krosnick, L. Chang, H. S. Javitz, M. S. Levendusky, A. Simpser, and R.

Wang (2011), “Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys

Conducted with Probability and Non-Probability Samples,” Public Opinion Quarterly, 75(4),

709-747.