efficacy, effectiveness, and behavior change trials in exercise

DEBATE Open Access

Efficacy, effectiveness, and behavior change trialsin exercise researchKerry S Courneya

Abstract

Background: The widespread incorporation of behavioral support interventions into exercise trials has sometimescaused confusion concerning the primary purpose of a trial. The purpose of the present paper is to offer someconceptual and methodological distinctions among three types of exercise trials with a view towards improvingtheir design, conduct, reporting, and interpretation.

Discussion: Exercise trials can be divided into “health outcome trials” or “behavior change trials” based on theirprimary outcome. Health outcome trials can be further divided into efficacy and effectiveness trials based on theirpotential for dissemination into practice. Exercise efficacy trials may achieve high levels of exercise adherence bysupervising the exercise over a short intervention period ("traditional” exercise efficacy trials) or by the adoption ofan extensive behavioral support intervention designed to accommodate unsupervised exercise and/or an extendedintervention period ("contemporary” exercise efficacy trials). Exercise effectiveness trials may emanate from thedesire to test exercise interventions with proven efficacy ("traditional” exercise effectiveness trials) or the desire totest behavioral support interventions with proven feasibility ("contemporary” exercise effectiveness trials). Efficacy,effectiveness, and behavior change trials often differ in terms of their primary and secondary outcomes, theoreticalmodels adopted, selection of participants, nature of the exercise and comparison interventions, nature of thebehavioral support intervention, sample size calculation, and interpretation of trial results.

Summary: Exercise researchers are encouraged to clarify the primary purpose of their trial to facilitate its design,conduct, and interpretation.

Randomized controlled trials (RCTs) are a commondesign for testing many health interventions includingexercise. In recent years, exercise RCTs have becomemore complex and challenging because of a greaterfocus on methodological rigor and the desire to testlonger exercise interventions, unsupervised exerciseinterventions, and higher volume exercise interventions.These changes have necessitated a greater focus onexercise adherence in these trials and have stimulatedthe incorporation of behavioral support interventionsinto many exercise trials for which behavior change wasnot the primary purpose. Behavioral support interven-tions consist of strategies to improve exercise adherencesuch as incentives, print materials, telephone counsel-ling, group sessions, websites, and other behavior modi-fication techniques. The incorporation of behavioral

support interventions into exercise RCTs has been amajor methodological advance in exercise research,however, it has sometimes caused confusion concerningthe primary purpose of a trial. Was the trial designed toimprove a health outcome or to understand exercisebehavior change? Was the trial designed to demonstrateefficacy or effectiveness? This confusion may lead toimproperly designed trials and/or the misinterpretationof trial results. The purpose of the present paper is tooffer some conceptual and methodological distinctionsamong different types of exercise trials with a viewtowards improving their design, conduct, reporting, andinterpretation.

Health Outcome Trials Versus BehaviorChange TrialsIn simplest terms, exercise RCTs can be divided into“health outcome trials” or “behavior change trials”(Figure 1). Health outcome trials are those trials in

Correspondence: [email protected] of Physical Education and Recreation, University of Alberta,Edmonton, Alberta, Canada

Courneya International Journal of Behavioral Nutrition and Physical Activity 2010, 7:81http://www.ijbnpa.org/content/7/1/81

© 2010 Courneya; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction inany medium, provided the original work is properly cited.

mailto:[email protected]

http://creativecommons.org/licenses/by/2.0

which the primary purpose is to examine the effects ofan exercise intervention on some health outcome suchas cardiorespiratory fitness, body composition, psychoso-cial functioning, biomarkers, or disease states. Behaviorchange trials, on the other hand, are those trials inwhich the primary purpose is to examine the effects of abehavioral support intervention on some aspect of exer-cise behavior itself such as the type, volume, intensity,or maintenance of exercise. In the past, these two typesof exercise trials largely led separate lives and wereviewed as independent but related steps towardsimproving health through exercise. In recent years, how-ever, several major changes in the exercise field havebrought these two types of trials together.

The Rise of Behavioral Support Interventions inHealth Outcome TrialsSeveral key changes have occurred in the exercise fieldover the past two decades that have prompted the inclu-sion of behavioral support interventions into manyhealth outcome trials: (a) the adoption of the Consoli-dated Standards of Reporting Trials (CONSORT) [1],(b) longer exercise intervention trials, (c) unsupervisedexercise intervention trials, and (d) higher volume exer-cise intervention trials. Prior to the CONSORT guide-lines [1], exercise health outcome trials rarelyincorporated behavioral support interventions into theirmethodology, probably because of the exclusively super-vised exercise protocols, the relatively short exerciseinterventions (e.g., 12-24 weeks), and methodologicalpractices that were accepted at the time. In pre-CONSORT exercise health outcome trials, participantswho did not adhere to the exercise intervention wereoften removed from the analyses and/or not reported inthe trial. The solution to nonadherence in these trialswas to recruit more participants until the desired samplesize of “adherers” was achieved. The acceptability of thispractice was brought into question with the adoption ofthe CONSORT guidelines [1] by most medical, health,

and behavioral journals, including one of the leadingexercise science journals, Medicine & Science in Sports& Exercise. Among its many recommendations forimproving the quality of RCTs, the CONSORT guide-lines stated that all trial participants must be analyzedaccording to their assigned condition regardless of theirlevel of adherence to the trial protocol. Consequently,nonadherence became a potentially important issue evenin health outcome trials with supervised exercise proto-cols and shorter length interventions.At the same time, researchers began conducting

longer exercise interventions (e.g., 1-2 years) to deter-mine the sustained effects of exercise over time andalso to examine more distal effects of exercise such asdisease endpoints. The expansion of health outcometrials to one year, two years, and even longer providedfurther impetus for the inclusion of behavioral supportinterventions as it was obvious that sustained exerciseadherence would be a challenge even with motivatedparticipants and supervised exercise protocols. More-over, other researchers began testing unsupervised exer-cise interventions with the goal of improving thepotential for translation of exercise research into prac-tice. In these trials, it was also clear that adherence tounsupervised exercise would require some level of beha-vioral support, even for shorter length interventions.Finally, researchers became interested in testing highervolumes of exercise that were hypothesized to be neces-sary for addressing some of the more intractable healthoutcomes (e.g., obesity, disease endpoints). Again, it wasclear that the testing of these higher exercise volumes(e.g., 150-300 minutes per week) would require signifi-cant behavioral support. These four factors-adoption ofthe CONSORT guidelines [1], longer exercise interven-tions, unsupervised exercise interventions, and highervolume exercise interventions-prompted the incorpora-tion of behavioral support interventions into manyhealth outcome trials for which behavior change wasnot the primary purpose.

Figure 1 Schematic representation of three types of exercise trials.


of 12

Efficacy (Explanatory) Versus Effectiveness(Pragmatic) TrialsComplicating the matter further is the growing impor-tance of the distinction between efficacy and effective-ness trials [2]. Efficacy trials (also called explanatorytrials) determine whether an intervention produces theintended effect under ideal circumstances whereas effec-tiveness trials (also called pragmatic trials) determinewhether an intervention produces the intended effectunder “real-world” conditions [3]. Most RCTs, includingexercise, contain a mix of efficacy and effectiveness ele-ments [4]. In this sense, efficacy and effectiveness trialsexist along a continuum and their designation is amatter of degree. Moreover, there are many individualfeatures of an RCT and each can vary along the effi-cacy-effectiveness continuum; not just the overall RCT[2,5-8]. One of the key differences between efficacy andeffectiveness trials from this paper’s perspective is theanticipated level of adherence to the intervention.Reflecting this key issue, efficacy refers to whether anintervention works in people who receive it (i.e., whoadhered to the protocol) whereas effectiveness refers towhether an intervention works in people to whom it hasbeen offered (i.e., it assumes at least some level of non-adherence) [4]. The incorporation of behavioral supportinterventions into many exercise health outcome trialshas further clouded the complex distinction betweenefficacy and effectiveness trials in exercise research.

Traditional and Contemporary Efficacy Trials inExercise ResearchIn exercise research, a distinction might be madebetween “traditional” exercise efficacy trials and “con-temporary” exercise efficacy trials. In traditional exerciseefficacy trials, exercise adherence was typically assuredby a completely supervised exercise intervention testinga relatively modest exercise volume (e.g., 3 days perweek for 30 minutes) over a relatively short intervention(e.g., 12-24 weeks). In one sense, the supervised exerciseitself was the “behavioral support intervention” thatensured the high level of exercise adherence over theshort intervention period (i.e., optimal conditions). Assuch, there was relatively little need to incorporate anymodern behavioral support techniques into traditionalexercise efficacy trials. As noted earlier, the adoption ofthe CONSORT guidelines [1] may make behavioral sup-port interventions potentially important even for thesetraditional exercise efficacy trials.Contemporary exercise efficacy trials, on the other

hand, typically include one or more of the following fea-tures: (a) an unsupervised or partially supervised exer-cise intervention, (b) an extended exercise intervention(e.g., 1-2+ years), and/or (c) a relatively large volume ofexercise (e.g., 150-300 minutes of exercise per week).

The nature of these trials makes it unlikely that they areable achieve a high level of adherence sufficient to testthe efficacy of the exercise intervention without includ-ing some level of behavioral support. In contemporaryexercise efficacy trials, the goal of the behavioral supportintervention is to obtain a high level of exercise adher-ence over an extended period of time so that the pri-mary question concerning the particular health outcomecan be answered. The behavioral support intervention,in essence, is viewed as the primary mechanism forensuring a high level exercise adherence in the absenceof completely supervised exercise over a short interven-tion period. The level of behavioral support needed incontemporary exercise efficacy trials likely varies by thedegree of exercise supervision, the length of the exerciseintervention, and the volume of the exercise interven-tion (Figure 2).

Traditional and Contemporary Effectiveness Trialsin Exercise ResearchExercise efficacy trials and effectiveness trials are bothhealth outcome trials with the primary purpose of deter-mining the effects of an exercise intervention on ahealth outcome. Contemporary exercise efficacy trialsand effectiveness trials also both include modern beha-vioral support interventions to help answer the primaryhealth outcome question. Consequently, the distinguish-ing feature between contemporary exercise efficacy trialsand effectiveness trials appears to be the nature of thebehavioral support intervention. In contemporary exer-cise efficacy trials, researchers typically develop or adopta behavioral support intervention that is intensiveenough to achieve a high level of adherence to the exer-cise intervention over an extended period of time sothat the primary health outcome question can beanswered. Efficacy trials, by definition, are less con-cerned about whether the behavioral support interven-tion is feasible in practice. Conversely, an exerciseeffectiveness trial typically adopts a behavioral supportintervention that is consistent with practice or at leastdeemed potentially feasible to put into practice. Thegoal is then to determine whether the exercise interven-tion achieved by a feasible behavioral support interven-tion is sufficient for improving some important healthoutcome.There also appears to be two approaches to effective-

ness trials in exercise research. The “traditional”approach to an effectiveness trial is usually one that pro-ceeds from a successful efficacy trial demonstrating thebenefits of an exercise intervention under optimal con-ditions. The purpose of the traditional effectiveness trialis then to determine if the benefits of the exercise inter-vention are still evident under “real-world” conditions (i.e., in practice). The “contemporary” approach to an


of 12

effectiveness trial, on the other hand, is usually one thatproceeds from a successful behavior change trialdemonstrating that a feasible behavioral support inter-vention is able to increase exercise behavior to a poten-tially meaningful level. The purpose of thecontemporary effectiveness trial is then to determine ifthe exercise intervention achieved by the feasible beha-vioral support intervention is sufficient for improvingsome health outcome in practice.

Efficacy, Effectiveness, and Behavior ChangeTrials in Exercise ResearchAs noted earlier, the widespread adoption of behavioralsupport interventions into many exercise RCTs hassometimes caused confusion concerning the primaryintent of the trial. At the simplest level, it is sometimesunclear if the intent was to conduct a health outcometrial with behavioral support or a behavior change trialwith secondary health outcomes. At a more complexlevel, it is sometimes unclear if the intent was to con-duct an efficacy trial but an inadequate level of beha-vioral support resulted in modest adherence and gavethe impression of an effectiveness trial. In other cases, itis unclear if the intent was to conduct an effectivenesstrial but an inadequate level of behavioral supportresulted in no improvements in health outcomes andgave the impression of a behavior change trial. Still, inother cases, it is unclear if the intent was to conduct abehavior change trial but improvements in some healthoutcomes gave the impression of an effectiveness trial.In the following sections, some potentially distinguishingfeatures among efficacy, effectiveness, and behaviorchange trials are discussed under the headings of selec-tion of primary and secondary outcomes, adoption oftheoretical models, selection of participants, nature ofthe exercise intervention, nature of the behavioral sup-port intervention, nature of the comparison interven-tion, sample size calculation and analysis, andinterpretation of trial results (Table 1).

Selection of Primary and Secondary OutcomesPerhaps the most obvious and important distinctionbetween health outcome trials and behavior changetrials is the identification of the primary outcome. Inefficacy and effectiveness trials, the primary outcome isa health outcome (e.g., aerobic fitness, biomarkers, qual-ity of life, disease outcomes). In behavior change trials,the primary outcome is an aspect of exercise behavior(e.g., type, dose, attendance, number of steps). Althoughthis distinction may appear simple and straight forward,confusion can arise in exercise trials when both behaviorand health variables are identified as outcomes, andespecially when both are identified as primary outcomes.Behavior change does not appear to be an appropriate

primary or secondary outcome in efficacy or effective-ness trials because outcomes in an RCT are defined asevents that are present after the participants receive theintervention [4]. Given that exercise is the actual inter-vention in a health outcome trial, adherence to theintervention itself cannot be an outcome of the inter-vention. Failure to exercise in a health outcome trial isnot a negative effect on a primary or secondary out-come, it is a protocol deviation. The fact that the exer-cise may be unsupervised does not make it an outcomein an efficacy or effectiveness trial.Conversely, it does seem acceptable, and even meritor-

ious, to identify health outcomes as secondary outcomesin behavior change trials because these are potentialoutcomes that may follow from the behavioral supportintervention and the ensuing exercise behavior change.If improved, these health outcomes may indicate a parti-cularly meaningful level of behavior change. In this con-text, the goal is to determine whether the exercisebehavior that results from such a behavioral supportintervention is sufficient for improving some importanthealth outcome. For the same reasons as above, how-ever, it does not seem appropriate to identify adherenceto the behavioral support intervention (e.g., telephonecalls completed, attendance at group sessions, visits tothe website) as a primary or secondary outcome in abehavior change trial. Failure to complete the behavioralsupport intervention in a behavior change trial is a pro-tocol deviation.

Adoption of Theoretical ModelsAnother important distinction among efficacy, effective-ness, and behavior change trials is the nature of the the-ories that inform the research. In simplest terms, thereare two types of theories or models: theories thatexplain the outcomes of a behavior and theories thatexplain the determinants of a behavior (see Figure 1).Outcome theories attempt to explain how the exerciseintervention (e.g., type, dose, intensity, frequency)improves a health outcome. These theories or modelsoften include biological or physiological variables butmight also include health-related fitness or psychosocialvariables depending on the nature of the health out-come. Behavioral theories, on the other hand, attemptto explain how a behavioral support intervention (e.g.,goal setting, physician recommendation, telephonecounselling) might improve exercise behavior. Thesetheories often focus on social cognitive variables such asattitude, self-efficacy, and social support but might alsoinclude a wider range of variables such as physiologicaland environmental factors.In efficacy trials, researchers should select an outcome

theory that will explain how the exercise intervention isthought to change the health outcome. Traditional


of 12

efficacy trials involving minimal behavioral supportwould not normally need to adopt a behavioral theory.Contemporary efficacy trials, however, would likely ben-efit from following a behavioral theory because of thesubstantial behavioral support intervention that is oftenincluded and the greater risk of nonadherence. In effec-tiveness trials, it would seem that researchers shouldadopt both an outcome theory to guide the exerciseintervention and a behavioral theory to guide the beha-vioral support intervention. In behavior change trials,researchers typically need only to adopt a behavioraltheory. Confusion can arise in exercise trials when thereis a “mismatch” between the type of theory and thereported outcomes. That is, exercise trials should notcite behavioral theories to claim they are “theory-based”when reporting health outcomes and, conversely, theyshould not cite outcome theories to claim they are “the-ory-based” if they report on the determinants of exerciseadherence.

Selection of ParticipantsThe selection of participants may also vary across effi-cacy, effectiveness, and behavior change trials. In effi-cacy trials, participants are typically selected based ontheir motivation and ability to complete and respond

to the particular intervention [6,8]. In exercise efficacytrials, this usually means selecting participants not cur-rently performing the particular exercise interventionyet capable of performing the exercise intervention. Ineffectiveness trials, participants are only moderatelyselected based on their motivation and ability becauseof the desire to reflect “real-world” conditions thatbroaden eligibility criteria [6,8]. In exercise effective-ness trials, this may mean including participants cur-rently performing some or even all of the intendedexercise intervention if this reflects the target group inclinical or public health practice. In behavior changetrials, participants may have various levels of motiva-tion and ability depending on the focus of the trial.Because the goal is not to improve a health outcomeper se, a behavior change trial might target peoplewith no interest in exercise, some interest in exercise,or a strong interest in exercise. Moreover, itmay include participants currently performing no exer-cise, some exercise, or already meeting the exerciseprescription with the goal of maintaining exercisebehavior. Given the differences in participant selection,the generalizability of findings is usually lower in effi-cacy trials and higher in effectiveness and behaviorchange trials [6].

Figure 2 Level of behavioral support needed for exercise efficacy trials based on the degree of exercise supervision and length of theexercise intervention.


of 12

Table 1 Summary of some distinguishing features among efficacy, effectiveness, and behavior change exercise trials

Efficacy Trial

Traditional Contemporary Effectiveness Trial Behavior Change Trial

Primary Outcome: A health outcome A health outcome A health outcome An exercise behavioroutcome

Secondary Outcomes: Other health outcomes Other health outcomes Other health outcomes Other behavior/healthoutcomes

Protocol Deviations: Not exercising Not exercising/behavioralsupport

Not exercising/behavioralsupport

Not completing behavioralsupport

Theoretical Model: Outcome model needed Outcome model needed Outcome model needed Outcome model not needed

Behavior model notneeded

Behavior model beneficial Behavior model needed Behavior model needed

Participants: Highly selected Highly selected Moderately selected Variably selected

Not exercising/limitedexercise

Not exercising/limitedexercise

Exercise levels reflective ofpractice

Variable exercise levels

Lower generalizability Lower generalizability Higher generalizability Higher generalizability

Exercise Intervention: Completely supervised Completely/partiallysupervised

Unsupervised/partiallysupervised

NA (exercise is the outcome)

Precise exercise type/volume

Partially flexible types/volumes

Flexible exercise types/volumes Flexible exercise types/volumes

Agree to exercise Agree to exercise Agree to exercise Asked to exercise

No/limited behavioralsupport

Agree to behavioralsupport

Agree to behavioral support Agree to behavioral support

Behavioral SupportIntervention:

Minimal/informal Extensive/comprehensive Moderate/feasible fordissemination

Varied but systematic andfeasible

Comparison Intervention: No exercise intervention Minimal exerciseintervention

Standard of care At least exerciserecommendation

Agree not to exercise Allowed to exercise Allowed to exercise Asked to exercise

No behavioral support No/limited behavioralsupport

Behavioral support as perstandard care

Behavior support based onpurpose

Sample Size: Powered for healthoutcome

Powered for healthoutcome

Powered for health outcome Powered for behavioraloutcome

No allowance fornonadherence

Allowance fornonadherence

Allowance for nonadherence No allowance fornonadherence

Analysis: Intention-to-treat Intention-to-treat Intention-to-treat Intention-to-treat

Prespecified subgroupanalyses

Prespecified subgroupanalyses

Prespecified subgroup analyses Prespecified subgroupanalyses

Interpretation of Results:

Positive trial Improves primary healthoutcome

Improves primary healthoutcome

Improves primary healthoutcome

Improve primary behavioroutcome

Adequate exerciseadherence


Adequate exercise adherence Adequate support adherence

Support adherence notrequired


Negative trial No change in healthoutcome

No change in healthoutcome

No change in health outcome No change in behavioraloutcome


of 12

Moreover, participants in exercise RCTs should gener-ally be asked to agree to the trial protocol prior to ran-domization. In traditional efficacy trials, the trialprotocol includes supervised exercise. In contemporaryefficacy and effectiveness trials, the trial protocol mightinclude supervised and unsupervised exercise as well asthe behavioral support intervention. Consequently, parti-cipants in health outcome trials should be asked toagree to do (or try to do) the exercise intervention aswell as the behavioral support intervention as a methodof supporting their exercise adherence. In behaviorchange trials, the trial protocol includes the behavioralsupport intervention but not the exercise itself becausethat is the outcome. Consequently, intervention partici-pants in a behavior change trial should be asked toagree to follow the behavioral support intervention butnot to agree to do the exercise because that is the pri-mary outcome of the trial.

Nature of the Exercise InterventionThe nature of the exercise intervention may also varyamong exercise trials [8]. In efficacy trials, there is oftena precise exercise intervention for participants to followthat includes the type, frequency, intensity, duration,and progression of exercise. For example, participantsmay be asked to exercise on a cycle ergometer threetimes per week for 30 minutes at 70-85% of their maxi-mum capacity. Typically, there is limited flexibility inhow participants achieve the exercise intervention. Ineffectiveness trials, there is often a general exerciseintervention for participants to follow (e.g., a desired

volume) and there may be considerable flexibility inhow participants are allowed to achieve the exerciseintervention. For example, participants may be asked toexercise for 10 metabolic equivalent task (MET) hoursper week but be allowed to choose any type, frequency,intensity, or duration of exercise that meets this generalprescription. Alternatively, participants may receive pre-specified ranges of exercise from which to chose (e.g.,several types of aerobic exercise, moderate-to-vigorousexercise, 3-5 days/week, 20-60 minutes in duration).Moreover, effectiveness trials often establish only aminimum exercise goal, not a maximum goal (e.g., parti-cipants may be asked to perform at least 150 minutes ofexercise/week but be allowed to exceed this goal). Inbehavior change trials, there is no exercise interventionper se because exercise is the outcome. Nevertheless,there is often an exercise behavioral goal that partici-pants are asked to achieve as part of the behavioral sup-port intervention.

Nature of the Behavioral Support InterventionAnother major distinction among exercise trials is thenature of the behavior support intervention. As notedearlier, in traditional exercise efficacy trials, there is typi-cally no formalized behavioral support intervention.Exercise adherence is usually achieved by highly selectparticipants, the shorter length of the intervention, andthe supervised exercise that likely entails at least somepositive social support and accountability. In contem-porary efficacy trials involving unsupervised exercise oran extended intervention, the behavioral support

Table 1 Summary of some distinguishing features among efficacy, effectiveness, and behavior change exercise trials(Continued)



Adequate exercise adherence Adequate support adherence



Inconclusive trial No change in healthoutcome

No change in healthoutcome

No change in health outcome No change in behavioraloutcome

Inadequate exerciseadherence

Inadequate exerciseadherence

Inadequate exercise adherence Inadequate supportadherence

Nature of the Result:

Definitive Negative trial Negative trial Positive trial Negative trial

No effect in practice islikely

No effect in practice islikely

Effect in practice is likely No effect in practice is likely

Ambiguous Positive trial Positive trial Negative trial Positive trial

Effect in practice isunknown

Effect in practice isunknown

Effect under better conditionsunknown

Effect on health outcomesunknown

(unless shown in trial orliterature)


of 12

intervention is usually sophisticated and comprehensive,but it is typically a “kitchen sink” approach. That is,contemporary efficacy trials usually incorporate as manybehavioral support techniques and motivational strate-gies as necessary because behavior change is a means toan end and not an end in itself. The goal of these trialsis to determine if a particular exercise interventionimproves an important health outcome under optimalconditions, not to systematically disentangle what com-ponents of the behavioral support intervention might behelpful or to be concerned about the feasibility of theintervention. In efficacy trials, close monitoring of exer-cise adherence and behavioral support adherence isrequired during the trial to ensure corrective action istaken if adherence falters [8].In effectiveness trials, the behavioral support interven-

tion is usually one that reflects practice or at least hasthe potential for being translated into practice. Similarto efficacy trials, however, the goal of effectiveness trialsis to determine if a particular exercise interventionimproves some health outcome, not to systematicallydisentangle what components of the behavioral supportintervention worked. Any monitoring and correctiveaction for exercise adherence and/or behavioral supportadherence in effectiveness trials should be consistentwith the intended practice context [8]. Conversely, inbehavior change trials, the behavioral support interven-tion may be simple or sophisticated depending on thequestion; however, the focus is on systematically evalu-ating the individual or packaged behavioral supportintervention to determine its effects on exercise beha-vior. There is no ongoing monitoring of exercise beha-vior in behavior change trials because exercise behavioris the primary outcome. Monitoring and correctiveaction for the behavioral support intervention may beconsidered depending on the nature of the trial.

Nature of the Comparison InterventionThe comparison intervention also differs among effi-cacy, effectiveness, and behavior change trials. In tradi-tional efficacy trials, participants in the comparisongroup are often asked not to exercise or not to increasetheir exercise from baseline and to agree to theseinstructions prior to randomization. These instructionsare optimal for efficacy trials because contamination inthe comparison group is as problematic as nonadher-ence in the intervention group. In efficacy trials, thegoal is to determine if the exercise interventionimproves a health outcome under optimal conditions.The optimal condition includes 100% adherence in theintervention group and 0% contamination in the com-parison group. Asking comparison participants not toexercise is often accepted by research ethics boardsdepending on the purpose of the study, the length of

the intervention, and the risk to participants of notexercising.The comparison group intervention in contemporary

efficacy trials and effectiveness trials is much more com-plex for methodological and ethical reasons. Askingcomparison participants not to exercise or not toincrease their exercise may not be considered ethical ifit is for an extended period of time or if the populationis considered at-risk. It may not be scientifically justifi-able in effectiveness trials where the goal is to replicate“real-world” environments where comparison partici-pants are free to exercise if they want. This complexityhas led to a great deal of variability in what comparisonparticipants are asked to do in contemporary efficacyand effectiveness trials. In some trials, comparison parti-cipants are not asked anything but are given standardhealth materials that often include information on exer-cise (e.g., public health exercise guidelines). In othertrials, they are explicitly recommended to exercise butare not provided with any of the behavioral supportintervention or only a limited version of the intervention(e.g., printed materials only, one behavioral support ses-sion, a few telephone calls). Still, in other trials, they aregiven a more extensive health intervention that may beimportant for general health or for their specific diseasecondition but it may or may not include exercise. Thelack of clear exercise instructions to the comparisongroup in contemporary efficacy trials is probably anattempt to balance the ethical concern of having com-parison participants not exercise, with the scientific con-cern of maximizing differences in exercise behaviorbetween the intervention and comparison groups.In effectiveness trials, participants should receive the

current standard of care for exercise if one already existsin the particular practice setting [8]. If not, the lack ofclear exercise instructions to comparison participants isprobably the correct approach ethically and scientificallybecause the goal is to determine the effects of exercisein a “real world” setting in which comparison partici-pants are allowed to exercise on their own but may notreceive any support.In behavior change trials, the comparison group inter-

vention has also varied. It does not seem scientificallyjustifiable, however, to ask comparison participants in abehavior change trial not to exercise or not to increasetheir exercise; or even not to provide them with anyinstructions. The utility of a behavioral support inter-vention that is superior to a comparison group that wasasked not to exercise or that was not provided with anyinstructions to exercise seems dubious. Consequently,except in cases where the recommendation itself is thebehavior support intervention under investigation (e.g.,physician recommendation, oncologist recommenda-tion), it would seem that comparison groups in behavior


of 12

change trials, at a minimum, should be recommendedthe same level of exercise as the intervention group.Any additional behavioral support provided to the com-parison group in a behavior change trial (including anycontact control) would depend on the nature and pur-pose of the trial and what the researchers wanted to beable to say about the intervention.

Sample Size Calculation and AnalysisIn traditional efficacy trials, the trial is powered for theprimary health outcome and, given that nonadherenceor contamination are not anticipated, they are typicallynot factored into the power calculation. Allowance maybe made for loss-to-follow-up, but this is a differentissue. In contemporary efficacy and effectiveness trials,the trial is also powered for the primary health outcome,however, an a priori allowance for some prespecifiedlevel of nonadherence and contamination may be fac-tored into the power calculation. In behavior changetrials, the trial should be powered for the primary beha-vioral outcome and typically no allowance is made forany nonadherence or contamination with the behavioralsupport intervention. The primary analysis in all trialsshould follow the intention-to-treat principle [1]. Anysecondary analysis that deviate from intention-to-treatshould be prespecified [1].

Interpretation of Trial ResultsAccurate interpretation of trial results is important butnot always provided [9]. In an efficacy trial, failure toimprove the primary health outcome in the face of ade-quate exercise adherence (however defined) is inter-preted as a negative trial (i.e., the exercise interventiondoes not work for that health outcome). Improvementsin important secondary health outcomes may be note-worthy but do not change the overall interpretation ofthe trial. Poor exercise adherence in an efficacy trial is amajor protocol deviation. Failure to improve the primaryhealth outcome in the face of inadequate exercise adher-ence in an efficacy trial results in an inconclusive trial.An inconclusive trial is one in which the primary ques-tion could not be answered because of some substantialmethodological problem such as poor protocol adher-ence. Researchers conducting efficacy trials should resistthe temptation of interpreting an inconclusive efficacytrial as a negative effectiveness trial or a positive beha-vior change trial.In an effectiveness trial, interpretation of the results

is the same as for an efficacy trial. In addition, pooradherence to the behavioral support intervention in aneffectiveness trial or contemporary efficacy trial is alsoa protocol deviation. Nevertheless, even if poor adher-ence to the behavioral support intervention was thelikely cause of the poor exercise adherence, it is the

lack of exercise adherence that renders the trial incon-clusive, not the poor adherence to the behavioral sup-port intervention. Researchers conductingcontemporary efficacy trials or effectiveness trialsshould resist the temptation of interpreting a negativeor inconclusive health outcome trial as a positive beha-vior change trial.In a behavior change trial, failure to improve the pri-

mary behavioral outcome in the face of adequate adher-ence to the behavioral support intervention (howeverdefined) is interpreted as a negative trial (i.e., the beha-vioral support intervention does not work). Even if sec-ondary health outcomes improve, the trial would still beconsidered negative as these outcomes are unlikely to beattributable to the exercise behavior change itself(assuming accurate measurement of exercise) and, inany case, were not the primary outcome [9]. Conversely,a positive change in the primary behavioral outcome isinterpreted as a positive trial even if secondary healthoutcomes do not improve. Certainly, improvements insecondary health outcomes would further substantiatethe importance of the behavior change achieved in thetrial, but they were not the primary focus. In a behaviorchange trial, failure to improve the primary behavioraloutcome in the face of inadequate adherence to thebehavioral support intervention is a major protocoldeviation that results in an inconclusive trial. Research-ers conducting behavior change trials should resist thetemptation of interpreting a positive, negative, or incon-clusive behavior change trial that improved secondaryhealth outcomes as a positive effectiveness trial.The types of exercise RCTs also vary in regard to the

nature of the definitive result that each trial produces(Figure 3) [7]. For efficacy trials, the definitive result is anegative trial. In that case, it is certain that the exerciseintervention has no effect on the health outcome. Apositive efficacy trial is an ambiguous finding because itis unknown if the health outcome can be achieved inpractice. The converse is true for effectiveness trials [7].A positive effectiveness trial is the definitive resultbecause the trial demonstrates that the health outcomecan be improved under “real-world” conditions althoughadditional steps toward dissemination may be required[2]. A negative effectiveness trial may be ambiguousbecause it is unclear if a positive effect on the healthoutcome could have been achieved under more favor-able conditions [7]. Finally, a negative behavior changetrial is a definitive result because it is clear that thebehavioral support intervention would not improve anyhealth outcomes in practice. A positive behavior changetrial may be ambiguous because it is unclear if it willactually improve health outcomes in practice (unlessimportant secondary health outcomes were alsoimproved in the trial or there is compelling evidence of


of 12

the effect of such behavior change from previousstudies).

DiscussionThe purpose of this paper was to offer some conceptualand methodological distinctions between health out-come trials (efficacy and effectiveness) and behaviorchange trials. It was argued that these trials differ interms of many important attributes and design issues.Several issues raised in this paper warrant furtherdiscussion.One important issue is whether it is possible to con-

duct both a behavior change trial and a health outcometrial as part of the same trial. The difficulty with thisapproach is that it requires multiple primary outcomes,which is not recommended by CONSORT [1]. Multipleprimary outcomes may lead to internally inconsistentdecisions because the selection of a primary outcomedrives all other methodological decisions in the trial[1,7,8]. For example, an exercise RCT may select exer-cise behavior, weight loss, and depression as its primaryoutcomes. In this trial, would the eligibility criteriainclude only nonexercisers who are obese anddepressed? Or would it allow current exercisers who areobese and depressed, or obese people who are not

depressed and depressed people who are not obese?What exercise intervention would be selected? Perhapsa very high volume of exercise might be best for weightloss but not for depression. Perhaps yoga would begreat for depression but not for weight loss. What aboutthe selection of gold standard measures given limitedresources? Should the researchers spend significantmoney on the gold standard measures of exercise beha-vior (e.g., direct observation, accelerometers) or obesity(e.g., DEXA scans, CT scans) or depression (e.g., clinicalinterviews)? Will the sample size be determined byexpected changes in exercise, obesity, or depression?What will the comparison group receive? Should thecomparison group be asked not to exercise? How willthe intervention be interpreted? Will it be successful ifpeople exercise but are still obese and depressed? Is itsuccessful if they lose weight but are still depressed, orare less depressed but still obese? Trial design issues areoften difficult to resolve when there are competing pri-mary outcomes.It seems reasonable for a behavior change trial to have

a secondary purpose of documenting changes in healthoutcomes but the trial would still have been conceptua-lized and designed as a behavior change trial with allthe important trial decisions based on this purpose. It

Figure 3 Nature of the trial result produced by three types of exercise trials.


of 12

does not seem reasonable, however, for a health out-come trial to have a secondary purpose of testing abehavior support intervention because the former isactually contingent on the latter. That is, the health out-come trial is unable to answer its primary question ifbehavior change does not occur (i.e., it becomes aninconclusive trial). If behavior change is uncertain in ahealth outcome trial, it seems the best approach wouldbe to conduct a feasibility study demonstrating thatbehavior change is highly likely before moving to thehealth outcome trial.Another interesting issue is whether it is possible to

make the same conceptual distinction between efficacyand effectiveness for behavior change trials as it is forhealth outcome trials. In the present paper, behaviorchange trials have been largely conceptualized as effec-tiveness trials by default. Perhaps it would have beenmore prudent to develop a 2 × 2 typology of trial out-come (behavior change versus health outcome) by trialattitude (efficacy versus effectiveness). In consideringthis option, it does not seem helpful to conduct efficacy-oriented behavior change trials because the primary pur-pose of such trials is usually to translate health outcomeresearch into practice. Moreover, it does not seem tomake sense for behavior change trials to only selecthighly motivated participants or to use a behavioral sup-port intervention that is not feasible for practice whenthe goal is to disseminate the exercise research to asmany people as possible in the target population. Itwould seem that the only reason for incorporating abehavioral support intervention that is not feasible inpractice is because the goal of the trial is to establishthe efficacy of the health intervention. Having said that,the utility of distinguishing between efficacy and effec-tiveness behavior change trials may warrant furtherdiscussion.It is also critical to note that what is feasible to be

put into practice varies dramatically from context tocontext [7,8]. For example, supervised exercise proto-cols delivered by highly qualified personnel may indeedbe readily transferable into practice in certain clinicalsettings with dedicated space, sophisticated equipment,and highly qualified personnel (e.g., cardiac rehabilita-tion unit). Similarly, supervised exercise protocols mayeven be feasible to deliver in many community-basedsettings such as public and private fitness and recrea-tional centers that possess the necessary equipmentand qualified staff. Conversely, supervised exercise pro-tocols may not be readily transferable into practice inrural communities or in public health settings becauseof the lack of facilities, personnel, and limited funding.The key point is that what is considered feasible to betranslated into exercise practice varies with the prac-tice setting. Consequently, exercise researchers

conducting effectiveness trials should make explicit theintended practice setting for their intervention anddesign their trial accordingly [8].Another important issue is the order of conducting

the different types of exercise RCTs. As noted earlier, itis generally argued that efficacy trials should precedeeffectiveness trials [4]. That is, once the efficacy of anintervention is proven, then it needs to be tested for itseffectiveness. In exercise research, efficacy trials followedby an attempt to put the positive results into practicewithout the demonstration of effectiveness may fail insettings that cannot deliver supervised exercise interven-tions or provide extensive behavioral support interven-tions (e.g., rural, public health). One might reasonablyargue that, in practice settings where supervised exerciseor extensive behavioral support interventions are notlikely feasible, it may be more productive to begin withcontemporary effectiveness trials that test exercise inter-ventions that are feasible even though they may nothave proven efficacy. Thus, in exercise research, effec-tiveness trials might also emanate from positive behaviorchange trials with proven feasibility in a given practicesetting.This paper provides a first attempt to highlight some

conceptual and methodological issues that have arisenwith the incorporation of modern behavioral supportinterventions into many exercise trials. It was a difficultpaper to write because of the complexity of this topicand the diversity of exercise research. There are signifi-cant limitations in this paper. One limitation is that thepaper does not attempt to integrate the distinctionbetween health outcome trials and behavior changetrials into other existing research typologies beyond thecommon efficacy-effectiveness distinction (e.g., Phase I-IV trials, translational trials, dissemination trials, com-parative effectiveness trials, research phases, etc.). A sec-ond limitation is that every exercise trial is unique andsome of the general observations made in this papermay not apply to individual exercise trials. A third lim-itation is that there does not appear to be a clear dis-tinction between efficacy and effectiveness trials becauseit is the individual elements of a trial that vary along theefficacy-effectiveness continuum and researchers mightintentionally, and appropriately, mix efficacy and effec-tiveness attributes [8]. Another limitation is that thepaper does not address research designs beyond the tra-ditional RCT. Finally, it is unclear if the issues raised inthis paper apply to other health behavior interventions(e.g., nutrition) or to interventions that target multiplehealth behaviors.In summary, exercise RCTs are growing in sophistica-

tion and complexity because of the desire to test longerexercise interventions, unsupervised exercise interven-tions, and higher volume exercise interventions. These


of 12

complexities have led to the incorporation of behavioralsupport interventions into many exercise RCTs and hassometimes confused the primary purpose of the trial.Researchers conducting exercise RCTs are encouragedto consider the nature of their trial in terms of its healthoutcome versus behavior change focus and its positionon the efficacy versus effectiveness continuum. Clarityin the goals of the trial may improve the trial methodsand result in better quality trials more likely to informclinical, community, and public health exerciseinterventions.

AcknowledgementsKerry S. Courneya is supported by the Canada Research Chairs Program. Iwould like to thank Christine M. Friedenreich, PhD, Edward McAuley, PhD,Ronald C. Plotnikoff, PhD, and Laura Q. Rogers, MD for their review andfeedback on an earlier version of this manuscript. This acknowledgementdoes not indicate their endorsement of the arguments in this paper.

Competing interestsThe authors declare that they have no competing interests.

Received: 18 May 2010 Accepted: 12 November 2010Published: 12 November 2010

References1. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ,

Elbourne D, Egger M, Altman DG: CONSORT 2010 Explanation andElaboration: updated guidelines for reporting parallel group randomisedtrials. BMJ 2010, 340:c869.

2. Flay BR, Biglan A, Boruch RF, Castro FG, Gottfredson D, Kellam S,Moscicki EK, Schinke S, Valentine JC, Jil P: Standards of Evidence: Criteriafor Efficacy, Effectiveness and Dissemination. Prevention Science 2005,6:151-175.

3. Cochrane AL: Effectiveness and efficiency: random reflection on healthservices. London: Nuffield Provincial Hospitals Trust; 1972, 423.

4. Jadad AR, Enkin MW: Randomized controlled trials: Questions, answers, andmusings. 2 edition. Oxford, UK: Blackwell Publishing; 2007, 311.

5. Alford L: On differences between explanatory and pragmatic clinicaltrials. NZ Journal of Physiotherapy 2007, 35:12-16.

6. Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS: Criteria fordistinguishing effectiveness from efficacy trials in systematic reviews. InTechnical Review 12 (Prepared by the RTI-International-University of NorthCarolina Evidence-based Practice Center under Contract No. 290-02-0016.)AHRQ Publication No. 06-0046. Volume 227. Rockville, MD: Agency forHealthcare Research and Quality; 2006.

7. Karanicolas PJ, Montori VM, Devereaux PJ, Schunemann H, Guyatt GH: Anew ‘’Mechanistic-Practical’’ Framework for designing and interpretingrandomized trials. J Clin Epidemiol 2009, 62:479-484.

8. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG,Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatorycontinuum indicator summary (PRECIS): a tool to help trial designers. JClin Epidemiol 2009, 62:464-475.

9. Boutron I, Dutton S, Ravaud P, Altman DG: Reporting and Interpretation ofRandomized Controlled Trials With Statistically Nonsignificant Results forPrimary Outcomes. JAMA 2010, 303:2058-2064.

doi:10.1186/1479-5868-7-81Cite this article as: Courneya: Efficacy, effectiveness, and behaviorchange trials in exercise research. International Journal of BehavioralNutrition and Physical Activity 2010 7:81.

Submit your next manuscript to BioMed Centraland take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at www.biomedcentral.com/submit


of 12

http://www.ncbi.nlm.nih.gov/pubmed/20332511?dopt=Abstract













efficacy, effectiveness, and behavior change trials in exercise

Documents