are mass-media campaigns effective in preventing drug use? a … · 13 change. secondly, mass-media...

For peer review only

Are Mass-media Campaigns Effective in Preventing Drug Use? A Cochrane Systematic Review and Meta-analysis

Journal: BMJ Open

Manuscript ID: bmjopen-2014-007449

Article Type: Research

Date Submitted by the Author: 15-Dec-2014

Complete List of Authors: Allara, Elias; Università del Piemonte Orientale, Department of Translational Medicine; University of Torino, School of Public Health Ferri, Marica; European Monitoring Centre for Drugs and Drug Addiction, Consequences, Responses and Best Practices unit Bo, Alessandra; European Monitoring Centre for Drugs and Drug Addiction, Consequences, Responses and Best Practices unit Gasparrini, Antonio; London School of Hygiene & Tropical Medicine, Faggiano, Fabrizio; Università del Piemonte Orientale, Department of Translational Medicine

<b>Primary Subject Heading</b>:

Public health

Secondary Subject Heading: Addiction

Keywords: PUBLIC HEALTH, Substance misuse < PSYCHIATRY, Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT, MENTAL HEALTH

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on M

ay 14, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2014-007449 on 3 Septem

ber 2015. Dow

nloaded from

http://bmjopen.bmj.com/


1

Are Mass-media Campaigns Effective in Preventing Drug Use? A 1

Cochrane Systematic Review and Meta-analysis 2

3

Authors and affiliations: 4

Elias Allara (1,2) *, Marica Ferri (3), Alessandra Bo (3), Antonio Gasparrini (4), Fabrizio 5

Faggiano (1) 6

7

(1) Department of Translational Medicine, Università del Piemonte Orientale, Novara, Italy 8

(2) School of Public Health, University of Torino, Torino, Italy 9

(3) Consequences, Responses and Best Practices unit, European Monitoring Centre for Drugs 10

and Drug Addiction, Lisbon, Portugal 11

(4) London School of Hygiene & Tropical Medicine, London, UK 12

13

* Contact person: 14

Elias Allara 15

Email: [email protected]. Telephone: +39 348 5922753 16

Department of Translational Medicine, Università del Piemonte Orientale 17

Via Solaroli 17, 28100 Novara, Italy 18

19

Manuscript word court: 2654 20

Abstract word court: 228 21

Last update: 15 December 2014 22

23

Page 1 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on May 14, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2014-007449 on 3 S

eptember 2015. D

ownloaded from



2

Note: This article is based on a Cochrane Review published in the Cochrane Database of 1

Systematic Reviews (CDSR) 2013, Issue 6, DOI: .1002/14651858.CD009287.pub2 (see 2

www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as 3

new evidence emerges and in response to feedback, and the CDSR should be consulted for 4

the most recent version of the review.5

Page 2 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



3

Abstract 1

Objective To determine whether there is evidence that mass-media campaigns can be 2

effective in reducing illicit drug consumption and the intent to consume. 3

Design Systematic review of randomized and non-randomized studies. 4

Methods We searched 4 electronic databases (MEDLINE, EMBASE, ProQuest Dissertations & 5

Theses A&I, and CENTRAL) and further explored 7 additional resources to obtain both 6

published and unpublished materials. We appraised the quality of included studies using 7

standardized tools. We carried out meta-analyses of randomized controlled trials and a 8

pooled analysis of interrupted time-series and controlled before-and-after studies. 9

Results We identified 19 studies comprising 184,811 participants. Pooled analyses and 10

narrative synthesis provided mixed evidence of effectiveness. Eight interventions evaluated 11

with randomized controlled trials leaned toward no evidence of an effect, both on drug use 12

(standardized mean difference [SMD] -0.02; 95% confidence interval [CI] -0.15 to 0.12) and 13

the intention to use drugs (SMD -0.07; 95% CI -0.19 to 0.04). Four campaigns provided some 14

evidence of beneficial effects in preventing drug use and two interventions provided evidence 15

of iatrogenic effects. 16

Conclusions Studies were considerably heterogeneous in type of mass-media intervention, 17

outcome measures, underlying theory, comparison groups, and design. Such factors can 18

contribute to explaining the observed variability in results. Due to the risk of adverse effects, 19

caution is needed in disseminating mass-media campaigns tackling drug use. Large studies 20

conducted with appropriate methodology are warranted to consolidate the evidence base.21

Page 3 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



4

Strengths and limitations of this study 1

• This systematic review is based on an expanded evidence base of both published and 2

unpublished findings and aims to determine whether mass-media campaigns can be 3

effective in preventing the use of or intention to use illicit drugs. 4

• Pooled analyses of eight mass-media interventions provide no evidence of an effect on 5

drug use or intention to use illicit drugs. Four interventions provide evidence of 6

beneficial effects. Two interventions provide evidence of iatrogenic effects. 7

• Due to the paucity and inconsistency of available evidence we cannot draw general 8

conclusions as to whether mass-media interventions are effective in preventing the 9

use of or intention to use illicit drugs. 10

• This review provides an insight into research gaps around the impact of mass-media 11

drug prevention interventions and can serve to highlight that new campaigns should 12

be implemented in the framework of rigorous evaluation studies, in order to avoid 13

dissemination of interventions that are ineffective or have unintended effects. 14

15

Page 4 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



5

Introduction 1

Mass-media campaigns are a powerful means for disseminating health promotion messages. 2

A wide and diverse audience can be reached through television commercials, the Internet, 3

mobile phones, newspapers, and roadside advertising hoardings. In the field of drug addiction 4

and dependence, adverts may contribute to shaping patterns of drug use and the intention to 5

use drugs, as well as modifying mediators such as awareness, knowledge, and attitudes about 6

drugs. 7

8

However, ethical and economic considerations are often raised. Mass-media campaigns—9

unlike other health interventions—are imposed on populations that have not consented to 10

their implementation.[1] This is a considerable ethical issue in modern, person-centred public 11

health, where taking decisions shared with the public is essential for promoting behaviour 12

change. Secondly, mass-media campaigns can be very expensive, especially when 13

implemented at national or state level. Large-scale purchasing of public service 14

announcement time during popular shows and broad dissemination via printed media is often 15

accessible only to governmental institutions. For example, the first and second versions of the 16

U.S. Office of National Drug Control Policy’s National Youth Anti-Drug Media Campaign cost 17

2.7 billion dollars over more than 10 years.[2] Although such campaigns underwent careful 18

evaluation, most mass-media interventions are not developed in compliance with the classical 19

circle of public health, which consists in designing interventions based on evidence and in 20

evaluating their impact. 21

22

Page 5 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



6

A systematic review of the studies assessing media campaigns aiming to prevent use of illicit 1

drugs can inform future strategies and help design effective campaigns. The objective of this 2

review is to assess the effectiveness of mass-media campaigns in preventing or reducing drug 3

use or the intention to use illicit drugs among young people. 4

5

6

Methods 7

We conducted this systematic review in accordance with the Preferred Reporting Items for 8

Systematic Reviews and Meta-Analyses (PRISMA)[3] statement and with the procedures 9

specified in a previously published protocol.[4] As described in detail previously,[5] we 10

systematically searched 4 electronic databases: MEDLINE (1966 to 29 January 29 2013), 11

EMBASE (1974 to 30 January 2013), ProQuest Dissertations & Theses A&I (1861 to 3 February 12

2013), and CENTRAL (2013, Issue 1). Search strategies are available as supplementary files 13

(Appendix 1). We further explored 7 additional resources to obtain both published and 14

unpublished materials: four websites of registered studies (i.e., http://www.controlled-15

trials.com, http://apps.who.int/trialsearch/, http://clinicaltrials.gov/, 16

https://eudract.emea.europa.eu/), references embedded in book chapters, references 17

included in the annual national reports written by EMCDDA national focal points, and any 18

publications recommended by prominent researchers in the field. We did not set any 19

constraints, such as language or time, to our search. 20

21

Page 6 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



7

Selection Criteria 1

As described in detail previously,[5] we considered studies involving participants under the 2

age of 26 and evaluating mass-media campaigns explicitly aimed at influencing the use or 3

intention to use illicit drugs.[6] The following were deemed acceptable comparison groups: (i) 4

no intervention; (ii) community-based or school-based drug prevention programmes; (iii) 5

lower exposure to intervention; (iv) time before exposure to intervention. We included 6

randomized controlled trial (RCT), cohort studies, interrupted time-series (ITS) studies, and 7

controlled before and after (CBA) studies providing evidence on drug use or intention not to 8

use, to reduce use or to stop use of illicit drugs. 9

10

Two authors independently inspected search hits by reading titles and abstracts and assessed 11

studies for inclusion. Any disagreement was solved by consensus. Multiple publications 12

pertaining to the same study were collated as one single study. 13

14

Quality Appraisal 15

Four authors independently performed quality assessments, and any disagreement was 16

solved by consensus. We contacted study authors whenever information was missing or 17

unclear. 18

19

We used standardized assessment tools for each study design—details are available as 20

supplementary materials (Appendix 2). For RCTs, we used the Cochrane Collaboration Risk of 21

Bias assessment tool.[7] For cohort studies, we followed the Scottish Intercollegiate 22

Page 7 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



8

Guidelines Network (SIGN) Quality Criteria.[8] For ITS and CBA studies, we used the tool 1

recommended by the Cochrane Effective Practice and Organization of Care Group.[9] 2

3

Statistical Analysis 4

As for RCTs, we performed a random effects meta-analysis to estimate the pooled effect of 5

mass-media interventions on drug use whilst accounting for between study heterogeneity, as 6

described in detail previously.[5] We carried out a fixed effects meta-analysis to estimate the 7

pooled intervention effect on intention to use drugs. We tested between study heterogeneity 8

using the chi-squared test and the I2 statistic. Since most studies assessed their outcome 9

variables with different scales, we used standardized mean difference as the summary 10

measure of choice. For two clustered RCTs,[10,11] we inflated standard errors to account for 11

within-cluster correlations.[7] 12

13

We pooled the effect estimates of the Meth Project studies using mixed effects logistic 14

regression.[12–16] We fitted the following model: logit(useij) = β0 + u0j + β1timei + β2agei + 15

β3intervi + β4age×intervi, in which use was prevalence of methamphetamine use, time was a 16

continuous variable, age and intervention were two-level categorical variables.[17] We 17

allowed log odds of methamphetamine use to vary randomly by each j-th state. 18

19

20

Results 21

Out of 18,343 titles and abstracts, we selected 24 papers corresponding to 19 individual 22

studies (Figure 1). 23

Page 8 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



9

1

Study Characteristics 2

Overall, 184,811 subjects were included, with most studies comprising participants who were 3

aged between 10 and 19 years old (Table 1). Although most studies included both boys and 4

girls, two studies focused on girls.[18,19] One study considered Asian-Americans as the only 5

ethnic group eligible for inclusion,[18] whilst the other studies did not focus on any specific 6

ethnic groups. Seventeen studies were conducted in the U.S., one in the U.S. and Canada,[19] 7

and one in Australia.[11] 8

9

Eleven studies (58%) evaluated multi-component interventions, 3 regarding radio/ television 10

and printed advertising,[10,20,21] and 8 regarding radio and television commercials, printed 11

advertisements, and Internet advertising.[2,12–16,22,23] Eight studies evaluated standalone 12

interventions, 4 consisting in radio and television commercials,[24–27] and 4 consisting in 13

Internet-based interventions.[11,18,19,28] The included studies in this review were grounded 14

in a wide range of underlying theories (Table 1). 15

16

Comparison groups varied considerably across studies. For thirteen studies (68%), the 17

comparison group consisted in no exposure to any intervention. Four studies compared high 18

exposure vs. low exposure to the same mass media intervention.[2,21–23] For one study, the 19

comparison group consisted in the standard drug education curriculum.[11] One study had 20

four study arms consisting either in another intervention or no intervention.[26] 21

22

Page 9 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



10

Eight studies were conducted in an experimental setting by explicitly inviting participants and 1

these studies were randomized controlled trials (RCTs).[10,11,18,19,25–28] Ten studies were 2

conducted in a field setting without explicitly inviting participants, as would usually happen 3

with most mass-media campaigns. Of them, 2 were cohort studies,[22,23] 6 were 4

ITS,[2,12,14–16,24] and 2 were CBA studies.[13,20] One study had a double design as it was 5

conducted in an experimental setting with a RCT design, and in a field setting with a cohort 6

design.[21] When specified, follow-up varied from 6 months [19,28] to 4.7 years.[22] 7

8

Study Quality 9

On the whole, the quality of the RCTs is acceptable (Table 2). As described in detail 10

previously,[5] the strongest domain appears to be the risk of attrition bias and the weakest 11

domain the risk of selection bias (unclear description of randomization procedure). In one 12

paper findings of secondary outcomes were reported only as a predictor of the primary 13

outcome, and the paper concerned was deemed at high risk for reporting bias.[19] 14

All cohort studies focused on a clear and appropriate question. Sub-group comparisons 15

between participants and drop-outs were carried out only in one study.[23] The same study, 16

however, failed to control for potential confounders. 17

The proportion of subjects with no missing data was reported only in one controlled CBA 18

study.[20] Potential confounders were accounted for only in one ITS study [2] and in one CBA 19

study.[20] A formal test of trend was not performed in the five Meth Project studies.[12–16] 20

One ITS study [2] and the two CBA studies [13,20] had three or less data points, which are 21

generally considered insufficient for drawing reliable conclusions with regard to intervention 22

effectiveness. 23

Page 10 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



11

1

Effects of Mass-media Campaigns 2

Use of Illicit Drugs 3

Experimental studies. Pooled analyses of five RCTs [10,11,18,19,28] comprising n=5,470 4

subjects showed no evidence (p=0.79) of an effect of mass-media campaigns in modifying use 5

of illicit drugs (standardized mean difference [SMD] -0.02; 95% confidence interval [CI] -0.15 6

to 0.12) (Figure 2 and Table 3). There was some evidence (p=0.020) of heterogeneity between 7

studies. 8

The RCT part of a mixed RCT-cohort study (n=3,236) found evidence of effectiveness 9

(p=0.026) for a media-community intervention (odds ratio [OR] 0.60; 95% CI 0.38 to 0.94) 10

(Table 3).[21] 11

12

Field studies. Two studies found that the Office of National Drug Control Policy (ONDCP) 13

National Youth Anti-Drug Media Campaign (first version) increased use of illicit drugs among 14

adolescents (Table 3). One study (n=3,529) reported a significant increase in past year-use of 15

marijuana (OR 1.21; 95% CI 1.19 to 1.65).[22] The other study (n=2,515) found some evidence 16

(‘p<0.05’) of a iatrogenic effect among those aged 15 to 18 (mean change = 0.144), whilst 17

there was no evidence (‘p>0.05’) of an effect among those aged 13 to 14 (mean change = -18

0.022).[23] 19

The revamped version of the same ONDCP campaign, Above the Influence, was found 20

effective in a mixed RCT-cohort study (n=3,236), whose cohort part found strong evidence 21

(p<0.001) of effectiveness (OR 0.26; 95% CI 0.19 to 0.35).[21] On a similar note were the 22

findings of an ITS study (n=130,245) which evaluated Above the Influence and found evidence 23

Page 11 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



12

of reductions in marijuana use in the past month (OR 0.67; 95% CI 0.52 to 0.87) among 1

eighth-grade girls.[2] 2

The pooled findings of the five Meth Project studies (n=26,273) suggested no evidence of a 3

change in past-month use of methamphetamine among subjects aged 12 to 17 (OR 1.16; 95% 4

CI 0.83 to 1.61), nor among those aged 18 to 24 (OR 1.63; 95% CI 0.70 to 3.79) (Figure 3(a) 5

and Table 3). There was, however, evidence (p=0.001) of a reduction in past-year use of 6

methamphetamine among those aged 12 to 17 (OR 0.59; 95% CI 0.43 to 0.81), whilst there 7

was no evidence of a similar effect among those aged 18 to 24 (OR 0.70; 95 % CI 0.34 to 1.45) 8

(Figure 3(b) and Table 3). 9

One ITS (n=6,371) showed evidence of effectiveness for past 30-day use of marijuana among 10

high sensation seekers (p=0.001 for the Fayette sample, p=0.001 for the first campaign in the 11

Knox sample, p=0.002 for the second campaign in the Knox sample).[24] 12

One CBA study found an increase in use of LSD (‘p<0.001’) (Table 3) whilst no evidence 13

(‘p>0.05’) of differences were found for marijuana, cocaine, amphetamine, and heroin.[20] 14

15

Intention to Use Drugs 16

Experimental studies. In one meta-analysis of 4 randomized controlled studies involving 1,270 17

subjects, there was no evidence of an effect (p=0.21) of media campaigns in changing 18

intention to use drugs (SMD -0.07; 95% CI -0.19 to 0.04) (Figure 2 and Table 3).[18,25–27] 19

There was no evidence (p=0.840) of heterogeneity across studies. 20

21

Page 12 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



13

Field studies. One study (n=2,915) found some evidence (p=0.053) of a reduction in intentions 1

to use marijuana (OR 0.89; 95% CI 0.79 to 1.00) (Table 3) for the first version of the ONDCP’s 2

media campaign.[22] 3

4

5

Discussion 6

Mass-media campaigns are commonly used throughout the world to tackle a broad array of 7

preventable risk factors or injuries. Such campaigns are seldom evaluated, thus making it 8

difficult to inform policymakers regarding their effectiveness and sustainability. In this 9

panorama of overall uncertainty, mass-media campaigns tackling tobacco and traffic 10

accidents are noteworthy exceptions as they have been evaluated more frequently and have 11

shown strong evidence for benefit.[29] 12

13

In our attempt to summarize evidence on the effectiveness of mass-media campaigns 14

targeting illicit drugs, we included 19 studies evaluating a number of heterogeneous 15

interventions. We grouped interventions according to whether they were evaluated with 16

studies conducted in experimental settings in which participants were aware of being 17

exposed to media interventions; or were assessed with studies carried out in a field 18

environment which is more likely to show the real-life effects of large national media 19

campaigns—but is also more prone to risk of bias. 20

21

Findings appear to vary considerably according to the type of intervention and study design. 22

Pooled analyses of 8 interventions evaluated in an experimental setting provided no evidence 23

Page 13 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



14

of beneficial effects for use or intention to use illicit drugs, an indicator of possible future 1

behaviour.[30,31] Four interventions evaluated with 8 field studies revealed some evidence of 2

beneficial effects: (i) the revamped campaign by the Office of National Drug Control Policy 3

(ONDCP) called Above the Influence, which was found effective in one study and effective 4

among eighth-grade girls in another study; (ii) the Be Under Your Own Influence media-5

community intervention; (iii) the Meth Project campaign, which was found effective on past-6

year methamphetamine use, although only among adolescents aged 12 to 17 years old; and 7

(iv) the U.S. televised anti-marijuana campaigns broadcasted in Fayette County (Lexington), 8

Kentucky, and in Knox County (Knoxville), Tennessee, which were found to be effective on 9

high-sensation seekers. However, 2 interventions evaluated with 3 field studies were found to 10

have iatrogenic effects: most notably, the first version of the ONDCP’s media campaign My 11

Anti Drug, which was evaluated by 2 studies and found to increase use of marijuana. An 12

adverse effect was also found for a media-community intervention evaluated by a CBA study 13

which provided evidence of increased frequency of LSD use. 14

15

Some of the studies could not be pooled due to the large variation in mass-media 16

intervention type and study design. Similar interventions were often evaluated with different 17

study designs whilst different interventions were sometimes evaluated with the same study 18

design. Pooled analyses could thus be undertaken only for a few similar interventions 19

evaluated with the same study design, and such small sets of pooled studies did not allow 20

carrying out sensitivity analyses. 21

22

Page 14 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



15

We did not set any time or language constraints to our search, accepted all types of 1

controlled study designs, and obtained unpublished data by establishing direct contact with 2

the authors of the original papers. Unfortunately, due to the paucity and inconsistency of 3

available evidence we cannot draw general conclusions as to whether media campaigns are 4

effective in preventing the use or the intention to use illicit drugs. This observation is in line 5

with the findings of similar reviews that used more restrictive inclusion criteria.[29,32] 6

7

An expanded evidence base would probably lead to a more detailed picture and avoid the 8

dissemination of ineffective and harmful campaigns.[33,34] In particular, we feel that further 9

RCTs are warranted to evaluate the efficacy of media interventions in an experimental 10

setting. Cohort or ITS designs should also be conducted to assess the effectiveness of mass-11

media campaigns in an environment closer to real life. ITS and CBA designs are both subtypes 12

of the broader cross sectional study design which experience tells us is not ideal for 13

intervention evaluation. However, this approach could be used whenever carrying out cohort 14

studies is not feasible.[35] 15

16

17

Conclusions 18

Although relatively limited compared to other areas of primary prevention, the evidence base 19

accrued so far on media campaigns targeting illicit drugs allows us to make at least two 20

remarks. First, such campaigns can be evaluated—a fact that is often questioned in several 21

parts of the world—and properly conducted evaluation studies can provide benefits to both 22

Page 15 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



16

research and practice. Second, in the worst-case scenario, media campaigns can be not only 1

ineffective, but even harmful. Contrary to common belief, antidrug media campaigns may be 2

damaging and their dissemination is ethically unacceptable without a prior assessment of 3

their effects. 4

5

Careful evaluation of media campaigns is essential in order to avoid the dissemination of 6

ineffective and iatrogenic interventions, and new campaigns should be implemented in the 7

framework of rigorous evaluation studies. A better understanding of which media 8

interventions work best is likely to result in a more effective prevention of drug use and 9

increased efficiency in the management of public resources. 10

11

Page 16 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



17

Acknowledgements 1

We are grateful to the Cochrane Drug and Alcohol Group for their support in publishing the 2

Cochrane review from which this paper has been extracted.[5] We thank Gregor Burkhart 3

and V. Anna Gyamathy for their input to the drafting of the protocol for this review, and 4

Marie-Christine Ashby for her carefully proofreading of the manuscript. We are grateful to 5

Ling Fang, John Horan, Nicola Newton, Philip Palmgreen, Joan Polansky, Michael Slater, 6

Jennifer Stagnaro, Violeta Taneva, and Marco Yzer for their useful advice and for providing us 7

with their studies' unpublished data whenever possible. 8

9

Contributorship statement 10

EA structured and drafted this paper. MF and FF conceived the systematic review from which 11

this paper originates and overviewed the inclusion of studies and their methodological 12

assessment. MF and EA selected the studies for inclusion. EA and AB extracted the data from 13

the studies and contributed to writing the review. EA contacted study authors. EA and AB did 14

the meta-analysis of randomized controlled trials. AG and EA conducted the meta-analysis of 15

interrupted time-series studies. 16

All authors regularly discussed each step of review process and equally participated in each 17

decision regarding the studies and the analysis. All authors revised the paper. All authors read 18

and approved the final version of this manuscript. 19

20

Page 17 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



18

Competing interests 1

We declare no competing interests. 2

3

Funding 4

No specific funding was available for this review and review protocol. All authors received 5

funding directly from their respective institutions, which allowed them to conduct this study 6

during office hours and to use working resources including IT equipment and journal access. 7

8

Data sharing 9

Data and statistical code used for pooled analyses are available and can be requested from 10

the corresponding author (Elias Allara, [email protected]). 11

12

Page 18 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



19

References 1

1 Sumnall HR, Bellis MA. Can health campaigns make people ill? The iatrogenic potential 2

of population-based cannabis prevention. J Epidemiol Community Health 2007;61:930–3

1. doi:10.1136/jech.2007.060277 4

2 Carpenter CS, Pechmann C. Exposure to the Above the Influence antidrug 5

advertisements and adolescent marijuana use in the United States, 2006-2008. Am J 6

Public Health 2011;101:948–54. doi:10.2105/AJPH.2010.300040 7

3 Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews 8

and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. 9

doi:10.1136/bmj.b2535 10

4 Ferri M, Burkhart G, Allara E, et al. Media campaigns for the prevention of illicit drug 11

use in young people. Cochrane database Syst Rev Published Online First: 2011. 12

doi:10.1002/14651858.CD009287 13

5 Ferri M, Allara E, Bo A, et al. Media campaigns for the prevention of illicit drug use in 14

young people. Cochrane database Syst Rev 2013;6:CD009287. 15

doi:10.1002/14651858.CD009287.pub2 16

6 Brinn MP, Carson K V, Esterman AJ, et al. Mass media interventions for preventing 17

smoking in young people. Cochrane database Syst Rev 2010;:CD001006. 18

doi:10.1002/14651858.CD001006.pub2 19

7 Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. 20

2011.www.cochrane-handbook.org 21

8 Scottish Intercollegiate Guidelines Network. Critical Appraisal: Notes And Checklists. 22

2010.http://www.sign.ac.uk/methodology/checklists.html (accessed 12 Sep2010). 23

9 Cochrane Effective Practice and Organisation of Care Group. Draft EPOC Methods 24

Paper Including Interrupted Time Series (ITS) Designs in a EPOC Review. 1998. 25

10 Slater MD, Kelly KJ, Edwards RW, et al. Combining in-school and community-based 26

media efforts: reducing marijuana and alcohol uptake among younger adolescents. 27

Health Educ Res 2006;21:157–67. doi:10.1093/her/cyh056 28

11 Newton NC, Teesson M, Vogl LE, et al. Internet-based prevention for alcohol and 29

cannabis use: final results of the Climate Schools course. Addiction 2010;105:749–59. 30

doi:10.1111/j.1360-0443.2009.02853.x 31

12 The Meth Project. Colorado Meth: Use & Attitudes Survey 2011. 2011. 32

Page 19 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



20

13 The Meth Project. Georgia Meth: Use & Attitudes Survey 2011. 2011. 1

14 The Meth Project. Idaho Meth: Use & Attitudes Survey 2010. 2010. 2

15 The Meth Project. Hawaii Meth: Use & Attitudes Survey 2011. 2011. 3

16 The Meth Project. Wyoming Meth: Use & Attitudes Survey 2011. 2011. 4

17 Gilmour S, Degenhardt L, Hall W, et al. Using intervention time series analyses to assess 5

the effects of imperfectly identifiable natural events: a general method and example. 6

BMC Med Res Methodol 2006;6:16. doi:10.1186/1471-2288-6-16 7

18 Fang L, Schinke SP, Cole KCA. Preventing substance use among early Asian-American 8

adolescent girls: initial evaluation of a web-based, mother-daughter program. J Adolesc 9

Health 2010;47:529–32. doi:10.1016/j.jadohealth.2010.03.011 10

19 Schwinn TM, Schinke SP, Di Noia J. Preventing drug abuse among adolescent girls: 11

outcome data from an internet-based intervention. Prev Sci 2010;11:24–32. 12

doi:10.1007/s11121-009-0146-9 13

20 Miller WR, Toscova RT, Miller JH, et al. A theory-based motivational approach for 14

reducing alcohol/drug problems in college. Health Educ Behav 2000;27:744–59. 15

21 Slater MD, Kelly KJ, Lawrence FR, et al. Assessing media campaigns linking marijuana 16

non-use with autonomy and aspirations: “Be Under Your Own Influence” and ONDCP’s 17

“Above the Influence”. Prev Sci 2011;12:12–22. doi:10.1007/s11121-010-0194-1 18

22 Hornik R. Personal Influence and the Effects of the National Youth Anti-Drug Media 19

Campaign. Ann Am Acad Pol Soc Sci 2006;608:282–300. 20

doi:10.1177/0002716206291972 21

23 Scheier LM, Grenard JL. Influence of a nationwide social marketing campaign on 22

adolescent drug use. J Health Commun 2010;15:240–71. 23

doi:10.1080/10810731003686580 24

24 Palmgreen P, Donohew L, Lorch EP, et al. Television campaigns and adolescent 25

marijuana use: tests of sensation seeking targeting. Am J Public Health 2001;91:292–6. 26

25 Polansky JM, Buki LP, Horan JJ, et al. The Effectiveness of Substance Abuse Prevention 27

Videotapes with Mexican American Adolescents. Hisp J Behav Sci 1999;21:186–98. 28

doi:10.1177/0739986399212005 29

26 Yzer MC, Cappella JN, Fishbein M, et al. The effectiveness of gateway communications 30

in anti-marijuana campaigns. J Health Commun 2003;8:129–43. 31

doi:10.1080/10810730305695 32

Page 20 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



21

27 Zhao X, Sayeed S, Cappella J, et al. Targeting norm-related beliefs about marijuana use 1

in an adolescent population. Health Commun 2006;19:187–96. 2

doi:10.1207/s15327027hc1903_1 3

28 Lee CM, Neighbors C, Kilmer JR, et al. A brief, web-based personalized feedback 4

selective intervention for college student marijuana use: a randomized clinical trial. 5

Psychol Addict Behav 2010;24:265–73. doi:10.1037/a0018859 6

29 Wakefield MA, Loken B, Hornik RC. Use of mass media campaigns to change health 7

behaviour. Lancet 2010;376:1261–71. doi:10.1016/S0140-6736(10)60809-4 8

30 Litchfield R, White K. Young Adults’ Willingness and Intentions to use Amphetamines : 9

An Application of the Theory of Reasoned Action. E-Journal Appl Psychol 2006;2:45–51. 10

doi:10.7790/ejap.v2i1.34 11

31 Olds RS, Thombs DL, Tomasek JR. Relations between normative beliefs and initiation 12

intentions toward cigarette, alcohol and marijuana. J Adolesc Health 2005;37:75. 13

doi:10.1016/j.jadohealth.2004.09.020 14

32 Werb D, Mills EJ, Debeck K, et al. The effectiveness of anti-illicit-drug public-service 15

announcements: a systematic review and meta-analysis. J Epidemiol Community Health 16

2011;65:834–40. doi:10.1136/jech.2010.125195 17

33 Faggiano F, Giannotta F, Allara E. Strengthening Prevention Science to Ensure 18

Effectiveness of Intervention in Practice: Setting up an International Agenda. In: 19

Sloboda Z, Petras H, eds. Defining Prevention Science. Boston, MA: : Springer US 2014. 20

597–615. doi:10.1007/978-1-4899-7424-2_26 21

34 Faggiano F, Allara E, Giannotta F, et al. Europe Needs a Central, Transparent, and 22

Evidence-Based Approval Process for Behavioural Prevention Interventions. PLOS Med 23

2014;11:e1001740. doi:10.1371/journal.pmed.1001740 24

35 Sackett D, Straus S, Richardson W, et al. Evidence-based medicine: how to practice and 25

teach EBM. 2nd ed. Edinburgh: : Churchill Livingstone 2000. 26

27

Page 21 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



22

Figure 1. PRISMA Flow Diagram

Note: adapted from a previous publication.[5]

Page 22 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



23

Figure 2. Pooling of Randomized Controlled Trials

(a) Drug use

(b) Intention to use drugs


Page 23 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



24

Figure 3. Pooling of the Meth Project Interrupted Time-Series studies: Predicted and Observed Probabilities

(a) Past-month Use of Methamphetamines

(b) Past-year Use of Methamphetamines


Page 24 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



25

Table 1. Characteristics of Included Studies

Study Underpinning

theory Design (goal)a

Interventionb Comparisonc Primary outcome

Secondary outcome(s) Analysis sample

Follow-up (total time) [months]

Polansky 1999[25] Decision theory RCT (E) PSA Other –

Intention to use; attitudes; knowledge and

disposition to select socially appropriate

responses

312 n.a. (n.s.)

Miller 2000[20] Self regulation theory CBA (F) PSA; printed No intervention

Use of drugs (incl. cannabis and cocaine)

Risk perception ; problems related to drug

use 1,024 12 (18)

Palmgreen 2001[24] Influence of sensation-seeking on drug use

ITS (F) PSA No intervention Past-30-day use of marijuana – 6,371 n.a. (32)

Yzer 2003[26]

Theories of behavioral change: persuasion effects

RCT (E) PSA No

intervention; other

–

Intention to use marijuana; attitude; perceptions about

marijuana

418 n.a. (n.s.)

Slater 2006[10]

Social-ecological framework (norms and expectations influence drug use)

RCT (E) PSA; printed No intervention Lifetime and

past-30-day use of marijuana

– 4,216 24 (42)

Zhao 2006[27] Normative beliefs RCT (E) PSA No intervention – Intention to use; beliefs

towards marijuana; social norms

435 n.a. (n.s.)

Hornik 2006[22] Unclear Cohort (F)

PSA; printed; internet

Lower exposure

Lifetime, past-year, and past-30-day use of

marijuana

Intention to use; attitudes and self

efficacy; perceptions and social norms

8,117 56 (58)

Scheier 2010[23] Social marketing Cohort (F)


Lower exposure

Past-12-month cannabis

intoxications – 2,515 n.a. (48)

Schwinn 2010[19] Social learning theory RCT (E) Internet No intervention Past-30-day

substance use – 236 6 (n.s.)

Lee 2010[28] Readiness to change RCT (E) Internet No intervention Past-90-day use

of marijuana Intention to change

marijuana use; consequences

341 6 (n.s.)

Fang 2010[18] Family-oriented RCT (E) Internet No intervention Past-30-day use of marijuana

Intention to use marijuana 216 6.25 (16)

Newton 2010[11] Social influence approach RCT (E) Internet Other Use of cannabis Cannabis knowledge;

attitudes; related harms 724 12 (21)

Meth Project studies[12–16]

Perception of risk and perception of social disapproval are correlated with drug consumption

4 ITS and 1 CBA

(F)

PSA; printed; internet No intervention

Past-30-day use of

methamphetamine

Attitudes on methamphetamine and

other drugs; perceptions; information sources and advertising awareness;

26,405

n.a. (Colorado

26; Georgia

18; Hawaii 25;

Idaho 40; Wyoming

34)

Page 25 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



26

Slater 2011[21]

Autonomy and aspiration perceptions as mediators marijuana use

RCT (E); Cohort

(F) PSA; printed Lower

exposure

Lifetime, past-90-day and past-

30-day use of marijuana

Autonomy and aspiration inconsistent with marijuana use

3,236 24 (42)

Carpenter 2011[2]

Unclear; evaluated many heterogeneous mass-media campaigns

ITS (F) PSA; printed; internet

Lower exposure

Past-30-day and lifetime use of

marijuana – 130,245 n.a. (36)

n.a. = not applicable. n.s. = not specified. a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. E = experimental/efficacy setting. F = field/effectiveness setting. b PSA = public service announcement (e.g., television/radio). c Other = other intervention or different combination of same intervention. Lower exposure = lower exposure to same intervention.

Page 26 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



27

Table 2. Risk of Bias of Included Studies

Designa Study Random

sequence generation

Allocation concealment

Blinding of outcome

assessment Attrition

Selective reporting

Comparability of groups

Acceptance among

recruited

Attrition by exposure

status

Strategies alternative to blinding

Discussion of potential

confounders

Statistical accuracy

Overall risk of bias

(cohort)

Sufficient data points

for inference

RCT

Polansky 1999[25] Yzer 2003[26] Slater 2006[10] Zhao 2006[27] Schwinn 2010[19] Lee 2010[28] Fang 2010[18] Newton 2010[11]

Coho

rt Slater 2011[21] b,c Hornik 2006[22] b Scheier 2010[23] b

ITS

Palmgreen 2001[24] d Carpenter 2011[2] d 4 Meth Project studies[12,14–16] d

CBA 1 Meth Project study[13] d Miller 2000[20] d

████ = low risk of bias. ████ = unclear risk of bias. ████ = high risk of bias. White = not applicable Study quality was appraised with three different tools depending on study design. Redundant or similar items were collapsed. The 'the intervention was independent of other changes' item of the ITS checklist was considered equivalent to the 'discussion of potential confounder' item for cohort studies, the 'formal test for trend' ITS item was considered equivalent to the 'statistical accuracy' item for cohort studies, and the 'completeness of dataset' ITS item was considered equivalent to the 'attrition' item for RCT and cohort studies. a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. b All cohort studies had low risk of bias for the following items: 'likelihood of outcome already present at enrolment', 'clarity of outcome', 'reliability of assessment of exposure', 'use of other sources to corroborate outcome measure', and 'multiple measure of exposure'. c Slater 2011 is a mixed RCT-cohort study. d All ITS studies and the CBA study had low risk of bias for the following items: 'intervention unlikely to affect data collection', and 'reliable primary outcome measure(s)'.

Page 27 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on May 14, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2014-007449 on 3 September 2015. Downloaded from



28

Table 3. Main Findings for Use or Intention to Use Illicit Drugs

Pooling Outcome Designa References Sub-groups No. of

subjects exp vs. ctrlb

Effect measure

Effect size (95% CI) or effect direction (p-

value)c

Heterogeneity p-valued

Pool

ed a

naly

ses

Use of illicit drugs RCT

Slater 2006[10]; Fang 2010[18]; Newton 2010[11]; Schwinn 2010[19]; Lee 2010[28]

– 2,701 vs. 2,769

SMD, random effects

-0.02 (-0.15 to 0.12) 0.020 *

Intention to use illicit drugs RCT

Polansky 1999[25]; Yzer 2003[26]; Zhao 2006[27]; Fang 2010[18]

– 771 vs. 499 SMD, fixed effects -0.07 (-0.19 to 0.04) 0.840

Past-month use of methamphetamine

4 ITS and 1 CBA


age 12-17 14,865 vs. 7,497

OR, random effects

1.16 (0.83 to 1.61) –

age 18-24 347 vs. 632 OR,

random effects

1.63 (0.70 to 3.79) –

Past-year use of methamphetamine

4 ITS and 1 CBA


age 12-17 17,105 vs 7,497

OR, random effects

0.59 (0.43 to 0.81) ** –

age 18-24 1,039 vs. 632 OR,

random effects

0.70 (0.34 to 1.45) –

Sing

le s

tudi

es

Lifetime, past-90-day, or past-30-day use of marijuana

RCT (community-

media) Slater 2011[21]

–

n.a. (3,236)

OR, random effects

0.60 (0.38 to 0.94) * –

Cohort (mass-media)

– OR,

random effects

0.26 (0.19 to 0.35) *** –

Past-year use of marijuana Cohort Hornik 2006[22]

– n.a. (3,529) OR, fixed effects 1.21 (1.19 to 1.65) * –

Intention to use marijuana – n.a. (2,915) OR, fixed

effects 0.89 (0.79 to 1.00) § –

Past-12-month episodes of cannabis intoxication

Cohort Scheier 2010[23]

age 13-14

n.a. (2,515)

mean difference,

SEM -0.022 –

age 15-18 mean

difference, SEM

0.144 * –

Past-30-day use of marijuana among high sensation seekers

ITS Palmgreen 2001[24]

Fayette n.a. (3,174) test for slope ↓ (p=0.001) –

Knox, first campaign

n.a. (3,197)

test for slope ↓ (p=0.001)

Knox, second campaign

test for slope ↓ (p=0.002) –

Past-30-day use of marijuana (girls, 8th grade)

ITS Carpenter 2011[2] – n.a. (130,245) OR, fixed effects 0.67 (0.52 to 0.87) ** –

Frequency of use of 10 types of drugs

CBA Miller 2000[20] – 567 vs 431 mean

difference, ANOVA

for LSD: ↑ (p<0.001) for marijuana, cocaine,

amphetamine, and heroin: 'no longer

significant' differences

–

§ < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 OR = odds ratio SMD = standardized mean difference SEM = structural equation modeling a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. b n.a. = breakdowns of students exposed to the interventions were not available. Number of analyzed subjects is between brackets. c Whenever the effect size was not reported, ↓ = decreased use or intention to use, and ↑ = increased use or intention to use. d Heterogeneity test for meta-analyses of RCTs.

Page 28 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



29

Appendix 1. Search Strategy

Appendix 2. Quality Appraisal

Appendix 3. PRISMA Checklist

Page 29 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from




Note: search strategies have already been described in detail previously [5].

(a) PubMed (MEDLINE) Search Strategy

Search Query Items found

#16 Search (((#3) AND #4) AND #11) AND #15 5877

#15 Search ((#12) OR #13) OR #14 3,041,802

#14 Search adolescen*[tiab] OR preadolescen*[tiab] OR child*[tiab] OR teen*[tiab] OR youth*[tiab] OR young[tiab] OR kid*[tiab] OR juvenile*[tiab] OR minors[tiab] OR boy*[tiab] OR girl*[tiab]

1,662,519

#13 Search "Child"[Mesh] 1,457,004

#12 Search "Adolescent"[Mesh] 1,498,465

#11 Search ((((#5) OR #7) OR #8) OR #9) OR #10 797,788

#10

Search media[tiab] OR Communication*[tiab] OR audiovisual[tw] OR telecommunication*[tw] OR Educat*[tiab] OR radio[tw] OR television[tw] OR TV[tiab] OR internet[tw] OR campaign*[tw] OR advert*[tw] OR twitter[tw] OR facebook[tw] OR "instant messaging"[tw]

751,996

#9 Search "Telecommunications"[Mesh] 54,815

#8 Search Videotape Recording[Mesh] 9970

#7 Search "Internet"[Mesh] 43,359

#5 Search "Mass Media"[Mesh] 37,325

#4

Search "heroin"[Mesh] OR heroin[tiab] OR "Street Drugs"[Mesh] OR "Designer Drugs"[Mesh] OR "Crack Cocaine"[Mesh] OR "Lysergic Acid Diethylamide"[Mesh] OR drug*[tiab] OR polydrug[tiab] OR substance[tiab] OR hallucinogen*[tw] OR cocaine[tw] OR amphetamine*[tw] OR "lysergic acid diethylamide"[tw] OR LSD [tiab] OR ketamine[tw] OR cannabis[tw] OR marihuana[tw] OR marijuana[tiab] OR hashish[tw] OR steroid*[tw] OR morphine[tiab] OR ecstasy[tw] OR MDMA[tw] OR benzodiazepine[tw]

1,136,251

#3 Search (#1) OR #2 1,812,638

#2 Search abus*[tiab] OR consumption[tiab] OR misus*[tiab] OR use*[tiab] OR addict*[tiab] OR disorder*[tiab]

1,570,344

#1 Search "Substance-Related disorders"[Mesh] 344,574

(b) EMBASE Search Strategy

ID Query

#1 'substance abuse'/exp

#2 'drug abuse'/exp

#3 abus*:ab,ti OR consumption:ab,ti OR misus*:ab,ti OR use*:ab,ti OR addict*:ab,ti OR disorder*:ab,ti

#4 #1 OR #2 OR #3

#5

heroin:ab,ti OR drug*:ab,ti OR polydrug:ab,ti OR substance:ab,ti OR hallucinogen*:ab,ti OR cocaine:ab,ti OR amphetamine*:ab,ti OR 'lysergic acid diethylamide':ab,ti OR lsd:ab,ti OR ketamine:ab,ti OR cannabis:ab,ti OR marihuana:ab,ti OR marijuana:ab,ti OR hashish:ab,ti OR steroid*:ab,ti OR morphine:ab,ti OR ecstasy:ab,ti OR mdma:ab,ti OR benzodiazepine:ab,ti

#6 'diamorphine'/exp

#7 'designer drug'/exp

#8 'street drug'/exp

#9 'cocaine'/exp

Page 30 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



#10 'cannabis smoking'/exp

#11 #5 OR #6 OR #7 OR #8 OR #9 OR #10

#12 'mass medium'/exp

#13 'internet'/exp

#14 'videorecording'/exp

#15 'telecommunication'/exp

#16 media:ab,ti OR communication*:ab,ti OR audiovisual:ab,ti OR telecommunication*:ab,ti OR educat*:ab,ti OR radio:ab,ti OR television:ab,ti OR tv:ab,ti OR internet:ab,ti OR campaign*:ab,ti OR advert*:ab,ti OR twitter:ab,ti OR facebook:ab,ti

#17 #12 OR #13 OR #14 OR #15 OR #16

#18 'adolescent'/exp

#19 'child'/exp

#20 adolescen*:ab,ti OR preadolescen*:ab,ti OR child*:ab,ti OR teen*:ab,ti OR youth*:ab,ti OR young:ab,ti OR kid*:ab,ti OR juvenile*:ab,ti OR minors:ab,ti OR boy*:ab,ti OR girl*:ab,ti

#21 #18 OR #19 OR #20

#22 #4 AND #11 AND #17 AND #21 AND [embase]/lim

(c) ProQuest Search Strategy

(media campaigns OR mass media) AND illicit drug* AND preventi*.

(d) CENTRAL Search Strategy

ID Search Hits

#1 MeSH descriptor: [Substance-Related Disorders] explode all trees 10,355

#2 ((stimulant* or polydrug* or drug* or substance) near/3 (abuse* or abusing or consumption or addict* or disorder* or intoxicat* or misus* or use*)):ti,ab

14,750

#3 (abuse* or abusing or consumption or addict* or disorder* or intoxicat* or misus* or use*):ti,ab 198,966

#4 MeSH descriptor: [Narcotics] explode all trees 681

#5 heroin:ti,ab 762

#6 MeSH descriptor: [Street Drugs] explode all trees 196

#7 MeSH descriptor: [Amphetamine] explode all trees 632

#8 (amphetamine* or dextroamphetamine* or methamphetamine or Methylamphetamine*):ti,ab,kw (Word variations have been searched)

1442

#9 (ecstasy or MDMA or hallucinogen*):ti,ab,kw (Word variations have been searched) 234

#10 MeSH descriptor: [Cocaine] explode all trees 576

#11 (crack or cocaine):ti,ab,kw (Word variations have been searched) 1953

#12 MeSH descriptor: [Cannabis] explode all trees 245

#13 (cannabis or marijuana or marihuana or Hashish):ti,ab,kw (Word variations have been searched)

1158

#14 (Lysergic next Acid):ti,ab,kw 76

#15 LSD:ti,ab,kw (Word variations have been searched) 131

#16 (benzodiazepine* or barbiturate* or ketamine or solvent or inhalant):ti,ab,kw (Word variations have been searched)

6370


6370

#18 #4 or #5 or #6 or #7 or #8 or #9 or #10 or #11 or #12 or #13 or #14 or #15 or #16 or #17 11,919

#19 #3 and #18 6547

#20 #1 or #2 or #19 26,077

Page 31 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



#21 MeSH descriptor: [Mass Media] explode all trees 1337

#22 MeSH descriptor: [Internet] explode all trees 1248

#23 MeSH descriptor: [Videotape Recording] explode all trees 790

#24 "Tv":ti,ab,kw (Word variations have been searched) 386

#25 (media or communication* or audiovisual or telecommunication* or radio or television or internet or campaign* or advert* or twitter or facebook) (Word variations have been searched)

27,766

#26 #21 or #22 or #23 or #24 or #25 28,828

#27 MeSH descriptor: [Adolescent] explode all trees 68,885

#28 adolescen* or preadolescen* or child* or teen* or youth* or young or kid* or juvenile* or minors or boy* or girl*:ti,ab,kw (Word variations have been searched)

157,753

#29 #27 or #28 157,753

#30 #20 and #26 and #29 566

Page 32 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Appendix 2. Details on Quality Appraisal

(a) Assessment of Randomized Controlled Trials

The recommended approach for assessing risk of bias in studies included in Cochrane Reviews is a two-part

tool, addressing (a) sequence generation and allocation concealment, (b) blinding of participants and

providers, (c) blinding of outcome assessor, (d) incomplete outcome data, (e) selective outcome reporting,

and (f) other source of bias. Blinding of participants is not always applicable for mass media campaigns, and

we therefore considered blinding of personnel and outcome assessors. A study was deemed to have low

risk of bias for blinding if the data were obtained with a questionnaire administered anonymously or by

computer.

(b) Assessment of Cohort Studies

This tool comprises several questions intended to serve as guidance for the study assessors, regarding (a)

blinding of outcome assessment; (b) clarity of outcome; (c) likelihood of outcome already present at

enrolment; (d) use of other sources to corroborate outcome measure; (e) reliability of assessment of

exposure; (f) multiple measure of exposure; (g) attrition; (h) comparability of groups; (i) acceptance among

recruited; (j) attrition by exposure status; (k) strategies alternative to blinding; (l) discussion of potential

confounders; (m) statistical accuracy, and (n) overall risk of bias.

(c) Assessment of interrupted time series (ITS) and before and after (CBA)

studies

This tool requires assessing whether there is presence of (a) an intervention independent of other changes;

(b) sufficient data points to enable reliable statistical inference; (c) formal tests for trend; (d) an

intervention unlikely to affect data collection; (e) blinded assessment of primary outcome(s); (f)

completeness of dataset; (g) reliable primary outcome measures.

Page 33 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist

Section/topic # Checklist item Reported on page #

TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both. 1

ABSTRACT

Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

3

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known. 5,6

Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

6

METHODS

Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

6

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,

language, publication status) used as criteria for eligibility, giving rationale.

7

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

6

Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

Appendix 1

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

7

Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

7, 8, Appendix 2

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

7, 8, Appendix 2, protocol

Risk of bias in individual studies

12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

8, Appendix 2

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). 8

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency

(e.g., I2) for each meta-analysis.

8

Page 34 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 1 of 2


Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

N/A

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

8

RESULTS

Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

8, Figure 1

Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.

9, 10, Table 1

Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). 10, Table 2

Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.

Table 3, Figure 2, Figure 3

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. 11-13,Table 3

Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). 10, Table 2

Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). 12, Table 3 (Meth Proj.)

DISCUSSION

Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).

13, 14

Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).

15

Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. 15, 16

FUNDING

Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

N/A

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097

For more information, visit: www.prisma-statement.org.

Page 2 of 2

Page 35 of 35


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Are Mass-media Campaigns Effective in Preventing Drug Use? A Cochrane Systematic Review and Meta-analysis

Journal: BMJ Open

Manuscript ID: bmjopen-2014-007449.R1

Article Type: Research

Date Submitted by the Author: 25-Mar-2015

Complete List of Authors: Allara, Elias; Università del Piemonte Orientale, Department of Translational Medicine; University of Torino, School of Public Health Ferri, Marica; European Monitoring Centre for Drugs and Drug Addiction, Consequences, Responses and Best Practices unit Bo, Alessandra; European Monitoring Centre for Drugs and Drug Addiction, Consequences, Responses and Best Practices unit Gasparrini, Antonio; London School of Hygiene & Tropical Medicine, Faggiano, Fabrizio; Università del Piemonte Orientale, Department of Translational Medicine

<b>Primary Subject Heading</b>:

Public health

Secondary Subject Heading: Addiction

Keywords: PUBLIC HEALTH, Substance misuse < PSYCHIATRY, Health policy < HEALTH SERVICES ADMINISTRATION & MANAGEMENT, MENTAL HEALTH


BMJ Open on M

ay 14, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2014-007449 on 3 Septem

ber 2015. Dow

nloaded from



Are Mass-media Campaigns Effective in Preventing Drug Use? A

Cochrane Systematic Review and Meta-analysis

Authors and affiliations:

Elias Allara (1,2) *, Marica Ferri (3), Alessandra Bo (3), Antonio Gasparrini (4), Fabrizio

Faggiano (1)

(1) Department of Translational Medicine, Università del Piemonte Orientale, Novara, Italy

(2) School of Public Health, University of Torino, Torino, Italy

(3) Consequences, Responses and Best Practices unit, European Monitoring Centre for Drugs

and Drug Addiction, Lisbon, Portugal

(4) London School of Hygiene & Tropical Medicine, London, UK

* Contact person:

Elias Allara

Email: [email protected]. Telephone: +39 348 5922753

Department of Translational Medicine, Università del Piemonte Orientale

Via Solaroli 17, 28100 Novara, Italy

Manuscript word court: 2749

Abstract word court: 228

Last update: 25 March 2015

Page 1 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Note: This article is based on a Cochrane Review published in the Cochrane Database of

Systematic Reviews (CDSR) 2013, Issue 6, DOI: .1002/14651858.CD009287.pub2 (see

www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as

new evidence emerges and in response to feedback, and the CDSR should be consulted for

the most recent version of the review.

Page 2 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Abstract

Objective To determine whether there is evidence that mass-media campaigns can be

effective in reducing illicit drug consumption and the intent to consume.

Design Systematic review of randomized and non-randomized studies.

Methods We searched 4 electronic databases (MEDLINE, EMBASE, ProQuest Dissertations &

Theses A&I, and CENTRAL) and further explored 7 additional resources to obtain both

published and unpublished materials. We appraised the quality of included studies using

standardized tools. We carried out meta-analyses of randomized controlled trials and a

pooled analysis of interrupted time-series and controlled before-and-after studies.

Results We identified 19 studies comprising 184,811 participants. Pooled analyses and

narrative synthesis provided mixed evidence of effectiveness. Eight interventions evaluated

with randomized controlled trials leaned toward no evidence of an effect, both on drug use

(standardized mean difference [SMD] -0.02; 95% confidence interval [CI] -0.15 to 0.12) and

the intention to use drugs (SMD -0.07; 95% CI -0.19 to 0.04). Four campaigns provided some

evidence of beneficial effects in preventing drug use and two interventions provided evidence

of iatrogenic effects.

Conclusions Studies were considerably heterogeneous in type of mass-media intervention,

outcome measures, underlying theory, comparison groups, and design. Such factors can

contribute to explaining the observed variability in results. Due to the risk of adverse effects,

caution is needed in disseminating mass-media campaigns tackling drug use. Large studies

conducted with appropriate methodology are warranted to consolidate the evidence base.

Page 3 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Strengths and limitations of this study

• This systematic review is based on an expanded evidence base of both published and

unpublished findings and aims to determine whether mass-media campaigns can be

effective in preventing the use of or intention to use illicit drugs.

• Pooled analyses of eight mass-media interventions provide no evidence of an effect on

drug use or intention to use illicit drugs. Four interventions provide evidence of

beneficial effects. Two interventions provide evidence of iatrogenic effects.

• Due to the paucity and inconsistency of available evidence we cannot draw general

conclusions as to whether mass-media interventions are effective in preventing the

use of or intention to use illicit drugs.

• This review provides an insight into research gaps around the impact of mass-media

drug prevention interventions and can serve to highlight that new campaigns should

be implemented in the framework of rigorous evaluation studies, in order to avoid

dissemination of interventions that are ineffective or have unintended effects.

Page 4 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Introduction

Mass-media campaigns are a powerful means for disseminating health promotion messages.

A wide and diverse audience can be reached through television commercials, the Internet,

mobile phones, newspapers, and roadside advertising hoardings. In the field of drug addiction

and dependence, adverts may contribute to shaping patterns of drug use and the intention to

use drugs, as well as modifying mediators such as awareness, knowledge, and attitudes about

drugs.

However, ethical and economic considerations are often raised. Mass-media campaigns—

unlike other health interventions—are imposed on populations that have not consented to

their implementation.[1] This is a considerable ethical issue in modern, person-centred public

health, where taking decisions shared with the public is essential for promoting behaviour

change. Secondly, mass-media campaigns can be very expensive, especially when

implemented at national or state level. Large-scale purchasing of public service

announcement time during popular shows and broad dissemination via printed media is often

accessible only to governmental institutions. For example, the first and second versions of the

U.S. Office of National Drug Control Policy’s National Youth Anti-Drug Media Campaign cost

2.7 billion dollars over more than 10 years.[2] Although such campaigns underwent careful

evaluation, most mass-media interventions are not developed in compliance with the classical

circle of public health, which consists in designing interventions based on evidence and in

evaluating their impact.

Page 5 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



A systematic review of the studies assessing media campaigns aiming to prevent use of illicit

drugs can inform future strategies and help design effective campaigns. The objective of this

review is to assess the effectiveness of mass-media campaigns in preventing or reducing drug

use or the intention to use illicit drugs among young people.

Methods

We conducted this systematic review in accordance with the Preferred Reporting Items for

Systematic Reviews and Meta-Analyses (PRISMA)[3] statement and with the procedures

specified in a previously published protocol.[4] As described in detail previously,[5] we

systematically searched 4 electronic databases: MEDLINE (1966 to 29 January 29 2013),

EMBASE (1974 to 30 January 2013), ProQuest Dissertations & Theses A&I (1861 to 3 February

2013), and CENTRAL (2013, Issue 1). Search strategies are available as supplementary files

(Appendix 1). We further explored 7 additional resources to obtain both published and

unpublished materials: four websites of registered studies (i.e., http://www.controlled-

trials.com, http://apps.who.int/trialsearch/, http://clinicaltrials.gov/,

https://eudract.emea.europa.eu/), references embedded in book chapters, references

included in the annual national reports written by EMCDDA national focal points, and any

publications recommended by prominent researchers in the field. We did not set any

constraints, such as language or time, to our search.

Page 6 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Selection Criteria

As described in detail previously,[5] we considered studies involving participants under the

age of 26 and evaluating mass-media campaigns explicitly aimed at influencing the use or

intention to use illicit drugs.[6] The following were deemed acceptable comparison groups: (i)

no intervention; (ii) community-based or school-based drug prevention programmes; (iii)

lower exposure to intervention; (iv) time before exposure to intervention. We included

randomized controlled trial (RCT), cohort studies, interrupted time-series (ITS) studies, and

controlled before and after (CBA) studies providing evidence on drug use or intention not to

use, to reduce use or to stop use of illicit drugs.

Two authors independently inspected search hits by reading titles and abstracts and assessed

studies for inclusion. Any disagreement was solved by consensus. Multiple publications

pertaining to the same study were collated as one single study.

Quality Appraisal

Four authors independently performed quality assessments, and any disagreement was

solved by consensus. We contacted study authors whenever information was missing or

unclear.

We used standardized assessment tools for each study design—details are available as

supplementary materials (Appendix 2). For RCTs, we used the Cochrane Collaboration Risk of

Bias assessment tool.[7] For cohort studies, we followed the Scottish Intercollegiate

Page 7 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Guidelines Network (SIGN) Quality Criteria.[8] For ITS and CBA studies, we used the tool

recommended by the Cochrane Effective Practice and Organization of Care Group.[9]

Statistical Analysis

As for RCTs, we performed a random effects meta-analysis to estimate the pooled effect of

mass-media interventions on drug use whilst accounting for between study heterogeneity, as

described in detail previously.[5] We carried out a fixed effects meta-analysis to estimate the

pooled intervention effect on intention to use drugs. We tested between study heterogeneity

using the chi-squared test and the I2 statistic. A p-value lower than 0.10 in the chi-square test

and an I2 statistic higher than 50% suggested evidence of heterogeneity. Since most studies

assessed their outcome variables with different scales, we used standardized mean difference

(SMD) as the summary measure of choice. SMDs were used for both drug use and intention to

use drugs and were calculated by dividing the difference in mean outcome between groups

by the standard deviation of outcome between participants.[7] SMDs and their standard

errors were then pooled in a meta-analysis performed with RevMan.[10] For two clustered

RCTs,[11,12] we inflated standard errors to account for within-cluster correlations.[7]

We pooled the effect estimates of the Meth Project studies using mixed effects logistic

regression.[13–17] An ITS design was applied for estimating the differences in prevalence of

methamphetamine use before and after the Meth Project intervention, adjusting for any

underlying temporal trend. We fitted the following model: logit(useij) = β0 + u0j + β1timei +

β2agei + β3intervi + β4age×intervi, in which use was prevalence of methamphetamine use,

time was a continuous variable, age and intervention were two-level categorical variables, u0j

Page 8 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



was a random intercept we allowed log odds of methamphetamine use to vary randomly by

each j-th state.[18] The relatively few data points did not allow exploring more complex

models, e.g. the temporal trend could not be assumed to vary randomly across states.

Results

Out of 18,343 titles and abstracts, we selected 24 papers corresponding to 19 individual

studies (Figure 1).

Study Characteristics

Overall, 184,811 subjects were included, with most studies comprising participants who were

aged between 10 and 19 years old (Table 1). Although most studies included both boys and

girls, two studies focused on girls.[19,20] One study considered Asian-Americans as the only

ethnic group eligible for inclusion,[19] whilst the other studies did not focus on any specific

ethnic groups. Seventeen studies were conducted in the U.S., one in the U.S. and Canada,[20]

and one in Australia.[12]

Eleven studies (58%) evaluated multi-component interventions, 3 regarding radio/ television

and printed advertising,[11,21,22] and 8 regarding radio and television commercials, printed

advertisements, and Internet advertising.[2,13–17,23,24] Eight studies evaluated standalone

interventions, 4 consisting in radio and television commercials,[25–28] and 4 consisting in

Internet-based interventions.[12,19,20,29] The included studies in this review were grounded

in a wide range of underlying theories (Table 1).

Page 9 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Comparison groups varied considerably across studies. For thirteen studies (68%), the

comparison group consisted in no exposure to any intervention. Four studies compared high

exposure vs. low exposure to the same mass media intervention.[2,22–24] For one study, the

comparison group consisted in the standard drug education curriculum.[12] One study had

four study arms consisting either in another intervention or no intervention.[27]

Eight studies were conducted in an experimental setting by explicitly inviting participants and

these studies were randomized controlled trials (RCTs).[11,12,19,20,26–29] Ten studies were

conducted in a field setting without explicitly inviting participants, as would usually happen

with most mass-media campaigns. Of them, 2 were cohort studies,[23,24] 6 were

ITS,[2,13,15–17,25] and 2 were CBA studies.[14,21] One study had a double design as it was

conducted in an experimental setting with a RCT design, and in a field setting with a cohort

design.[22] When specified, follow-up varied from 6 months [20,29] to 4.7 years.[23]

Study Quality

On the whole, the quality of the RCTs is acceptable (Table 2). As described in detail

previously,[5] the strongest domain appears to be the risk of attrition bias and the weakest

domain the risk of selection bias (unclear description of randomization procedure). In one

paper findings of secondary outcomes were reported only as a predictor of the primary

outcome, and the paper concerned was deemed at high risk for reporting bias.[20]

Page 10 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



All cohort studies focused on a clear and appropriate question. Sub-group comparisons

between participants and drop-outs were carried out only in one study.[24] The same study,

however, failed to control for potential confounders.

The proportion of subjects with no missing data was reported only in one controlled CBA

study.[21] Potential confounders were accounted for only in one ITS study [2] and in one CBA

study.[21] A formal test of trend was not performed in the five Meth Project studies.[13–17]

One ITS study [2] and the two CBA studies [14,21] had three or less data points, which are

generally considered insufficient for drawing reliable conclusions with regard to intervention

effectiveness.

Effects of Mass-media Campaigns

Use of Illicit Drugs

Experimental studies. Pooled analyses of five RCTs [11,12,19,20,29] comprising n=5,470

subjects showed no evidence (p=0.79) of an effect of mass-media campaigns in modifying use

of illicit drugs (standardized mean difference [SMD] -0.02; 95% confidence interval [CI] -0.15

to 0.12) (Figure 2 and Table 3). There was some evidence (p=0.020) of heterogeneity between

studies.

The RCT part of a mixed RCT-cohort study (n=3,236) found evidence of effectiveness

(p=0.026) for a media-community intervention (odds ratio [OR] 0.60; 95% CI 0.38 to 0.94)

(Table 3).[22]

Field studies. Two studies found that the Office of National Drug Control Policy (ONDCP)

National Youth Anti-Drug Media Campaign (first version) increased use of illicit drugs among

Page 11 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



adolescents (Table 3). One study (n=3,529) reported a significant increase in past year-use of

marijuana (OR 1.21; 95% CI 1.19 to 1.65).[23] The other study (n=2,515) found some evidence

(‘p<0.05’) of a iatrogenic effect among those aged 15 to 18 (mean change = 0.144), whilst

there was no evidence (‘p>0.05’) of an effect among those aged 13 to 14 (mean change = -

0.022).[24]

The revamped version of the same ONDCP campaign, Above the Influence, was found

effective in a mixed RCT-cohort study (n=3,236), whose cohort part found strong evidence

(p<0.001) of effectiveness (OR 0.26; 95% CI 0.19 to 0.35).[22] On a similar note were the

findings of an ITS study (n=130,245) which evaluated Above the Influence and found evidence

of reductions in marijuana use in the past month (OR 0.67; 95% CI 0.52 to 0.87) among

eighth-grade girls.[2]

The pooled findings of the five Meth Project studies (n=26,273) suggested no evidence of a

change in past-month use of methamphetamine among subjects aged 12 to 17 (OR 1.16; 95%

CI 0.83 to 1.61), nor among those aged 18 to 24 (OR 1.63; 95% CI 0.70 to 3.79) (Figure 3(a)

and Table 3). There was, however, evidence (p=0.001) of a reduction in past-year use of

methamphetamine among those aged 12 to 17 (OR 0.59; 95% CI 0.43 to 0.81), whilst there

was no evidence of a similar effect among those aged 18 to 24 (OR 0.70; 95 % CI 0.34 to 1.45)

(Figure 3(b) and Table 3).

One ITS (n=6,371) showed evidence of effectiveness for past 30-day use of marijuana among

high sensation seekers (p=0.001 for the Fayette sample, p=0.001 for the first campaign in the

Knox sample, p=0.002 for the second campaign in the Knox sample).[25]

One CBA study found an increase in use of LSD (‘p<0.001’) (Table 3) whilst no evidence

(‘p>0.05’) of differences were found for marijuana, cocaine, amphetamine, and heroin.[21]

Page 12 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Intention to Use Drugs

Experimental studies. In one meta-analysis of 4 randomized controlled studies involving 1,270

subjects, there was no evidence of an effect (p=0.21) of media campaigns in changing

intention to use drugs (SMD -0.07; 95% CI -0.19 to 0.04) (Figure 2 and Table 3).[19,26–28]

There was no evidence (p=0.840) of heterogeneity across studies.

Field studies. One study (n=2,915) found some evidence (p=0.053) of a reduction in intentions

to use marijuana (OR 0.89; 95% CI 0.79 to 1.00) (Table 3) for the first version of the ONDCP’s

media campaign.[23]

Discussion

Mass-media campaigns are commonly used throughout the world to tackle a broad array of

preventable risk factors or injuries. Such campaigns are seldom evaluated, thus making it

difficult to inform policymakers regarding their effectiveness and sustainability. In this

panorama of overall uncertainty, mass-media campaigns tackling tobacco and traffic

accidents are noteworthy exceptions as they have been evaluated more frequently and have

shown some evidence for benefit.[30] In our attempt to summarize evidence on the

effectiveness of mass-media campaigns targeting illicit drugs, we included 19 studies

evaluating a number of heterogeneous interventions. We grouped interventions according to

whether they were evaluated with studies conducted in experimental settings in which

participants were aware of being exposed to media interventions; or were assessed with

Page 13 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



studies carried out in a field environment which are more likely to show the real-life effects of

large national media campaigns—but are also more prone to risk of bias.

Findings appear to vary considerably according to the type of intervention and study design.

Pooled analyses of 8 interventions evaluated in an experimental setting provided no evidence

of beneficial effects for use or intention to use illicit drugs, an indicator of possible future

behaviour.[31,32] Four interventions evaluated with 8 field studies revealed some evidence of

beneficial effects: (i) the revamped campaign by the Office of National Drug Control Policy

(ONDCP) called Above the Influence, which was found effective in one study and effective

among eighth-grade girls in another study; (ii) the Be Under Your Own Influence media-

community intervention; (iii) the Meth Project campaign, which was found effective on past-

year methamphetamine use, although only among adolescents aged 12 to 17 years old; and

(iv) the U.S. televised anti-marijuana campaigns broadcasted in Fayette County (Lexington),

Kentucky, and in Knox County (Knoxville), Tennessee, which were found to be effective on

high-sensation seekers. Two mass media campaigns showed clear iatrogenic effects; most

notably, the first version of the ONDCP’s media campaign My Anti Drug, which was evaluated

by 2 studies and was found to increase use of marijuana. An adverse effect was also found

for a media-community intervention evaluated by a CBA study which provided evidence of

increased frequency of LSD use.

No characteristic clearly emerged as core feature of successful or unsuccessful campaigns,

neither regarding their explicit or implicit theoretical background nor their communication

strategies. However, it is worth noting that two out of the four interventions providing

Page 14 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



evidence of effectiveness, the ONDCP’s Above the Influence national campaign and the Be

Under Your Own Influence media-community intervention, promoted non-use of drugs as a

way to support the goals of autonomy and achievement of competence, which have been

both conceptualized as innate psychological needs that persist over the lifespan.[22] Among

the interventions which provided evidence of harmful effects, the first version of the ONDCP’s

media campaign My Anti Drug was based on a social marketing approach which emphasized

resistance skills, self-efficacy, normative education, and negative consequences of drug

use.[33] These mediators are suspected to have increased the perception of prevalence of

drug use in the target population.[34]

An important reason of the weak evidence obtained by this review is the large variation in

mass-media intervention type and study design. Similar interventions were often evaluated

with different study designs whilst different interventions were sometimes evaluated with the

same study design. Pooled analyses could thus be undertaken only for a few similar

interventions evaluated with the same study design, and such small sets of pooled studies did

not allow carrying out sensitivity analyses. We did not set any time or language constraints to

our search, accepted all types of controlled study designs, and obtained unpublished data by

establishing direct contact with the authors of the original papers. Unfortunately, due to the

paucity and inconsistency of available evidence we cannot draw general conclusions as to

whether media campaigns are effective in preventing the use or the intention to use illicit

drugs. This observation is in line with the findings of similar reviews that used more restrictive

inclusion criteria.[30,35]

Page 15 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



The evidence base accrued so far on media campaigns targeting illicit drugs allows us to make

at least two remarks. First, such campaigns can be evaluated—a fact that is often questioned

in several parts of the world—and properly conducted evaluation studies can provide benefits

to both research and practice. Second, in the worst-case scenario, media campaigns can be

not only ineffective, but even harmful. Contrary to common belief, antidrug media

campaigns may be damaging and their dissemination is ethically unacceptable without a prior

assessment of their effects.[36,37] New campaigns should be implemented in the framework

of rigorous evaluation studies, ideally in field settings with cohort or ITS study designs. A

better understanding of which media interventions work best is likely to result in a more

effective prevention of drug use and increased efficiency in the management of public

resources.

Page 16 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Acknowledgements

We are grateful to the Cochrane Drug and Alcohol Group for their support in publishing the

Cochrane review from which this paper has been extracted.[5] We thank Gregor Burkhart and

V. Anna Gyamathy for their input to the drafting of the protocol for this review, and Marie-

Christine Ashby and Fozia Hamid for carefully proofreading the manuscript. We are grateful to

Ling Fang, John Horan, Nicola Newton, Philip Palmgreen, Joan Polansky, Michael Slater,

Jennifer Stagnaro, Violeta Taneva, and Marco Yzer for their useful advice and for providing us

with their studies' unpublished data whenever possible.

Contributorship statement

EA structured and drafted this paper. MF and FF conceived the systematic review from which

this paper originates and overviewed the inclusion of studies and their methodological

assessment. MF and EA selected the studies for inclusion. EA and AB extracted the data from

the studies and contributed to writing the review. EA contacted study authors. EA and AB did

the meta-analysis of randomized controlled trials. AG and EA conducted the meta-analysis of

interrupted time-series studies.

All authors regularly discussed each step of review process and equally participated in each

decision regarding the studies and the analysis. All authors revised the paper. All authors read

and approved the final version of this manuscript.

Page 17 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Competing interests

We declare no competing interests.

Funding

No specific funding was available for this review and review protocol. All authors received

funding directly from their respective institutions, which allowed them to conduct this study

during office hours and to use working resources including IT equipment and journal access.

Data sharing

Data and statistical code used for pooled analyses are available and can be requested from

the corresponding author (Elias Allara, [email protected]).

Page 18 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



References

1 Sumnall HR, Bellis MA. Can health campaigns make people ill? The iatrogenic potential

of population-based cannabis prevention. J Epidemiol Community Health 2007;61:930–

1. doi:10.1136/jech.2007.060277

2 Carpenter CS, Pechmann C. Exposure to the Above the Influence antidrug

advertisements and adolescent marijuana use in the United States, 2006-2008. Am J

Public Health 2011;101:948–54. doi:10.2105/AJPH.2010.300040

3 Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews

and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535.

doi:10.1136/bmj.b2535

4 Ferri M, Burkhart G, Allara E, et al. Media campaigns for the prevention of illicit drug

use in young people. Cochrane database Syst Rev Published Online First: 2011.

doi:10.1002/14651858.CD009287

5 Ferri M, Allara E, Bo A, et al. Media campaigns for the prevention of illicit drug use in

young people. Cochrane database Syst Rev 2013;6:CD009287.

doi:10.1002/14651858.CD009287.pub2

6 Brinn MP, Carson K V, Esterman AJ, et al. Mass media interventions for preventing

smoking in young people. Cochrane database Syst Rev 2010;:CD001006.

doi:10.1002/14651858.CD001006.pub2

7 Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions.

2011.www.cochrane-handbook.org

8 Scottish Intercollegiate Guidelines Network. Critical Appraisal: Notes And Checklists.

2010.http://www.sign.ac.uk/methodology/checklists.html (accessed 12 Sep2010).

9 Cochrane Effective Practice and Organisation of Care Group. Draft EPOC Methods

Paper Including Interrupted Time Series (ITS) Designs in a EPOC Review. 1998.

10 Review Manager (RevMan). 2014.

11 Slater MD, Kelly KJ, Edwards RW, et al. Combining in-school and community-based

media efforts: reducing marijuana and alcohol uptake among younger adolescents.

Health Educ Res 2006;21:157–67. doi:10.1093/her/cyh056

12 Newton NC, Teesson M, Vogl LE, et al. Internet-based prevention for alcohol and

cannabis use: final results of the Climate Schools course. Addiction 2010;105:749–59.

doi:10.1111/j.1360-0443.2009.02853.x

Page 19 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



13 The Meth Project. Colorado Meth: Use & Attitudes Survey 2011. 2011.

14 The Meth Project. Georgia Meth: Use & Attitudes Survey 2011. 2011.

15 The Meth Project. Idaho Meth: Use & Attitudes Survey 2010. 2010.

16 The Meth Project. Hawaii Meth: Use & Attitudes Survey 2011. 2011.

17 The Meth Project. Wyoming Meth: Use & Attitudes Survey 2011. 2011.

18 Gilmour S, Degenhardt L, Hall W, et al. Using intervention time series analyses to assess

the effects of imperfectly identifiable natural events: a general method and example.

BMC Med Res Methodol 2006;6:16. doi:10.1186/1471-2288-6-16

19 Fang L, Schinke SP, Cole KCA. Preventing substance use among early Asian-American

adolescent girls: initial evaluation of a web-based, mother-daughter program. J Adolesc

Health 2010;47:529–32. doi:10.1016/j.jadohealth.2010.03.011

20 Schwinn TM, Schinke SP, Di Noia J. Preventing drug abuse among adolescent girls:

outcome data from an internet-based intervention. Prev Sci 2010;11:24–32.

doi:10.1007/s11121-009-0146-9

21 Miller WR, Toscova RT, Miller JH, et al. A theory-based motivational approach for

reducing alcohol/drug problems in college. Health Educ Behav 2000;27:744–59.

22 Slater MD, Kelly KJ, Lawrence FR, et al. Assessing media campaigns linking marijuana

non-use with autonomy and aspirations: ‘Be Under Your Own Influence’ and ONDCP’s

‘Above the Influence’. Prev Sci 2011;12:12–22. doi:10.1007/s11121-010-0194-1

23 Hornik R. Personal Influence and the Effects of the National Youth Anti-Drug Media

Campaign. Ann Am Acad Pol Soc Sci 2006;608:282–300.

doi:10.1177/0002716206291972

24 Scheier LM, Grenard JL. Influence of a nationwide social marketing campaign on

adolescent drug use. J Health Commun 2010;15:240–71.

doi:10.1080/10810731003686580

25 Palmgreen P, Donohew L, Lorch EP, et al. Television campaigns and adolescent

marijuana use: tests of sensation seeking targeting. Am J Public Health 2001;91:292–6.

26 Polansky JM, Buki LP, Horan JJ, et al. The Effectiveness of Substance Abuse Prevention

Videotapes with Mexican American Adolescents. Hisp J Behav Sci 1999;21:186–98.

doi:10.1177/0739986399212005

27 Yzer MC, Cappella JN, Fishbein M, et al. The effectiveness of gateway communications

in anti-marijuana campaigns. J Health Commun 2003;8:129–43.

doi:10.1080/10810730305695

Page 20 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



28 Zhao X, Sayeed S, Cappella J, et al. Targeting norm-related beliefs about marijuana use

in an adolescent population. Health Commun 2006;19:187–96.

doi:10.1207/s15327027hc1903_1

29 Lee CM, Neighbors C, Kilmer JR, et al. A brief, web-based personalized feedback

selective intervention for college student marijuana use: a randomized clinical trial.

Psychol Addict Behav 2010;24:265–73. doi:10.1037/a0018859

30 Wakefield MA, Loken B, Hornik RC. Use of mass media campaigns to change health

behaviour. Lancet 2010;376:1261–71. doi:10.1016/S0140-6736(10)60809-4

31 Litchfield R, White K. Young Adults’ Willingness and Intentions to use Amphetamines :

An Application of the Theory of Reasoned Action. E-Journal Appl Psychol 2006;2:45–51.

doi:10.7790/ejap.v2i1.34

32 Olds RS, Thombs DL, Tomasek JR. Relations between normative beliefs and initiation

intentions toward cigarette, alcohol and marijuana. J Adolesc Health 2005;37:75.

doi:10.1016/j.jadohealth.2004.09.020

33 Hornik R, Jacobsohn L, Orwin R, et al. Effects of the National Youth Anti-Drug Media

Campaign on youths. Am J Public Health 2008;98:2229–36.

doi:10.2105/AJPH.2007.125849

34 Hornik R, Jacobsohn L. The best laid plans: disappointments of the National Youth Anti-

Drug Media Campaign. LDI Issue Brief;14:1–4.

35 Werb D, Mills EJ, Debeck K, et al. The effectiveness of anti-illicit-drug public-service

announcements: a systematic review and meta-analysis. J Epidemiol Community Health

2011;65:834–40. doi:10.1136/jech.2010.125195

36 Faggiano F, Giannotta F, Allara E. Strengthening Prevention Science to Ensure

Effectiveness of Intervention in Practice: Setting up an International Agenda. In:

Sloboda Z, Petras H, eds. Defining Prevention Science. Boston, MA: : Springer US 2014.

597–615. doi:10.1007/978-1-4899-7424-2_26

37 Faggiano F, Allara E, Giannotta F, et al. Europe needs a central, transparent, and

evidence-based approval process for behavioural prevention interventions. PLoS Med

2014;11:e1001740. doi:10.1371/journal.pmed.1001740

Page 21 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Table 1. Characteristics of Included Studies

Study Underpinning

theory Design (goal)a

Interventionb Comparisonc Primary outcome

Secondary outcome(s) Analysis sample

Follow-up (total time) [months]

Polansky 1999[26] Decision theory RCT (E) PSA Other –

Intention to use; attitudes; knowledge and

disposition to select socially appropriate

responses

312 n.a. (n.s.)

Miller 2000[21] Self regulation theory

CBA (F) PSA; printed No intervention Use of drugs

(incl. cannabis and cocaine)

Risk perception ; problems related to drug

use 1,024 12 (18)

Palmgreen 2001[25] Influence of sensation-seeking on drug use

ITS (F) PSA No intervention Past-30-day use

of marijuana – 6,371 n.a. (32)

Yzer 2003[27]

Theories of behavioral change: persuasion effects

RCT (E) PSA No

intervention; other

–

Intention to use marijuana; attitude; perceptions about

marijuana

418 n.a. (n.s.)

Slater 2006[11]

Social-ecological framework (norms and expectations influence drug use)

RCT (E) PSA; printed No intervention Lifetime and

past-30-day use of marijuana

– 4,216 24 (42)

Zhao 2006[28] Normative beliefs RCT (E) PSA No intervention – Intention to use; beliefs

towards marijuana; social norms

435 n.a. (n.s.)

Hornik 2006[23] Unclear Cohort

(F) PSA; printed;

internet Lower

exposure

Lifetime, past-year, and past-30-day use of

marijuana

Intention to use; attitudes and self

efficacy; perceptions and social norms

8,117 56 (58)

Scheier 2010[24] Social marketing Cohort

(F) PSA; printed;

internet Lower

exposure

Past-12-month cannabis

intoxications – 2,515 n.a. (48)

Schwinn 2010[20] Social learning theory

RCT (E) Internet No intervention Past-30-day

substance use – 236 6 (n.s.)

Lee 2010[29] Readiness to change

RCT (E) Internet No intervention Past-90-day use

of marijuana Intention to change

marijuana use; consequences

341 6 (n.s.)

Fang 2010[19] Family-oriented RCT (E) Internet No intervention Past-30-day use

of marijuana Intention to use

marijuana 216 6.25 (16)

Newton 2010[12] Social influence approach

RCT (E) Internet Other Use of cannabis Cannabis knowledge;

attitudes; related harms 724 12 (21)


Perception of risk and perception of social disapproval are correlated with drug consumption

4 ITS and 1 CBA

(F)


No intervention

Past-30-day use of

methamphetamine

Attitudes on methamphetamine and

other drugs; perceptions; information sources and advertising awareness;

26,405

n.a. (Colorado

26; Georgia

18; Hawaii 25;

Idaho 40; Wyoming

34)

Page 22 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Slater 2011[22]

Autonomy and aspiration perceptions as mediators marijuana use

RCT (E); Cohort

(F) PSA; printed

Lower exposure

Lifetime, past-90-day and past-

30-day use of marijuana

Autonomy and aspiration inconsistent with marijuana use

3,236 24 (42)

Carpenter 2011[2]

Unclear; evaluated many heterogeneous mass-media campaigns

ITS (F) PSA; printed;

internet Lower

exposure

Past-30-day and lifetime use of

marijuana – 130,245 n.a. (36)

n.a. = not applicable. n.s. = not specified. a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. E = experimental/efficacy setting. F = field/effectiveness setting. b PSA = public service announcement (e.g., television/radio). c Other = other intervention or different combination of same intervention. Lower exposure = lower exposure to same intervention.

Page 23 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Table 2. Risk of Bias of Included Studies

Designa Study Random

sequence generation

Allocation concealment

Blinding of outcome

assessment Attrition

Selective reporting

Comparability of groups

Acceptance among

recruited

Attrition by exposure

status

Strategies alternative to blinding

Discussion of potential

confounders

Statistical accuracy

Overall risk of bias

(cohort)

Sufficient data points

for inference

RC

T

Polansky 1999[26]

Yzer 2003[27]

Slater 2006[11]

Zhao 2006[28]

Schwinn 2010[20]

Lee 2010[29]

Fang 2010[19]

Newton 2010[12]

Coh

ort Slater 2011[22] b,c

Hornik 2006[23] b

Scheier 2010[24] b

ITS

Palmgreen 2001[25] d

Carpenter 2011[2] d

4 Meth Project studies[13,15–17] d

CBA 1 Meth Project study[14] d Miller 2000[21] d

████ = low risk of bias. ████ = unclear risk of bias. ████ = high risk of bias. White = not applicable Study quality was appraised with three different tools depending on study design. Redundant or similar items were collapsed. The 'the intervention was independent of other changes' item of the ITS checklist was considered equivalent to the 'discussion of potential confounder' item for cohort studies, the 'formal test for trend' ITS item was considered equivalent to the 'statistical accuracy' item for cohort studies, and the 'completeness of dataset' ITS item was considered equivalent to the 'attrition' item for RCT and cohort studies. a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. b All cohort studies had low risk of bias for the following items: 'likelihood of outcome already present at enrolment', 'clarity of outcome', 'reliability of assessment of exposure', 'use of other sources to corroborate outcome measure', and 'multiple measure of exposure'. c Slater 2011 is a mixed RCT-cohort study. d All ITS studies and the CBA study had low risk of bias for the following items: 'intervention unlikely to affect data collection', and 'reliable primary outcome measure(s)'.

Page 24 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Table 3. Main Findings for Use or Intention to Use Illicit Drugs

Pooling Outcome Designa References Sub-groups No. of

subjects exp vs. ctrlb

Effect measure

Effect size (95% CI) or effect direction (p-

value)c

Heterogeneity p-valued

Poo

led

anal

yses

Use of illicit drugs RCT

Slater 2006[11]; Fang 2010[19]; Newton 2010[12]; Schwinn 2010[20]; Lee 2010[29]

– 2,701 vs.

2,769

SMD, random effects

-0.02 (-0.15 to 0.12) 0.020 *

Intention to use illicit drugs

RCT

Polansky 1999[26]; Yzer 2003[27]; Zhao 2006[28]; Fang 2010[19]

– 771 vs. 499 SMD, fixed

effects -0.07 (-0.19 to 0.04) 0.840

Past-month use of methamphetamine

4 ITS and 1 CBA


age 12-17 14,865 vs.

7,497

OR, random effects

1.16 (0.83 to 1.61) –

age 18-24 347 vs. 632 OR,

random effects

1.63 (0.70 to 3.79) –

Past-year use of methamphetamine

4 ITS and 1 CBA


age 12-17 17,105 vs

7,497

OR, random effects

0.59 (0.43 to 0.81) ** –

age 18-24 1,039 vs. 632 OR,

random effects

0.70 (0.34 to 1.45) –

Sin

gle

stud

ies

Lifetime, past-90-day, or past-30-day use of marijuana

RCT (community-

media) Slater 2011[22]

–

n.a. (3,236)

OR, random effects

0.60 (0.38 to 0.94) * –

Cohort (mass-media)

– OR,

random effects

0.26 (0.19 to 0.35) *** –

Past-year use of marijuana

Cohort Hornik 2006[23] – n.a. (3,529)

OR, fixed effects

1.21 (1.19 to 1.65) * –

Intention to use marijuana

– n.a. (2,915) OR, fixed

effects 0.89 (0.79 to 1.00) § –

Past-12-month episodes of cannabis intoxication

Cohort Scheier 2010[24]

age 13-14

n.a. (2,515)

mean difference,

SEM -0.022 –

age 15-18 mean

difference, SEM

0.144 * –

Past-30-day use of marijuana among high sensation seekers

ITS Palmgreen 2001[25]

Fayette n.a. (3,174) test for slope

↓ (p=0.001) –

Knox, first campaign

n.a. (3,197)

test for slope

↓ (p=0.001)

Knox, second campaign

test for slope

↓ (p=0.002) –

Past-30-day use of marijuana (girls, 8th grade)

ITS Carpenter 2011[2] – n.a. (130,245) OR, fixed

effects 0.67 (0.52 to 0.87) ** –

Frequency of use of 10 types of drugs

CBA Miller 2000[21] – 567 vs 431 mean

difference, ANOVA

for LSD: ↑ (p<0.001) for marijuana, cocaine,

amphetamine, and heroin: 'no longer

significant' differences

–

§ < 0.10 * p < 0.05 ** p < 0.01 *** p < 0.001 OR = odds ratio SMD = standardized mean difference SEM = structural equation modeling a ITS = interrupted time-series. RCT = randomized controlled trial. CBA = controlled before and after. Cohort = prospective cohort. b n.a. = breakdowns of students exposed to the interventions were not available. Number of analyzed subjects is between brackets. c Whenever the effect size was not reported, ↓ = decreased use or intention to use, and ↑ = increased use or intention to use. d Heterogeneity test for meta-analyses of RCTs.

Page 25 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





Figure 2. Pooling of Randomized Controlled Trials


Figure 3. Pooling of the Meth Project Interrupted Time-Series studies: Predicted and Observed Probabilities


Page 26 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from




Appendix 2. Quality Appraisal

Appendix 3. PRISMA Checklist

Page 27 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from




121x116mm (300 x 300 DPI)

Page 28 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Figure 2. Pooling of Randomized Controlled Trials 423x245mm (300 x 300 DPI)

Page 29 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Figure 3. Pooling of the Meth Project Interrupted Time-Series studies: Predicted and Observed Probabilities 419x550mm (300 x 300 DPI)

Page 30 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from




Note: search strategies have already been described in detail previously [5].

(a) PubMed (MEDLINE) Search Strategy

Search Query Items found

#16 Search (((#3) AND #4) AND #11) AND #15 5877

#15 Search ((#12) OR #13) OR #14 3,041,802

#14 Search adolescen*[tiab] OR preadolescen*[tiab] OR child*[tiab] OR teen*[tiab] OR youth*[tiab] OR young[tiab] OR kid*[tiab] OR juvenile*[tiab] OR minors[tiab] OR boy*[tiab] OR girl*[tiab]

1,662,519

#13 Search "Child"[Mesh] 1,457,004

#12 Search "Adolescent"[Mesh] 1,498,465

#11 Search ((((#5) OR #7) OR #8) OR #9) OR #10 797,788

#10

Search media[tiab] OR Communication*[tiab] OR audiovisual[tw] OR telecommunication*[tw] OR Educat*[tiab] OR radio[tw] OR television[tw] OR TV[tiab] OR internet[tw] OR campaign*[tw] OR advert*[tw] OR twitter[tw] OR facebook[tw] OR "instant messaging"[tw]

751,996

#9 Search "Telecommunications"[Mesh] 54,815

#8 Search Videotape Recording[Mesh] 9970

#7 Search "Internet"[Mesh] 43,359

#5 Search "Mass Media"[Mesh] 37,325

#4

Search "heroin"[Mesh] OR heroin[tiab] OR "Street Drugs"[Mesh] OR "Designer Drugs"[Mesh] OR "Crack Cocaine"[Mesh] OR "Lysergic Acid Diethylamide"[Mesh] OR drug*[tiab] OR polydrug[tiab] OR substance[tiab] OR hallucinogen*[tw] OR cocaine[tw] OR amphetamine*[tw] OR "lysergic acid diethylamide"[tw] OR LSD [tiab] OR ketamine[tw] OR cannabis[tw] OR marihuana[tw] OR marijuana[tiab] OR hashish[tw] OR steroid*[tw] OR morphine[tiab] OR ecstasy[tw] OR MDMA[tw] OR benzodiazepine[tw]

1,136,251

#3 Search (#1) OR #2 1,812,638

#2 Search abus*[tiab] OR consumption[tiab] OR misus*[tiab] OR use*[tiab] OR addict*[tiab] OR disorder*[tiab]

1,570,344

#1 Search "Substance-Related disorders"[Mesh] 344,574

(b) EMBASE Search Strategy

ID Query

#1 'substance abuse'/exp

#2 'drug abuse'/exp

#3 abus*:ab,ti OR consumption:ab,ti OR misus*:ab,ti OR use*:ab,ti OR addict*:ab,ti OR disorder*:ab,ti

#4 #1 OR #2 OR #3

#5

heroin:ab,ti OR drug*:ab,ti OR polydrug:ab,ti OR substance:ab,ti OR hallucinogen*:ab,ti OR cocaine:ab,ti OR amphetamine*:ab,ti OR 'lysergic acid diethylamide':ab,ti OR lsd:ab,ti OR ketamine:ab,ti OR cannabis:ab,ti OR marihuana:ab,ti OR marijuana:ab,ti OR hashish:ab,ti OR steroid*:ab,ti OR morphine:ab,ti OR ecstasy:ab,ti OR mdma:ab,ti OR benzodiazepine:ab,ti

#6 'diamorphine'/exp

#7 'designer drug'/exp

#8 'street drug'/exp

#9 'cocaine'/exp

Page 31 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



#10 'cannabis smoking'/exp

#11 #5 OR #6 OR #7 OR #8 OR #9 OR #10

#12 'mass medium'/exp

#13 'internet'/exp

#14 'videorecording'/exp

#15 'telecommunication'/exp

#16 media:ab,ti OR communication*:ab,ti OR audiovisual:ab,ti OR telecommunication*:ab,ti OR educat*:ab,ti OR radio:ab,ti OR television:ab,ti OR tv:ab,ti OR internet:ab,ti OR campaign*:ab,ti OR advert*:ab,ti OR twitter:ab,ti OR facebook:ab,ti

#17 #12 OR #13 OR #14 OR #15 OR #16

#18 'adolescent'/exp

#19 'child'/exp

#20 adolescen*:ab,ti OR preadolescen*:ab,ti OR child*:ab,ti OR teen*:ab,ti OR youth*:ab,ti OR young:ab,ti OR kid*:ab,ti OR juvenile*:ab,ti OR minors:ab,ti OR boy*:ab,ti OR girl*:ab,ti

#21 #18 OR #19 OR #20

#22 #4 AND #11 AND #17 AND #21 AND [embase]/lim

(c) ProQuest Search Strategy

(media campaigns OR mass media) AND illicit drug* AND preventi*.

(d) CENTRAL Search Strategy

ID Search Hits

#1 MeSH descriptor: [Substance-Related Disorders] explode all trees 10,355

#2 ((stimulant* or polydrug* or drug* or substance) near/3 (abuse* or abusing or consumption or addict* or disorder* or intoxicat* or misus* or use*)):ti,ab

14,750

#3 (abuse* or abusing or consumption or addict* or disorder* or intoxicat* or misus* or use*):ti,ab 198,966

#4 MeSH descriptor: [Narcotics] explode all trees 681

#5 heroin:ti,ab 762

#6 MeSH descriptor: [Street Drugs] explode all trees 196

#7 MeSH descriptor: [Amphetamine] explode all trees 632

#8 (amphetamine* or dextroamphetamine* or methamphetamine or Methylamphetamine*):ti,ab,kw (Word variations have been searched)

1442

#9 (ecstasy or MDMA or hallucinogen*):ti,ab,kw (Word variations have been searched) 234

#10 MeSH descriptor: [Cocaine] explode all trees 576

#11 (crack or cocaine):ti,ab,kw (Word variations have been searched) 1953

#12 MeSH descriptor: [Cannabis] explode all trees 245

#13 (cannabis or marijuana or marihuana or Hashish):ti,ab,kw (Word variations have been searched)

1158

#14 (Lysergic next Acid):ti,ab,kw 76

#15 LSD:ti,ab,kw (Word variations have been searched) 131


6370


6370

#18 #4 or #5 or #6 or #7 or #8 or #9 or #10 or #11 or #12 or #13 or #14 or #15 or #16 or #17 11,919

#19 #3 and #18 6547

#20 #1 or #2 or #19 26,077

Page 32 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



#21 MeSH descriptor: [Mass Media] explode all trees 1337

#22 MeSH descriptor: [Internet] explode all trees 1248

#23 MeSH descriptor: [Videotape Recording] explode all trees 790

#24 "Tv":ti,ab,kw (Word variations have been searched) 386

#25 (media or communication* or audiovisual or telecommunication* or radio or television or internet or campaign* or advert* or twitter or facebook) (Word variations have been searched)

27,766

#26 #21 or #22 or #23 or #24 or #25 28,828

#27 MeSH descriptor: [Adolescent] explode all trees 68,885

#28 adolescen* or preadolescen* or child* or teen* or youth* or young or kid* or juvenile* or minors or boy* or girl*:ti,ab,kw (Word variations have been searched)

157,753

#29 #27 or #28 157,753

#30 #20 and #26 and #29 566

Page 33 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from



Appendix 2. Details on Quality Appraisal

(a) Assessment of Randomized Controlled Trials

The recommended approach for assessing risk of bias in studies included in Cochrane Reviews is a two-part

tool, addressing (a) sequence generation and allocation concealment, (b) blinding of participants and

providers, (c) blinding of outcome assessor, (d) incomplete outcome data, (e) selective outcome reporting,

and (f) other source of bias. Blinding of participants is not always applicable for mass media campaigns, and

we therefore considered blinding of personnel and outcome assessors. A study was deemed to have low

risk of bias for blinding if the data were obtained with a questionnaire administered anonymously or by

computer.

(b) Assessment of Cohort Studies

This tool comprises several questions intended to serve as guidance for the study assessors, regarding (a)

blinding of outcome assessment; (b) clarity of outcome; (c) likelihood of outcome already present at

enrolment; (d) use of other sources to corroborate outcome measure; (e) reliability of assessment of

exposure; (f) multiple measure of exposure; (g) attrition; (h) comparability of groups; (i) acceptance among

recruited; (j) attrition by exposure status; (k) strategies alternative to blinding; (l) discussion of potential

confounders; (m) statistical accuracy, and (n) overall risk of bias.

(c) Assessment of interrupted time series (ITS) and before and after (CBA)

studies

This tool requires assessing whether there is presence of (a) an intervention independent of other changes;

(b) sufficient data points to enable reliable statistical inference; (c) formal tests for trend; (d) an

intervention unlikely to affect data collection; (e) blinded assessment of primary outcome(s); (f)

completeness of dataset; (g) reliable primary outcome measures.

Page 34 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



jopen.bmj.com

/B

MJ O


eptember 2015. D

ownloaded from





TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both. 1

ABSTRACT

Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

3

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known. 5,6

Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

6

METHODS

Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.

6

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,

language, publication status) used as criteria for eligibility, giving rationale.

7

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

6

Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

Appendix 1

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

7

Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

7, 8, Appendix 2

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

7, 8, Appendix 2, protocol

Risk of bias in individual studies

12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

7, 8, Appendix 2

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). 8

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency

(e.g., I2) for each meta-analysis.

8, 9

Page 35 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960





Page 1 of 2


Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

N/A

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified.

8, 9

RESULTS

Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.

9, Figure 1

Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.

9, 10, Table 1

Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). 10,11,Table 2

Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.

Table 3, Figure 2, Figure 3

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. 11-13,Table 3

Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). 10, Table 2

Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). 12, Figure 3, Table 3

DISCUSSION

Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).

13, 14

Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).

15

Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. 16

FUNDING

Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.

N/A

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097

For more information, visit: www.prisma-statement.org.

Page 2 of 2

Page 36 of 36


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960



are mass-media campaigns effective in preventing drug use? a … · 13 change. secondly, mass-media...

Documents