Drug and alcohol prevention

How do we find out what works?

Many people worry about the use of drugs (including alcohol and tobacco) by young people

There are many interventions - in schools, families and the community - which claim to

reduce drug misuse.

?

?

How can we test whether they actually work?

?

?

Higher quality trials give more certainty about results

Best practice

Before and after data

Before and after data with a control group

Randomised controlled trial

Statistically matched control group

Percent drinking alcohol in past week

Measure outcomes before and after intervention

Before After0%

1%

2%

3%

4%

5%

3.5%4.0%

Intervention

What does this tell us?

Very little without a control group.

Before After0%

1%

2%

3%

4%

5%

3.5%4.0%

3.2%

4.7%Percent drinking alcohol in past week

With a control group...

Intervention

Control

Smaller increase in drinking in intervention group

What do we know about the control group?

The following methods give increasing confidence that the comparison is a fair one:

• Common sense – avoiding obvious pitfalls such as those trying out the intervention tending to be in more prosperous areas.

• Statistical ‘matching’ techniques using more information about individuals in the groups.

• Random allocation between the intervention and control groups: this is called a “Randomised controlled trial”

Randomised controlled trials (RCTs) are the ‘gold standard’ for measuring impact

Sample size matters• This is because a behaviour (for example drinking in

the past week) will vary between individuals across the wider population.

• If a sample of 1000 15 year olds in London are randomly selected and 200 (20%) drank in the past week, you can be confident that the true percentage for all 15 year olds in London is fairly close to 20%.

• In contrast, if you select ten young people at random and two of them drank in the past week, that doesn’t tell you much at all.

Statistical analysis• So, as well as looking at the size of the difference

between your intervention and control groups, the size of your sample is also important.

• Larger samples give more confidence that the difference between two groups is genuine and not due to random variation.

• Once it is calculated as being sufficiently unlikely (less than a 5% chance) that the result is due to random variation, the finding is said to be ‘statistically significant’.

Find out more...• The Centre for Analysis of Youth Transitions (CAYT) is

creating a repository of impact studies on young people’s services and programmes - http://www.ifs.org.uk/centres/caytRepository

• European Quality Standards have been developed for drug prevention and contain useful information on evaluating outcomes. http://www.emcdda.europa.eu/publications/manuals/prevention-standards

• ‘Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials’ is a more general discussion of RCTs in public policy http://bit.ly/MHJ4aP

www.mentoruk.org.uk@mentortweets

1st floor, 67 – 69 Cowcross StreetLondon EC1M 6PU

Registered Charity No: 1112339Registered Company No: 5609241Scottish Charity No: SC041210