formal experiments: randomisation + study size

Formal experiments: randomisation + study size

Concept of randomisation

• Biology, 1926: Sir Ronald Fisher

• Medicine, 1947: Sir Austin Bradford Hill Randomised Controlled Trial

• Criminal justice ?

Randomisation in medicine

• Toss of coin determines experimental or control treatment RCT assignment unpredictable

• Fair [=> ethical] allocation of scarce resource

• Balance treatment numbers overall, in each hospital, and for major prognostic factors

RCT Telephone Randomisation

Licensing of pharmaceuticals: requires efficacy in RCTs

• Patients’ informed consent [ethics]

• Sufficiently-large [precise answer]

• Randomised [unbiassed: like with like]

“What works” in UK criminal justice?

Large RCTsessentially untried . . .

(bar restorative justice)

Judges prescribe sentence on lesser evidence than doctors

prescribe medicines

Is

public

aware?

Drug Treatment &Testing Orders (DTTOs)

• England & Wales: 210 clients

• Scotland: 96 clients

• Targets for DTTO clients in E&W: 6,000+ per annum

• DTTO clients: 21,000+ by end 2003

RSS Court DTTO-eligible offenders: do DTTOs work ?

• Off 1 DTTO• Off 2 DTTO• Off 3 alternative =• Off 4 DTTO• Off 5 alternative =• Off 6 alternative =

Count offenders’ deaths, re-incarcerations etc . . .

UK courts’ DTTO-eligible offenders: ? guess

• Off 7 DTTO [ ? ]• Off 8 DTTO [ ? ]• Off 9 DTTO [ ? ]• Off10 DTTO [ ? ]• Off11 DTTO [ ? ]• Off12 DTTO [ ? ]• Off13 DTTO [ ? ]• Off14 DTTO [ ? ]

(before/after) Interviews versus . . . [ ? ]

Evaluations-charade• Failure to randomise

• Failure to find out about major harms

• Failure even to elicit alternative sentence funded guesswork on relative cost-effectiveness

• Volunteer-bias in follow-up interviews

• Inadequate study size re major outcomes . . .

Power (study size) matters!

Back-of-envelope sum for 80% power

e. g. Percentages

If MPs/social scientists don’t know,

UK plc keeps hurting

For 80% POWER, 5% significance: compare failure (re-conviction) rates

Randomise per treatment group, 8 times STEP 1 answer

STEP 1:

Success * fail rate + Success * fail rate

for new disposal for control ------------------------------------------------------------(new success rate – control success rate)2

DTTOs: target 60% versus control 70% re-conviction rate?

Randomise per ‘CJ disposal’ group,

8 times STEP 1 answer = 8 times 45 = 360

STEP 1 answer:

40 * 60 + 30 * 70 2400 + 2100 DTTOs control --------------------------------- = ---------------

(40 – 30)2 100

Four PQs for every CJ initiative• PQ1: Minister, why no randomised controls?

• PQ2: Minister, why have judges not even been asked to document offender’s alternative sentence that this CJ initiative supplants?

{cf electronic tagging}

• PQ3: What statistical power does Ministerial pilot have re well-reasoned targets?

{or, just kite flying . . .}

• PQ4: Minister, cost-effectiveness is driven by longer-term health & CJ harms, how are these ascertained? { database linkage}

Randomised controlled trials

to

police Policy

by

Home OfficePrisons

&

Criminal Justice

HMP Peterborough PILOT: Kalyx prison, Social Finance run, & payment by results.What happens?

HMP Peterborough Eligible Pre-release Inmate (serving less than 12 months)

Taken on by SF

Yes No

Transfer’dOut . . .

No Yes No Yes

Deselected

by SF

No Yes N/A or

Yes

AlreadyDone!

N/A

HMP Peterborough release

YES Yes, but . . .

NO Yes, but . . .

NO

Matchedcontrols?

? ?

HMP Peterborough PILOT: Kalyx prison, Social Finance run, & payment by results.

Per SF-release, comparators are? 10 same-sex ‘matched’ prisoners who also served

less than 12 months & were released on same day but from other prisons

{All Kalyx-run? Where? Functionality? Locality?De-selections & transfers?}

Reduce convictions in 1st year post-release by 7.5% . . . Conviction costed how . . . ???

{eg 60% to 55.5% convicted within 1 year of release ~ or reduce conviction-count by 7.5%}

Guardian Society: 17 Nov. 2004

“Some statisticians are so severe that they would stop social policy

making in its tracks.

For example, Bird would forbid the government to introduce any policy that had not been assessed through

controlled trials. . . ”

SIMPLE RANDOMISATION

STEP 1: Correspondence between random number (see tables) & CJ disposal:

EVEN random number (0) DTTO ODD random number alternative

STEP 2: Document starting point in tables & direction of reading: SMB = down

03 (row) 07 (column)

RANDOM NUMBER tables

72137 73850 32733

48083 50731 50584

16602 26772 81250 row 3, column 7:55480 29910 89693 read down77708 83761 47184

12601 54432

65664 73669

SIMPLE RANDOMISATION: down

Offender Random # Disposal Outcome Court & J

CJ01 6 DTTO CB2

CJ02 9 alternative CB2

CJ03 3 alternative CB2

CJ04 4 DTTO CB2

CJ05 3 alternative CB2

CJ06 etc CB2

CJ07 CB2

CJ08 CB2

Correspondence: random # & disposal

EVEN = DTTO

ODD = alternative

Row 03

Col 07

Randomisation by minimisation: next client = Cambridge, male,

18-24 years, >5 prison terms

Client RCT assignmentscharacteristic so far to DTTO alternative

Cambridge 20 15 *

male 205 190 *

18-24 years 100 * 108

>5 in-prison 180 * 185

SUM points: 505 498 **

RANDOMISE this client preferentially (eg 80:20) to ** because lower on points

Critical reading: BMJ, Lancet etc

Statistical guide-lines for contributors to medical journals (20+ years): beware “bars”

Structured ABSTRACT: essential design & primary outcome (s)

CONSORT flowchart for reporting RCTs: beware i) post-randomization exclusions from analysis;

ii) post-hoc subgroups.

STROBE guide-lines for reporting observational studies: beware bias in many guises

(especially: how explanatory variable is coded at analysis - eg binary!! & retrospective classification: deaths on transplant waiting list ~ count

survival contribution on the waiting list of transplantees beofre operation)