Chapter 9: Evaluating intelligence-led policing

Evaluation is key to crime control

Five features of a rational approach to crime control: Adequate investment in measuring and monitoring Open access to crime and justice information Reliance on evidence in the development of policy Commitment to rigorous evaluation A flexible and eclectic approach to control

Weatherburn, D. (2004) Law and Order in Australia: Rhetoric and Reality (Sydney: Federation Press) pp.36-38.

Basics of evaluation

Did you get what you expected? Was the program implemented as expected? Did it crime reduce? If so…

Compared to what? What is the baseline ‘null hypothesis’ situation that makes

a good comparison?

Two structures to evaluation

InterpretInterpret ImpactImpact

InfluenceInfluence

Information structures

Two structures to evaluation

InterpretInterpret ImpactImpact

InfluenceInfluence

Organizational structures

Two types of evaluation

Outcome evaluation Tells you whether a process had the desired effect, i.e. that

crime was reduced

Process evaluation Can tell you why the outcome occurred. Examines the

operation to see why an observed outcome happened

Operation Vendas

Pilot informally assessed as successful, but Operation Vendas did not have desired impact

New South Wales, Australia Sought to increase the risk and speed of capture for

offenders by boosting the volume of forensic evidence collection and reducing the time to get samples analyzed

Process evaluation Found stated aim of crime scene examinations never

attained Discovered a lack of forensic resources Identified training problems

Operation Safe Streets

Philadelphia Police Department Placed officers on permanent post at over 200 drug

corners in the city Outcome evaluation found officers had a localized

dampening effect on crime Giannetti (2007) reports that

Officers began to take calls away from corners Foot patrols reverted to roving car patrols Incentive to arrest was removed and information flow to

detectives reduced

Volume and type of anonymous drug tip

Analytical skills for evaluation

Some skills that may be required Analytical dexterity Proficiency in non-parametric and regression interpretation Spatial analysis Interrupted time series analysis Crime mapping

‘quite soon, crime mapping will become as much an essential tool of criminological research as statistical analysis is at present’ Clarke, R.V. (2004) 'Technology, criminology and crime science', European Journal on Criminal Policy and Research, 10:1, pp. 60.

Maryland Scientific Methods Scale

Scale of zero (no confidence in the findings) to five (high confidence in the results)

A hierarchy of evaluation standards (top to bottom) Randomized, controlled, double-blind trials Quasi-experimental studies (experiments without

randomization) Controlled observational studies Observational studies without a control group Expert opinion!

Realistic evaluation

Pawson and Tilley’s realistic evaluation or scientific realist approach

Researchers should Investigate the relationships between context, mechanism

and outcome Study using more qualitative, narrative, and ethnographic

research techniques. Understand that the key is to clarify how the choices that

people make affect the outcome of the programs under examination

Operation Anchorage

Canberra, Australia – Australian Capital Territory (ACT)

February 2001 to end of June 2001 Significant problem with burglary Anchorage placed significant emphasis on

Senior leadership Targeting of recidivist offenders through crime and

intelligence analysis The development of joint operations across different

branches of ACT Policing

Operation Anchorage

Four teams of 10-12 investigators 6 police analysts New targets were circulated every two weeks Anchorage came on the heels of two relatively

unsuccessful operations, called Chronicle and Dilute

Weekly burglary frequency in the ACT

Economic and social costs of crime

In Australia, a 2003 estimate of the cost of burglary to society found a cost in Australian dollars of

AU$2,400 per burglary AU$2,000 per residential burglary AU$4,500 per non-residential burglary

Total saving for Operation Anchorage AU$7,125,600

AU$1,257,600 during Anchorage AU$5,868,000 benefit after Anchorage

Financial benefits of Operation Anchorage

Studying recidivists

232 people arrested during Anchorage 119 had committed at least one offence before 1999 Chart their aggregate number of days in prison or on

remand…

Incarceration rates

Impact of recidivist incarceration

Measuring success in different ways

In regard to major criminal and terrorist operations…

‘Not only are we expected to anticipate the next move, but we also have to do something about bringing those involved to justice. This highlights one of the fundamental differences between intelligence that aims to warn and prevent, and investigations for which success is measured by successful prosecution and conviction’

Mick Keelty, Australian Federal Police Commissioner, 2004

Cost-effectiveness of the use of CIs

Reward cost only Full cost

Cost for each arrest £54 ($87) £697 ($1,125)

Cost for each crime clearance £27 ($44) £348 ($561)

Value of property recovered to cost ratio

£34 ($55) to 1 £2.60 ($4.20) to 1

Adapted from Dunnighan and Norris (1999)

Operation Green Ice

Operation Green Ice DEA set up their own bank in a sting operation to tempt

drug traffickers into money laundering Undercover agents laundered US$20 million of Colombian

drug cartel money Led to arrest of seven of the Cali drug cartel’s top financial

managers, the seizure of more than US$50 million in assets worldwide, and the arrest of 177 people

United Nations estimated Cali cartel profits to be about US$30 billion a year

RCMP Disruption Attributes ToolCore business Financial Personnel

Attribute description

The instruments / process central to the criminal enterprise

Financial capacity / status of the group including profits / financial assets gained through organized crime

Individuals employed through organized crime

High Removed the capacity to supply/operate

Removed and/or interrupted the organization’s financial ability to mount their large scale operations

Arrested and/or charged individuals with the majority of the knowledge, contact, expertise, experience, and executive influence

Medium Interrupted production and/or distribution of supply network

Seizure / restraint of significant proceeds of crime relative to the financial scope of the organization

Arrested and/or charged support personnel/skilled operators with expertise, knowledge and contacts

Low Seized commodities without disrupting production and/or distribution

Seizure / restraints of minor proceeds of crime or interruption of the means to launder and/or legitimize proceeds of crime

Arrested and/or charged replaceable unskilled operators / street level operators / couriers

Nil No commodities seized

No profits / financial assets seized No individuals arrested

Performance anxiety

Some performance areas are so vague as to create huge numbers of performance measures.

UK government priority areas include reducing crime investigating crime promoting safety providing assistance citizen focus resource use local policing

Unintended consequences of measures

Tunnel vision Sub-optimization Myopia Measure fixation Misrepresentation Misinterpretation Gaming Ossification Demoralization Discreditability