Forensic Vehicle Convoy AnalysisUsing ANPR Data

1 Haiyue Yuan, Shujun Li, Anthony T.S. Ho2 Paul Palmer, Andy Lloyd and 3 Graham Head

1Department of Computer Science, University of Surrey, UK2Surrey Police, UK 3Independent advisor, UK

1. Introduction

•Growing need for using Automated Number Plate Recog-nition (ANPR) data to detect and prevent crimes• 30 million ANPR reads per day across the UK (2015),

and lack of software tools enabling efficient automaticprocessing data•We are developing a digital forensic tool to identify un-

known and suspicious vehicles traveling in convoy ‘hid-den’ in ANPR data.– Enhance functionalities of current analysis tool– Combining data from multiple data sources– No known number plate is needed

2. Background information

2.1 Data sources

• ANPR data from Surrey Police: contains Vehicle Regis-tration Mark (VRM), date, time, camera ID, and cameraGPS location.

Table 1: Example of ANPR reads (VRMs and cameras’ geo-locationsare anonymised)

VRM DATE TIME CAM ID LATITUDE LONGITUDEA 01/01/2015 08:00:01 ANPR01.1 51.0000 1.1000B 01/01/2015 08:00:02 ANPR02.1 52.0000 1.2000C 01/01/2015 08:00:03 ANPR03.1 53.0000 1.3000

•Data from UK’s Driver and Vehicle Licensing Agency(DVLA): vehicle’s make, model, color, and tax code•Crime map from police.uk: public crime statistics•Goolge Map: travel time estimation

2.2 Concepts

• Single vehicle journey: describes all travel activities of asingle vehicle during a period of time.

00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 23:00

Cam 1 Cam 2 Cam 3 Cam 4 Cam 5 Cam 6

Journey

Figure 1: Illustration of a journey

• Single vehicle session: refers to a single vehicle continu-ously traveling for a period of time without a break.

00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 23:00

Cam 1 Cam 2 Cam 3 Cam 4 Cam 5 Cam 6Large time gap

Session 1 Session 2

Figure 2: Illustration of sessions

•Convoy session: two or more vehicles traveling togetherfor a period of time.

00:00 03:00 06:00 09:00 12:00 15:00 18:00 21:00 23:00

Cam 1 Cam 2 Cam 3 Cam 4 Cam 5 Cam 6

A

B

A

B

A

B

A

B

A

B

A

B

t1 t2 t3 t4 t5 t6

Figure 3: Illustration of a convoy session

3. Selected related work

• [1] uses data clustering techniques to extract and identifyunusual patterns of multi-vehicle convoy activities usingANPR data.• [2] investigates headway distance of two vehicles driving

together.• [3] investigates behavior of intentional following using

ANPR data.• [4] applies data mining techniques to discover vehicle ac-

tivity patterns from ANPR data.

4. Simple rule based algorithm

The work presented in this section is an extended edition ofa previous study [1].

4.1 Thresholds

• Session break threshold: the maximum time gap be-tween two adjacent ANPR reads of a single vehicle.

•Convoy session size threshold: the minimum numberof cameras that two vehicles passed together in a convoysession.

• Search window threshold: the maximum time differ-ence between two vehicles in a genuine convoy sessionwhile passing the same camera.

4.2 Algorithm design

• Journey extraction (Raw ANPR⇒ Journey data)

• Session extraction (Journey data⇒ Session data)

•Convoy session extraction (Session data⇒ Convoy ses-sions)

Journey Extraction

ANPR data

Journey data

Session data

Session Extraction

Convoy session

Convoy Session Extraction

Session break

Convoy size

Search window

Convoy size

INPUT

OUTPUT

Figure 4: Flow chart of the rule based algorithm

4.3 Post processing

•Detect repeated 2-vehicle convoy sessions

•Detect multi-vehicle convoy sessions

•Retrieve convoy sessions using make/model/color and/ortax code

•Retrieve convoy sessions that share the whole/part oftheir traveling routes

4.4 Results

Results are generated using session break threshold of 5hours, convoy session size threshold of 5, and search win-dow threshold of 30 seconds.

• Processed ANPR data for 185 days (around 200 millionreads in total)

• 35 repeated 2-vehicle convoy sessions

• 260 multi-vehicle convoy sessions

•Genuine convoy sessions confirmed by Surrey Police in-volving taxis, family vehicles, business vehicles, etc.

• Successfully detected a multi-vehicle convoy session cre-ated by our project partner Thales UK

5. Adaptive threshold based algorithm

• Aims to replace hard thresholds with adaptive ones usinghistorical ANPR data and public traffic data.

• Aims to adjust all parameters to control the number ofoutputs to make them manageable.

Journey Extraction

ANPR data

Journey data

Convoy extraction

Convoy session

Adaptive search window

INPUT

OUTPUT

Adaptive convoy sizeSession

extractionAdaptive session break

Travel time estimation

Convoy data

Time difference estimation

Session size estimation

Figure 5: Flow chart of the algorithm with adaptive thresholds

5.1 Adaptive session break threshold

This algorithm is used to estimate a distribution of traveltimes of all vehicles passing two ANPR cameras, giventhe time of a day and the day of a week. The idea isto use the estimated travel time distribution to determinethe session break threshold adaptively to define sessions.

0 1 2 3 4 5 6 7 8 9

x 104

0

0.5

1

1.5

2

2.5

3x 10

4

Travel time bins

Num

ber

of o

bser

vatio

ns

(a)

0 5 10 15 20 250

2000

4000

6000

8000

10000

Hours of a day

Num

ber

of v

ehic

les

trav

el fr

om A

to B

(b)

0 1 2 3 4 5 6

x 104

0

200

400

600

800

1000

1200

1400

1600

1800

Travel time bins

Num

ber

of o

bser

vatio

ns

(c)

0 2000 4000 6000 8000 10000 120000

50

100

150

200

250

Travel time bins

Num

ber

of o

bser

vatio

ns

(d)

Figure 6: (a) Distribution of travel times of all vehicles passing A andB on Monday (b) Distribution of the number of vehicles passing betweenA and B at different hours on Monday (c) Distribution of travel time of allvehicles passing between A and B from 08:00 to 09:00 on Monday (d)Distribution of travel time of all vehicles passing between A and B from21:00 to 22:00 on Monday

• Figure 6(a): the shorter travel times at the first peak rep-resent that vehicles pass A, turn around and pass B; Thelonger travel times at the second peak represent that ve-hicles leave early to work passing A, and return homeafter work passing B.• Figure 6(b): most of the vehicles are likely to pass A,

turn around and pass B within a short time window dur-ing night.

6. Micro-behaviors of drivers in convoysessions

6.1 Related driving behaviors

• Time difference (headway, daytime/night time, criminalbehavior)•Change order of driving (unusual overtaking behavior)•Change of lanes•Number of intervening vehicles

6.2 Machine learning based analysis

• Features based on micro-behaviors•Refinement and clustering of convoy sessions (see Fig-

ure 7).• Prioritising/Ranking convoy sessions based on behav-

ioral features

0 2 4 6 8 10 12 140

5

10

15

20

25

Standard deviation of time difference

Mean o

f tim

e d

iffe

rence

Cluster 1

Cluster 2

Centroid for cluster 2

Figure 7: Clustering convoy sessions using two featuresderived from time differences

7. Acknowledgments

This work was co-funded by the Innovate UK, under the Ap-plication Number 41698-293424. We also thank our projectpartners Thales UK Ltd for providing input to our work.

References

[1] A Homayounfar, ATS Ho, N Zhu, G Head & P Palmer, “Multi-vehicleconvoy analysis based on ANPR data,” in Proeedings of 4th Inter-national Conference on Imaging for Crime Detection and Prevention(ICDP 2011), 2011

[2] A Heino, HH van der Molen & GJS Wilde, “Differences in risk expe-rience between sensation avoiders and sensation seekers,” Person-ality and Individual Differences, 20(1):71-79, 1996

[3] B Gunay, “Using automatic number plate recognition technology toobserve drivers,” Journal of Advanced Transportation, 46(4):305-317, 2012

[4] Y Sun, X Zhou, L Sun & S Chen, “Vehicle Activity Analysis Based onANPR System,” in Proceedings of 12th IEEE International Confer-ence on Embedded and Ubiquitous Computing (ITQM 2014), Pro-cedia Computer Science, 31:48-57, 2014

DFRWS EU 2016, 29 March - 31 March 2016, Lausanne, Switzerland Corresponding Authors: 1{haiyue.yuan; shujun.li; a.ho}@surrey.ac.uk