hunchlab: spatial-temporal data mining - harvard university...hunchlab: spatial-temporal data mining...

HunchLab: Spatial-Temporal Data Mining Robert Cheetham

10 April 2009 Workshop on Spatial-Temporal Modeling

Geographic

Analysis

Geoprocessing

Web Services &

Solutions

Custom Web

Apps

PhillyHistory.org

Crime Mapping

Connect 211

Connect 211

Weighted Raster Overlay

x 5 x 2x 3x 1

+ ++

=

AfricaMap

AfricaMap

Background

1,500,000 humans

7,000 police

1,000 civilian employees

2,000,000 new incidents / year

Philadelphia Context

3 crime analysts

Weekly Compstat

Lots of maps

Automation of map creation

Web-based systems

What we did

- accelerate the cycle

- proactively notify

- automatically run

… But what if we could

1999 Internal research project at Phila PD

2000 Paper for NIJ Crime Mapping Conference

2004-2005 Prototype

2007-2010 National Science Foundation R & D Grant

Crime Early Warning System

ArcViewVB & MapObjects

MS SQL Server

Crime Incidents

Database

Shapefiles

and

GRIDs

Process

Documentation

.ini

file

Philadelphia Context

Philadelphia Context

Philadelphia Context

Philadelphia Context

Bad News

It was crap…

…sort of

We needed

1. Better Statistics

2. Notification

3. Super-simple

Round 2: Crime Spike Detector

Round 2: Crime Spike Detector

Burglaries, 6/20/2004, 1275ft. Radius, 30 Day Period Each Year

Round 2: What we learned

1. People used it but not the way we expected

2. More flexibility

3. More customized to user needs

� Automatic detection

� Subscribe to search patterns

� Automatically e-mail

� Configurable

� Fast

� Centralized

� Individualized

� Applicable to any space-time event

� No new software or hardware

Round 3: HunchLab Objectives

What is HunchLab?

HunchLab DatabaseCrime

Database

GIS Database

Alerts

Alerting SystemGeostatistical Engine

Crime Mapping Tools

Web Map Service

HunchLab Web Site

A crime spike is an “above average” amount, but:

How do we know when to send an alert?

� How much above average?

� What “average” are we using?

� When? Over what period of time?

� Where?

� What category of event?

1. Location

2. Time

3. Type of Event

Basic Data Requirements

1. Location

2. Time

3. Filter

Setting up a Hunch

Four options:

1) Address + Radius;

2) District/Sector/Beat

3) Custom Polygon

4) Mass Hunch

Hunch Parameters – Location

Time: two parts:

1) Current Data (recent past)

2) Historical Data

Hunch Parameters – Time

Hunch Parameters – Time

Event Filter:

� Category

� Time of Day

� Text (future)

Hunch Parameters – Event Filter

Hunch Parameters – Type of Hunch

� Statistical Hunch

� Event Trigger

Notification

Email Notification

� Too large area or too long time � nothing is a spike.

� Too small area or too short time � everything is a spike.

� Different crime classes require different calibration.

Defining Hunches can be tricky because…

� Doesn’t need lots of detailed data …

� … but has to be accurate data

� Humans still required

� No explanatory or interpretive power

� Not good for low volume events

The small print ….

High volume

databases:

humanly

impossible to track

all the data

When and where

has something

different

happened?

Geographic

Notification

Large space-time events databases

� Public Health

� Real Estate Sales

� Other data sets

Not just for criminals anymore….

HunchLab: Spatial-Temporal Data MiningRobert Cheetham cheetham@avencia.com

10 April 2009