secogis 20081 managing sensor data of urban traffic m. joliveau 1, f.de vuyst 1, g. jomier 2, c.m....

CADDYCADDY SeCoGIS 2008 1

Managing Sensor Data of Urban Traffic

M. Joliveau1, F.De Vuyst1, G. Jomier2,

C.M. Bauzer Medeiros3

ACI Masses de Données CADDY (2003-2007)

(1) MAS, Ecole Centrale de Paris

(2) LAMSADE, Université Paris-Dauphine

(3) IC, UNICAMP


Goals Urban road traffic

analysis congestions Query the past

behavior Foresee the future

behavior Show understandable

résults

(Google Maps)


Outline

Received Data Exploratory studies Deeper Analysis Work to do Concluding remarks

(Google Maps)


Data about the system to be studied

- Graph with hundreds of sensors

- Flow rate, occupancy rate, 3’

- States: fluid (0) / congestion (1)

- Annotations

From INRETS


Mass of Data

Sensor number (I)

Day number (J)

Number of measures in a day (K)

High rate of missing dataBad quality of dataSize order of the volume O(109) as I, J, K : O(103)


Exploratory study

Temporal view

Space-time view : dynamic vizualization of the sensor

state map

Flow rate Occupancy Rate

Hours 0->24h


Traffic States: fluid/congestion

It appears : 2 states are not enough to characterize the dynamic behavior of the system Urban Traffic

Spatio-temporal patterns


Space-Time Vizualization flow rate

x

Time

y


Analysis of temporal series

Extract of one week for a sensor among 400 Regularity of the human activity generating traffic


Schema of Data Base for Analysis

sensor-id

Sensors

day-id

Days

hour-id

Hours

annotation-id

Annotation

weather-id

Weather

sensor-idday-idhour-idannotation-idweather-id

Traffic

flow rateoccupancy ratetraffic state


Symbolic representation of sets of temporal series

Symbol = label associated to a class reduction of size and intelligibility Class identification of typical behavior, detection of atypical behaviors

Episod partitionSymbol AlphabetSymbolic Representation


Plan

Received Data Exploratory studies Deeper Analysis

STPCA Continuous Traffic

State Variable Concluding remarks


STPCA Spatio-Temporal Principal Component Analysis

Goal : data representation in a reduced number of spatial dimensions => sensors temporal dimensions => daily instants Result :Data projection simultaneously on the first spatial and temporal

eigenmodes

1st experiment : Flow rate (Monday to Friday) for a family of reliable sensors


Spatial Reduction

Xd (complete) matrix of daily realizations

element xi,t ,i sensor , t instant , d day

T number of instants by dayN number of daysI number of sensors

Y assembles horizontally N matrices Xd : Y = col (X1, X2,...... ,XN)


Sensors Number

Number of Measure Instants

Daily Data

Matrix Y for spatial reduction


Spatial Reduction

Y assembles horizontally N matrices Xd : Y = col (X1, X2,... ,XN)Each line is a temporal serie for 1 sensor

Singular value decomposition of Y Spatial correlation matrix: MS = YYT

Eigenvalues l1 >= l2 >= ... lKM

Eigenvectors (Fk) for k = 1…KM


Spatial Reduction

Spatial correlation matrix Ms = YYT

Eigenvalues: λ1 ≥ λ2 ≥... λKM

Eigenvectors: Ψk for k = 1…KM

P matrix of the K first eigenvectors Ψk

P = col (Ψ1, Ψ2, ... ΨK) for K<< KM


Spatial Reduction

Estimate X’d of each realization Xd :X’d = P PT Xd K reduced spatial orderReduced order matrix : Xr = PT Xcontains latent (hidden) variables of Xsize : K * T (T instants)

If T is large, the dimension of the reduced order representation is too large


Temporal Reduction

Z assembles vertically N day realizations Xd :

Z = row (X1, X2,... ,XN)

one colon corresponds to one instant t

one line corresponds to one sensor i for one day d

the data of one day d are grouped

I* N lines


Sensors Number

Number of Measure Instants

Daily Data

Matrix Z for temporal reduction


Temporal Reduction

Z assembles vertically N day realizations Xd :

Z = row (X1, X2,... ,XN)

Singular value decomposition of Z Temporal correlation matrix Mt = ZTZ

Eigenvalues μ1 ≥ μ2 ≥ ... μLM

Eigenvectors (Φl) for l = 1, 2…LM

Q matrix of the L first eigenvectors Φl

Q = col (Φ1, Φ2, ...ΦL) for L << LM


Temporal Reduction

Estimate X’ for each realization X:X’ = X Q QT

Reduced order matrix: Xr = XQcontains the latent variables of Xsize : I *L

If I (space : number of sensors) is large the dimension of the reduced order representation is too high


Results of temporal component analysis

temps temps

temps temps

tempstemps

Mode 1

Mode 3

Mode 5

Mode 2

Mode 4

Mode 6

temps temps

temps temps

tempstemps

Mode 1

Mode 3

Mode 5

Mode 2

Mode 4

Mode 6

The 6 first temporal modes(ACP-t)-order :colon- definethe matrix Q (Jr=6)


Reduction: projection on the first temporal mode

Flow rates, 1 work day, 6 sensors - Observed flow rate- Projection on the1rst temporal mode

tempstemps

tempstemps

temps temps

Cap

teur

1C

apte

ur 3

Cap

teur

5

Cap

teur

2C

apte

ur 4

Cap

teur

6

Time Time

Time Time

Time Time

Sens

or 1

Sens

or 2

Sens

or 3

Sens

or 4

Sens

or 5

Sens

or 6


Spatio-Temporal Reduction

Combines spatial and temporal analysis

new estimate of each realization X

X’ = PPTXQQT

Reduced order matrix:

Xr =PTXQ

contains the latent variables of X

size K*L


Cumulative Energy

Spatial correlation matrix

Eigenvalue Index

Eigenvalue Index

Cum

ulat

ive

Ene

rgy

Cum

ulat

ive

Ene

rgy

Temporal correlation matrix


Sensor 1 Sensor 2 Sensor 3 Sensor 4




Work days K=3, L=3


Mean Direct Error

Standard Deviation

Reduced-order Matrix

Size

Mean Direct Error

Standard Deviation


Size

Mean Direct Error

Standard Deviation


Size


g


Sensor 5

Sensor 9

Sensor 13

Sensor 6

Sensor 10

Sensor 14

Sensor 7

Sensor 11

Sensor 15

Sensor 8

Sensor 12

Sensor 16

Chrismas Day K=3, L=3


Error Distribution FunctionSensors

Num

ber

of S

enso

rs


Error Distribution Function Days

Nu

mb

er

of

da

ys


Plan

Received Data Exploratory studies Deeper Analysis

STPCA Continuous Traffic State

Variable Concluding remarks


Generation of 7 new traffic states using analysis in phase space

Saturé

Fluide

Grandecirculation

Occ

upan

cy R

ate

Flow Rate


Continuous traffic state variable

Occupancy rate

Throughput


Sensor 1

Time (hour)

Time (hour)

Time (hour)

Flo

w R

ate

(n

b v

eh

icle

s)O

ccu

pa

ncy

Ra

te (

%)

Circ

ula

tion

Sta

te (

%)


State Name Symb. State Num. Symbol E value at t Deriv. Sign in t

Calm

Negative

Very high level circ.

Saturation level 1

High level circul.

Saturation level 2

Saturation level 3

Positive

Back to Calm

New circulation states


Dynamic Visualization of the Traffic State

Fluid

Congestion

Animation :spatio-temporal patterns appear


Other results

Missing Data STPCA for state variables Spatio-temporal patterns

See Marc Joliveau ‘s PhD Thesis


Work to be done

Enrich the datawarehouse with summaries, GIS, results of STPCA…

Symbolic spatio-temporal analysis Adaptation to evolution Visualization, user interaction Refinement on types of days, episodes Datawarehouse : queries


Concluding Remarks Reduction : from data masses to intelligible and manipulable

elements

Generic Approach For spatio-temporal analysis of flow systems, described by data coming from a network of static

georeferenced sensors with diffuse sources and wells


Future Prospects

Data coming from embarked sensors

Go farther in spatio-temporal reduction

secogis 20081 managing sensor data of urban traffic m. joliveau 1, f.de vuyst 1, g. jomier 2, c.m....

Documents

unicamp secogis

spatial reductionestimate

sensors temporal dimensions

spatial reductionsecogis

spatial reductiony

temporal seriessymbol

temporal eigenmodes

data representation