change detection in monitoring time series - changes...

1Challenge the future

Change detection in monitoring time series

Thom BogaardDelft University of Technology

Formose - Changes Workshop


Change detection in monitoring series


Content

What is a time series? What are we monitoring? What is a change? How can we detect the time series changes?

Example for streams/rivers Example for landslides


time series is a sequence of data points, measured typically at successive points in time spaced at uniform time intervals

Time series analysis comprises methods for analyzing time series data in order to extract meaningful statistics and other characteristics of the data

What is a time series?


Observation interval = 5 daysObservation frequency = 6 times / month

Plot of hydrograph

0 72 144 216 288 360 432 504 576 648 720 792

Time interval (5 days)

62

63

64

65

66

67

68

Gro

und

wat

er le

vel (

m)

Characteristics of time series


Mean: central tendency

h n1 = h i

n

1=i

Characteristics of time series

Variance: variation around mean

)h - h( 1-n

1 = s2

i

n

1=i

2

0 12 24 36 48 60


38

39

40

41

42

43

44

45

Gro

undw

ater

leve

l (m

)


Time scale effect on time series


Time and spatial scale effect on discharge time series


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50

Distance (km)

r(d) Daily Rainfall Monthly rainfall

Scale effect on rainfall time seriesExample of 9 rain gauges Luxembourg


Decompose a time series

Time series

Trend

Periodicity

Catastrophic event

Noise (random)


How to test periodicity?Serial correlation

Random

Autoregressive, Markov process

Periodicity


How to test periodicity?Serial correlation and confidence limits

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42

Time lag (15 days)

Cor

relo

gram

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42

Time lag (15 days)

Cor

relo

gram

Lower limit Correlation coefficient Upper limit


Correlogram of seasonal time series

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

0 72 144 216 288 360 432 504 576 648 720 792

Time (5 days)

Seas

onal

var

iatio

ns (m

)

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.2

0 36 72 108 144 180

Time lag (5 dyas)

Cor

relo

gram

Lower limit Correlogram Upper limit


Floods: discharge, water level height, bed topography, …

What are we monitoring in natural hazards?

z

v


Landslide: displacement, groundwater level, precipitation



What is the difference between a cause and a trigger?


How to detect a change?

Methods

Visual inspection Double mass (residual mass) Statistics Physical modelling


… data quality and extremesHow to detect a change?


How to detect a change?… data quality and extremes


Patterns of movement in reactivated landslidesMassey ey al, 2013 Engineering Geology

Visual inspectionHow to detect a change?



Double mass plot

Double mass analysis

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

0 20000 40000 60000 80000

Cumulative Average Discharge (n-1 stations)C

umul

ativ

e di

scha

rge

( 1 s

tatio

n)

Plot cumulative observation time series against another (averaged) cumulative time series

Double Mass curve

0

500

1000

1500

2000

2500

0 2000 4000 6000 8000Station 1

Stat

ion

2



Double mass plot


Characteristics of Time SeriesStationarity

• Stationary: probability distribution doesn’t change with time

• First-order stationary: mean is a constant• Second-order stationary: mean is a constant and

covariance is only a function of time lag, not actual time• Non-stationary in the mean: presence of a trend or

periodicity



A stationary time series

0 12 24 36 48 60 72 84 96 108 120


-300

-280

-260

-240

-220

-200

Gro

undw

ater

leve

l (cm

)

Characteristics of Time Series


Non-stationary time series with a trend

0 12 24 36 48 60 72 84 96 108 120


2

3

4

5

6

7

8

Gro

undw

ater

leve

l (m

)

Groundwater level Linear trend



Non-stationary time series with periodic changes

0 12 24 36 48 60 72 84 96 108 120


-3

-2

-1

0

1

2

3

Gro

undw

ater

leve

l (m

)

Groundwater level Seasonal trend



0 12 24 36 48 60 72 84 96 108 120


36

3738394041424344

45

Gro

undw

ater

leve

l (m

)

Groundwater level Step trend

µ1 µ2

• Step trend

tn1>t121t + ) - ( + = h

Detection of a trend


• Hypothesis testH0 µ1 = µ2Ha µ1 µ2

t statistic

• t test resultGiven α(5%), find tα/2(n-2) from Student tableIf t > tα/2(n-2) accept Ha, step trend is significantIf t tα/2(n-2) accept H0, step trend is not significant

2)-t(n n/s2|x-x|=t

p

21

Detection of a step trend


t0.025(118) = 1.96

0 12 24 36 48 60 72 84 96 108 120


36373839404142434445

Gro

undw

ater

leve

l (m

)

Groundwater level Step trend

9.89= 1201.1/*2

|40.0-42.0|=t

0.42h 601 = h i

60

1=i1 0.40h

601 = h i

120

61=i2

1.1] )0.40 - h( + )0.42 - h( [ 2-120

1 = s 2i

120

16=i

2i

60

1=i

2p

Detection of a step trend


0 12 24 36 48 60 72 84 96 108 120


22

24

26

28

30

32

34

Gro

undw

ater

leve

l (m

)


Detection of a linear trend

t10t + t + = htb + b = h 10t


• Hypothesis testH0 β1 = 0Ha β1 0

• t statistic

• t test resultGiven α(5%), find tα/2(n-2) from Student tableIf t > tα/2(n-2) accept Ha, linear trend is significantIf t tα/2(n-2) accept H0, linear trend is not significant

2)-t(n 1)-1)(n+n(n/s12

|b|=tl

1



Example of detecting a linear trend

0 12 24 36 48 60 72 84 96 108 120


22

24

26

28

30

32

34

Gro

undw

ater

leve

l (m

)


05.0)t - (t

)t - )(th - h( = b

2n

1=t

t

n

1=t1

0.32tb - h = b 10



t0.025(118) = 1.96

0 12 24 36 48 60 72 84 96 108 120


22

24

26

28

30

32

34G

roun

dwat

er le

vel (

m)


18.97= 119*121*1201.0/*12

|-0.05|=t

0.1)tb - b - h( 2-n

1 = s2

10t

n

1=t

2l


Example of detecting a linear trend


Characteristics of a harmonic function

0 2 4 6 8 10 12 14 16 18 20 22 24

T ime (month)

-3

-2

-1

0

1

2

3

Har

mo

nic

func

tion

Hramonic seriesA=2 amplitude

A0=0 base level

0= 0 initial phase T=1/f =12 period

)tfin(2s A + A = h 00t

Detection of a periodic trend


0 12 24 36 48 60 72 84 96 108 120Time interval (15 days)

44

46

48

50

52

54

56

58

Gro

undw

ater

leve

l (m

) Measurements Harmonic se ries

Detection of a periodic trend

)12

t2(sin0.53+)12

t2(cos0.86+)24

t2(sin1.68+)24

t2(cos2.23+49.97=ht

Fit of harmonic series with 2 harmonics


Shifts in the Mean

1. Student’s t-test2. Bayesian analysis 3. Mann–Whitney U-test4. Wilcoxon rank sum5. Pettitt test 6. Mann-Kendall test7. Lepage test8. Standard normal homogeneity test

9. Regression-based approach10. CUSUM test11. Oerlemans method12. Signal-to-noise ratio13. Intervention analysis14. Markov chain Monte Carlo15. Lanzante method


Shifts in the Variance1. Downton-Katz test

Shifts in the Spectrum1. Nikiforov method

Shifts in the System1. Principal component analysis2. Average standard deviates3. Fisher information4. Vector autoregressive method


Example: Min Tu – Assessment of the effects of climate variability and land use change on the hydrology of the Meuse river basin (2006)


Change detection in monitoring time series


Summary

What is a time series? What are we monitoring? What is a change?

• Step, linear, periodicity, etc

How can we detect the time series changes? • Visual, double mass, statistical, ….

change detection in monitoring time series - changes...

Documents