Change-Point Detection of Climate Time Seriesby Nonparametric Method

Naoki Itoh ∗and Jurgen Kurths

Abstract—In one of the data mining techniques,change-point detection is of importance in evaluatingtime series measured in real world. For decades thistechnique has been developed as a nonlinear dynam-ics. We apply the method for detecting the changepoints, Singular Spectrum Transformation (SST), tothe climate time series. To know where the struc-tures of climate data sets change can reveal a climatebackground. In this paper we discuss the structuresof precipitation data in Kenya and Wrangel Island(Arctic land) by using the SST.

Keywords: Singular Spectrum Analysis, Principal

Component Analysis, climate time series analysis,

change-point detection

1 Introduction

Data mining aims to extract any information that is non-trivial but useful. With the dramatic development ofcomputer performance, we can manipulate large amountsof data and analyze the structured data dynamically.Change-point detection is defined as one of the tech-niques.

The work of change-point detection has been studied inmany scientific fields to give a better interpretation ofproperties extracted from a complicated system. Singu-lar Spectrum Transformation (SST)[5, 6, 7] that we intro-duce in this study is based on Principal Component Anal-ysis (PCA). The methodology itself has been describedas Singular Spectrum Analysis (SSA) by Golyandina etal. (2001)[2]. The basic idea of the change-point detec-tion using the SSA is explained by a paper of Moskviaand Zhigljavsky et al. (2003)[4]. SST is still being de-veloped and improved (e.g. Mohammad and Nishida[8]).An advantage of use of the SST is no requirements ofknowledge about ad-hoc tuning and modification for timeseries. Therefore, we suppose that it is suitable for ana-lyzing data in real world such that only a little a prioriknowledge about the data is provided.

In this paper Kenyan towns (Nakuru, Naivasha, andNarok) of the equatorial region and Wrangel Island inArctic land[3] significantly impacted by the global warm-ing such as drought and ice melting will be analyzed by

∗Center for Dynamics of Complex Systems University of Pots-dam, D-14476 Potsdam, Germany Email: naoki.itoh@gmail.com

this method.

Section 2 describes the algorithm of SSA. Section 3 de-fines change-point detection by using results from theSSA. The application to the climate time series is dis-cussed in section 4. Then finally the conclusion is de-scribed in section 5.

2 Singular Spectrum Analysis

SSA can be defined as a model-free technique, which aimsto decompose measured time series into some useful andinterpretable components such as global trends, harmonicterms, and noise. The tasks of SSA can basically beclassified as follows: 1.finding trends of different reso-lution; 2.smoothing; 3.extraction of seasonality compo-nents; 4.simultaneous extraction of cycles with small andlarge periods; 5.extraction of periodicities with varyingamplitudes; 6.simultaneous extraction of complex trendsand periodicities; 7.finding structure in short time series;and 8.change-point detection. From these items we willdiscuss a change-point detection by using the idea of Ideet al[5, 6, 7].

2.1 Algorithm

On the first stage, we make a trajectory matrix from asingle time series and then decompose it by using SingularValue Decomposition (SVD)

The trajectory matrix [X1, · · · , XK ] consists of the shortvectors, Xj = (yj , · · · , yj+L−1)T ∈ RL (j = 1, · · · ,K),from the single time series Y = (y1, · · · , yN ), which formsa Hankel matrix[9], where the parameters K and L canbe described by K = N−L+1. The L is called a windowlength restricted by 2 ≤ L ≤ N/2 as a single parameterfor the SSA:

X = [X1 : · · · : XK ] = (xij)L,Ki,j=1

=

y1 y2 · · · yK

y2 y3 · · · yK+1

......

. . ....

yL yL+1 · · · yN

. (1)

The trajectory matrix can be decomposed into submatri-

Proceedings of the World Congress on Engineering and Computer Science 2010 Vol I WCECS 2010, October 20-22, 2010, San Francisco, USA

ISBN: 978-988-17012-0-6 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

WCECS 2010

ces by the Singular Value Decomposition (SVD):

X = X1 + · · ·+ Xd, (2)

where

Xl = Ul

√λlV

Tl , (l = 1, · · · , d), (3)

which is defined as a rank-one orthogonal elementary ma-trix. A set of these three notations consisting of singu-lar value (

√λl), empirical orthogonal functions (Ul), and

principal components (Vl) is called the l-th eigentriple ofthe matrix X. The index of the elementary matrix shouldat most be taken until d which is the number of nonzerosingular values.

3 Change-Point Detection

Change-point detection problem is defined as a quanti-tative estimate of structural changes behind time series.The topic has been discussed by using several methods,such as a method based on Autoregressive model. Theparametric approach, however, does not always lead toa fine result from heterogeneous time series because it isnot easy to prepare any sufficient knowledge for such timeseries.

Singular Spectrum Transformation (SST) is a nonpara-metric change-point detection method, in which featureextraction can basically be achieved by using SVD de-fined in section 2. Since this method does not need toassume a certain stochastic model and to focus on somelocal solutions, it is applicable to heterogeneous time se-ries.

3.1 SST technique

Let us consider a time point t in time series. Then withreference to t we can define two kinds of time series ofthe past part, Y(p), and future part, Y(f), from originaltime series Y as follows:

Y(p) = (y1, · · · , yt−1), (4)Y(f) = (yt, · · · , yN ).

Then we make a trajectory matrix from each time series.

X(p)t = [Xt−K : · · · : Xt−1], (5)

X(f)t = [Xt+γ : · · · : Xt+γ+K−1],

where each vector consists of L elements of Y and theparameter γ which we can arbitrarily set is positive inte-ger. Note that, here, vectors in the empirical orthogonalfunctions (EOF), u

(p)i (i = 1, · · · , L) obtained from the

trajectory matrix of the past part, X(p), are described indescending order of the singular values. From X(f) de-fined here as test matrix, the EOF is described by u

(f)j

(j = 1, · · · , L). In particular, the vector of EOF belong-ing to the largest singular value, u

(f)1 , will be defined as

β(t). The change-point detection can be performed bya comparison between their vectors of the both parts,which is called change-point score:

z = 1−l∑

i=1

K(i, β)2, (6)

where K is an inner product of the vectors:

K(i, β) := βT u(p)i . (7)

3.2 Demonstration

We will express the result of SST by using simple exam-ples. The first example time series consists of three kindsof linear functions with the positive, zero, and negativeslopes. The second one is generated by three kinds of sinefunctions with noise[5]. Fig. 1 shows each original timeseries and its result by SST. We can see that the resultsof both data sets show high scores at about t = 150 andat t = 300. This means that change points of both casesare found at the same positions.

4 Application

In our study the monthly precipitation at the three mea-suring stations in Kenya (Nakuru, Naivasha, and Narok)and in Wrangel Island are analyzed. Fig. 2 shows theirpositional relation and the atmospheres of places. Al-though we have the data with different time length, asshown in Fig. 3 the region between January 1950 andDecember 1985 corresponds to 432 data points overlap-ping at all the stations.

In order to compare their data structures during theoverlapping time interval, we calculate the change-pointscores of them by SST. For the parameter L = 24 (i.e.for two years) the change-point scores can be depicted assharp curves (see Fig. 3).

Although it is not easy to discuss similarity and dissim-ilarity from these noisy original time series, the resultsfrom the application of SST provide those aspects. Bycomparing the four bottom panels in Fig. 3 we can es-pecially see that the result of the three Kenyan townsaround 1960 makes each high change-point score. Thismeans that the precipitation structure at this time is dif-ferent from the others. The result from Wrangel Islandcan be shown as a different kind of pattern than the re-sults from the Kenyan towns.

5 Conclusion

In this study, as a data mining of the climate, we wereable to demonstrate the capability of the SST method,which is defined here as a change-point detection based

Proceedings of the World Congress on Engineering and Computer Science 2010 Vol I WCECS 2010, October 20-22, 2010, San Francisco, USA

ISBN: 978-988-17012-0-6 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

WCECS 2010

100 200 300 4000

0.2

0.4

Time

100 200 300 4000

0.05

0.1

Time

CP

−S

core

100 200 300 400−2

0

2

Time

100 200 300 4000

0.05

0.1

Time

CP

−S

core

Figure 1: The results by SST; left panels: linear data set and right panels: harmonic data set

Figure 2: Measuring stations in Kenya (East Africa) and in Wrangel Island (Arctic land); left: their positionalrelation, middle: Lake-Nakuru in central Kenya, right: tundra on Wrangel Island.

1950 1960 1970 19800

200

400

Yr

Pre

cip

ita

tio

n

Precipitation Nakuru [1950−1985]

1950 1960 1970 19800

2

4x 10

−3

Yr

CP

Change−Point Detection

1950 1960 1970 19800

200

400

Yr

Pre

cip

ita

tio

n

Precipitation Naivasha [1950−1985]

1950 1960 1970 19800

0.005

0.01

Yr

CP

Change−Point Detection

1950 1960 1970 19800

500

Yr

Pre

cip

ita

tio

n

Precipitation Narok [1950−1985]

1950 1960 1970 19800

5x 10

−3

Yr

CP

Change−Point Detection

1950 1960 1970 19800

50

100

Yr

Pre

cip

ita

tio

n

Precipitation Arctic Land [1950−1985]

1950 1960 1970 19800

5x 10

−3

Yr

CP

Change−Point Detection

Figure 3: Top four panels: precipitation and bottom four panels: change-point score.

Proceedings of the World Congress on Engineering and Computer Science 2010 Vol I WCECS 2010, October 20-22, 2010, San Francisco, USA

ISBN: 978-988-17012-0-6 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

WCECS 2010

on singular spectrum analysis, to detect quantitativechanges in the structure of heterogeneous data sets.

As shown in section 4 the change-point scores have dis-played a characteristic expression according to location.From the measuring stations in Kenya the commonal-ity of the change-point could be shown. On the otherhand, Wrangel Island remote from the equatorial areahas shown quite a different pattern of the change-point.Thus, applying SST to more widespread areas, we mayhave more precise insights into the climate period back-ground.

Acknowledgment

The precipitation data from GHCN v2 database(http://www.ncdc.noaa.gov/) used in this study wereprovided by Norbert Marwan (PIK Potsdam). UdoSchwarz (Center for Dynamics of Complex Systems, Uni-versity of Potsdam) and Tsuyoshi Ide (IBM Tokyo Re-search Laboratory) gave some quite valuable commentsand suggestions. The authors would like to acknowledgethe help of them.

References

[1] H. Bjornsson, S.A. Venegas, “A Manual for EOF andSVD analyses of Climate Data,” Centre for Climateand Global Change Research, Report No. 97.1, 1997

[2] N. Golyandina, V. Nekrutkin and A. Zhigljavsky,Analysis of Time Series Structure, SSA and relatedtechniques, Chapman & Hall/CRC, 2001

[3] Kenji Matsuura and Cort J. Willmott, “Arctic Land-Surface Precipitation: 1930-2000 Gridded MonthlyTime Series,” Center for Climatic Research, Ver.1.10, March, 2004

[4] V. Moskvina and A. Zhigljavsky, “An algorithmbased on singular spectrum analysis for change-point detection,” Communications in Statistics–Simulation and Computation, pp. 319–352, 03

[5] Tsuyoshi Ide, Keisuke Inoue, “Knowledge Discov-ery Heterogeneous Dynamic Systems using Change-Point Correlations,” 2005 SIAM International Con-ference on Data Mining (SDM 05), 4/05

[6] Tsuyoshi Ide, Keisuke Inoue, “Knowledge Discov-ery from Time-series Data using Nonlinear Trans-formations,” Proceedings of the Fourth Data MiningWorkshop (The Japan Society for Software Scienceand Technology, Tokyo, 2004), No.29, pp. 1–8, 2004(Japanese)

[7] Tsuyoshi Ide, “Speeding up Change-Point Detec-tion using Matrix Compression,” 2006 Workshopon Information-Based Induction Science, 10–11/06(Japanese)

[8] Yasser Mohammad and Toyoaki Nishida, “RobustSingular Spectrum Transform,” the Twenty SecondInternational Conference on Industrial, Engineering& Other Applications of Applied Intelligent Systems(IEA/AIE 2009), Taiwan, 06/09

[9] J.L. Phillips, “The Triangular Decomposition ofHankel Matrices,” MATHEMATICS OF COMPU-TATION, V5, N115, 07/71

[10] M. Ghil, M.R. Allen, M.D. Dettinger, K. Ide, D.Kondrashov, M.E. Mann, A.W. Robertson, A. Saun-ders, Y. Tian, F. Varadi, and P. Yiou “ADVANCEDSPECTRAL METHODS FOR CLIMATIC TIMESERIES,” The American Geophysical Union Reviewof Geophysics, V40, pp. 1–41 9/01

Proceedings of the World Congress on Engineering and Computer Science 2010 Vol I WCECS 2010, October 20-22, 2010, San Francisco, USA

ISBN: 978-988-17012-0-6 ISSN: 2078-0958 (Print); ISSN: 2078-0966 (Online)

WCECS 2010