Try Test (S) on R 　

2014.9.3

Analyzing Time-Series Data

We introduce our R functionsYou should be able to:

• Testify stationary of time series Z(n)

• Identify fracture points of stationary time series Z(n)

• Compute determinacy and causality values of Z1(n) and Z2(n)

• Visualize the results using R plot functions

• Apply these tests to various time series data

2

Goals

• Using Statistics Programing language R• Prototyping Each Test

• Stationary Test: Test(S) • Abnormality Test: Test(ABN)• Determinacy Test: Test(D)• Causality Test: Test(ES)

• Merits• Easy to handle• Apply the tests to various data• Spread the theory throughout the world

History of Prototyping

• These tests are developed as – Basic Programs By Prof

• Test(S) , Test(ABN) 1988~• Test(ES), Test(D) 2004~

– Java Programs (Converted from the Basic Programs)• Test(S) , Test(ABN) 2009 ~ Test(ES), Test(D) 2011 ~

• Java programs have high performance.However, require programming skills

4

History: Prototyping Tests

• Developed by – Basic Programs By

• 1988 ~

– Java Programs By• 2009 ~

• R scripts + java class ( using Rjava ) By • 2012~

5

(continued)

Merits using R1. Usability: Easy to handle

2. Data processing Performance

3. Powerful Visualization

6

Demonstration of TestS-R

7

1 Easy to handle

• M2CD denotes a money supply data in Japan (1955q3 – 2001q4)• Provided function $testABN calls Java class testS

$testABN(ts, term)

> testABN(M2CD, term=98) Time Series Length = 186Term = 98 1 1 0 1 0 0 3 1 0 4 14 19 15 16 18 9 9 12 12 9 7 4 1 2 1 1 1 4 4 3 2 1 3 4 8 16 12 11 5 6 7 5 5 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 2 6 35 103 84 81 88 89 77 66 77 59 60 50 55 87 84 96 88 51 43

1 Easy to handle

• GDP denotes also Japan Nominal GDP data (1955q3 – 2001q4)• testD_result$result[4,] denotes the Determinacy value of V3(n)

$testD(ts1, ts2, term)

> testD_result<-testD(GDP, M2CD, 183)> testD_result$result[4,] [1] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000…………………………………………………(skip)………………………………[162] 0.0000000 0.0000000 0.0000000 0.7482668 0.8520689 0.8561230 0.8512788[169] 0.8785611 0.8996980 0.9045383 0.8903662 0.8810984 0.8808611 0.8864593[176] 0.9035437 0.8871590 0.8862733 0.8848502 0.8751460 0.8882548 0.8878465[183] 0.8835506 0.8778081

(continued)

Data Accessing

• testD_result is a data frame of matrix of R

testD_result$result[4,]

Row Access

Column Access

testD_result$result[4,]

Dataframe_id$filed_id

2 Data processing Performance

• Input M2CD data file

$scan, $log, $diff, $scale, $scale2

> M2CD_ld <- diff(log(M2CD))

> M2CD <-scan("M2CD.DAT")

Various data processing support: Log transformation, First difference and Standardization. M2CD_ld is used for test(ABN).

2 Data processing Performance

$plot, $ts

>M2CD <-scan("M2CD.DAT")>plot(ts(M2CD, start=1955.5,,frequency=4))

(continued)

2 Data processing Performance

>plot(ts(log(M2CD), start=1955.5,,frequency=4))

$plot, $ts, $log

(continued)

2 Data processing Performance

$plot, $ts, $diff

>plot(ts(diff(log(M2CD)), start=1955.5,,frequency=4))

(continued)

3 Visualization

> GDP <-scan("NGS-II.DAT") > M2CD <-scan("M2CD.DAT")>testD_result<-testD(GDP, M2CD, 183)>plotTestD(testD_result, 4, 1955.5) >abline(v=c(1981.25,1984,1996.5,1999),lty = 3)

$plotTestD(result, start)

Try Test(ABN), Test(D) with R

16

Non linear transformation

• Test(S) and Test(ABN) uses non–linear transformation up to rank 6, which constructs 19 one-dimensional time series ψi(Z(f)) from Z(n).

• $NLtransform returns 19 transformed time series as data frame.

$NLtransform(ts)

> NLtransform (diff(log(M2CD))…………………..Return value…………………………………..

Stationary Test: $testS

• $TestS is applied to a subsequence of log- and diff- transformed M2CD, whose data length is 99. ->M2CD[1:99]

$testS(ts, term)

> testS(diff(log(M2CD[1:99])), d=1)Dimension d= 1Time Series Length = 99Test(S) Passed [1] 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 0 0Time Series [2], [3], [5], [7], [18] is passed by TEST(S) is passed and considered stationary.

Stationary Test

• Test the stationarity of the target time series– Repeats Test(S) about subsequence TERM

– The length of TERM is set to 98: Economical Data

• $TestS returns an array of test(S) results about 19 transformed time series ψi(Z(f))

– one dimension: 19 results of 1d-test(S) for ψi(Z(f))

– Two dimension: results of 2d-test(S) for (ψi(Z(f)), ψj(Z(f)))

19

Test(S) : Preliminary

• Time Series Data Z(m ）– Sample mean vector μZ , – Sample covariance matrix function RZ

20

Test(S): Conditions

21

• Three Test Values (M) (V) (O)

• Conditions 　

(M) mean

(V) variance

(O) Co-variance

　　 Test(S): Conditions 　　

22

The Standard Rate of (M) i (V) i and (O)i under which Test(S) is accepted.

Total Passed ／ Trials (N － M ＋１ ) 　

•(M) i 　　　　　 80% •(V) i 70%•(O) i 80%

(continued)

Abnormality Test: $testABN

• $TestABN uses log- and diff- transformed M2CD• term is the length of subsequence tested by test(S) .

$testABN(ts, term)

>testABN_result <- testABN(diff(log(M2CD)), term=98) Time Series Length = 186Term = 98 1 1 0 1 0 0 3 1 0 4 14 19 15 16 18 9 9 12 12 9 7 4 1 2 1 1 1 4 4 3 2 1 3 4 8 16 12 11 5 6 7 5 5 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 2 6 35 103 84 81 88 89 77 66 77 59 60 50 55 87 84 96 88 51 43

Abnormality Test

• Test the “degree of breaks in a time series”

• Repeats Test(S) about subsequence TERM– The length of TERM is set to 98: Economical Data

• $TestABN returns a sequence of the Test(S) results– Detect the points N -> 0 (N: natural number)

24

Causality Test: $testCS

• testCS_result$shc[4,] expresses the Causality value of V3(n)

• term is the length of subsequence

$testCS(ts, term)

>testCS_result<- testCS(M2CD, 100)>testCS_result$shc [1] NA NA NA NA 0.8317427 0.8493772 0.8471115 [8] 0.8476854 0.8432251 0.8474183 0.8943973 0.8768367 0.8864003 0.8975523[15] 0.8943816 0.8876588 0.8841130 0.8554944 0.8761724 0.8838377 0.8857091[22] 0.8928515 0.8600705 0.8339162 0.8276518 0.8275824 0.8666339 0.8456843………………………………..(skipped)

Causality Test: $testCS

• Causality Test checks the linear causality relation from the time series X(n) to Y(n)

• X: e.g. GDP

• Y: e.g. M2CD

• $TestCS returns a data frame of 19 arrays of CR-values CR(SH)i(y)

26

Determinacy Test: $testD

• The test(D) checks linear causality from M2CD to GDP in Japan ( 1955q3 – 2001q4)

• testD_result$result[4,] denotes the Determinacy value of V3(n)

$testD(ts1, ts2, term)

> testD_result<-testD(GDP, M2CD, 183)> testD_result$result[4,] [1] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000…………………………………………………(skip)………………………………[162] 0.0000000 0.0000000 0.0000000 0.7482668 0.8520689 0.8561230 0.8512788[169] 0.8785611 0.8996980 0.9045383 0.8903662 0.8810984 0.8808611 0.8864593[176] 0.9035437 0.8871590 0.8862733 0.8848502 0.8751460 0.8882548 0.8878465[183] 0.8835506 0.8778081

Determinacy Test: $testD

• LN(6,2) determinacy property- Time series Z(n) hold the property if D(Z) >= CR(SH)

• $TestD returns java object TestD including 19 determinacy function values as to V(n)k . Fields of java class TestD

28

$result 19 determinacy function values

$index1,2 Index pair of (ψi(Z(f)), ψj(Z(f))

Visualization Example(1)

> full_result <-append (rep(NA,term), testABN_result) # fitting length to original data> ts.plot(ts(full_result , 1955.5,,frequency=4),gpars=list(xlab="year",ylab="",yaxt = "n"))> abline(v=c(1981.25),lty = 3)

$ts.plot, $abline

• $abline depicts vertical lines at time points specified by “v=“

Visualization Example(2)

> plotTestD(testD_result, 4, 1955.5)> abline(v=c(1981.25,1984,1996.5,1999),lty = 3)

$plotTestD(result, start)

Visualization Example(3)

>plotSample(testCS_result$shc, testD_result, 4 ,1955.5)>abline(v=c(1981.25,1984,1996.5,1999),lty = 3)

$plotSample(result, result, num, start)

Advanced Visualization: ggplot2

> abn<-read.table(file="plotABN.dat")>ggplot(abn , aes(x, y, fill=range)) + geom_area(alpha=0.3) 　　 + ggtitle("Stationary Graph")+xlab("year")+ylab("M2CD")

$ggplot2

Conclusion

• This is a provisional version– TestD and TestCS requires further refinement– TestS and TestABN are almost stable– We try to elaborate Visualization skills

• We hope these Tests will be commonly used.

33

References1. Yuji Nakano, Yasunori Okabe : A Time Series Analysis of Economical

Phenomena in Japan’s Lost Decade (1): Determinacy Property of the Velocity of Money and Equilibrium Solution ,Asia-Pacific Financial Markets November 2012, Volume 19, Issue 4, pp 371-389

34