Download - Test (S) on R
Try Test (S) on R
2014.9.3
Analyzing Time-Series Data
We introduce our R functionsYou should be able to:
• Testify stationary of time series Z(n)
• Identify fracture points of stationary time series Z(n)
• Compute determinacy and causality values of Z1(n) and Z2(n)
• Visualize the results using R plot functions
• Apply these tests to various time series data
2
Goals
• Using Statistics Programing language R• Prototyping Each Test
• Stationary Test: Test(S) • Abnormality Test: Test(ABN)• Determinacy Test: Test(D)• Causality Test: Test(ES)
• Merits• Easy to handle• Apply the tests to various data• Spread the theory throughout the world
History of Prototyping
• These tests are developed as – Basic Programs By Prof
• Test(S) , Test(ABN) 1988~• Test(ES), Test(D) 2004~
– Java Programs (Converted from the Basic Programs)• Test(S) , Test(ABN) 2009 ~ Test(ES), Test(D) 2011 ~
• Java programs have high performance.However, require programming skills
4
History: Prototyping Tests
• Developed by – Basic Programs By
• 1988 ~
– Java Programs By• 2009 ~
• R scripts + java class ( using Rjava ) By • 2012~
5
(continued)
Merits using R1. Usability: Easy to handle
2. Data processing Performance
3. Powerful Visualization
6
Demonstration of TestS-R
7
1 Easy to handle
• M2CD denotes a money supply data in Japan (1955q3 – 2001q4)• Provided function $testABN calls Java class testS
$testABN(ts, term)
> testABN(M2CD, term=98) Time Series Length = 186Term = 98 1 1 0 1 0 0 3 1 0 4 14 19 15 16 18 9 9 12 12 9 7 4 1 2 1 1 1 4 4 3 2 1 3 4 8 16 12 11 5 6 7 5 5 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 2 6 35 103 84 81 88 89 77 66 77 59 60 50 55 87 84 96 88 51 43
1 Easy to handle
• GDP denotes also Japan Nominal GDP data (1955q3 – 2001q4)• testD_result$result[4,] denotes the Determinacy value of V3(n)
$testD(ts1, ts2, term)
> testD_result<-testD(GDP, M2CD, 183)> testD_result$result[4,] [1] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000…………………………………………………(skip)………………………………[162] 0.0000000 0.0000000 0.0000000 0.7482668 0.8520689 0.8561230 0.8512788[169] 0.8785611 0.8996980 0.9045383 0.8903662 0.8810984 0.8808611 0.8864593[176] 0.9035437 0.8871590 0.8862733 0.8848502 0.8751460 0.8882548 0.8878465[183] 0.8835506 0.8778081
(continued)
Data Accessing
• testD_result is a data frame of matrix of R
testD_result$result[4,]
Row Access
Column Access
testD_result$result[4,]
Dataframe_id$filed_id
2 Data processing Performance
• Input M2CD data file
$scan, $log, $diff, $scale, $scale2
> M2CD_ld <- diff(log(M2CD))
> M2CD <-scan("M2CD.DAT")
Various data processing support: Log transformation, First difference and Standardization. M2CD_ld is used for test(ABN).
2 Data processing Performance
$plot, $ts
>M2CD <-scan("M2CD.DAT")>plot(ts(M2CD, start=1955.5,,frequency=4))
(continued)
2 Data processing Performance
>plot(ts(log(M2CD), start=1955.5,,frequency=4))
$plot, $ts, $log
(continued)
2 Data processing Performance
$plot, $ts, $diff
>plot(ts(diff(log(M2CD)), start=1955.5,,frequency=4))
(continued)
3 Visualization
> GDP <-scan("NGS-II.DAT") > M2CD <-scan("M2CD.DAT")>testD_result<-testD(GDP, M2CD, 183)>plotTestD(testD_result, 4, 1955.5) >abline(v=c(1981.25,1984,1996.5,1999),lty = 3)
$plotTestD(result, start)
Try Test(ABN), Test(D) with R
16
Non linear transformation
• Test(S) and Test(ABN) uses non–linear transformation up to rank 6, which constructs 19 one-dimensional time series ψi(Z(f)) from Z(n).
• $NLtransform returns 19 transformed time series as data frame.
$NLtransform(ts)
> NLtransform (diff(log(M2CD))…………………..Return value…………………………………..
Stationary Test: $testS
• $TestS is applied to a subsequence of log- and diff- transformed M2CD, whose data length is 99. ->M2CD[1:99]
$testS(ts, term)
> testS(diff(log(M2CD[1:99])), d=1)Dimension d= 1Time Series Length = 99Test(S) Passed [1] 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 0 0Time Series [2], [3], [5], [7], [18] is passed by TEST(S) is passed and considered stationary.
Stationary Test
• Test the stationarity of the target time series– Repeats Test(S) about subsequence TERM
– The length of TERM is set to 98: Economical Data
• $TestS returns an array of test(S) results about 19 transformed time series ψi(Z(f))
– one dimension: 19 results of 1d-test(S) for ψi(Z(f))
– Two dimension: results of 2d-test(S) for (ψi(Z(f)), ψj(Z(f)))
19
Test(S) : Preliminary
• Time Series Data Z(m )– Sample mean vector μZ , – Sample covariance matrix function RZ
20
Test(S): Conditions
21
• Three Test Values (M) (V) (O)
• Conditions
(M) mean
(V) variance
(O) Co-variance
Test(S): Conditions
22
The Standard Rate of (M) i (V) i and (O)i under which Test(S) is accepted.
Total Passed / Trials (N - M +1 )
•(M) i 80% •(V) i 70%•(O) i 80%
(continued)
Abnormality Test: $testABN
• $TestABN uses log- and diff- transformed M2CD• term is the length of subsequence tested by test(S) .
$testABN(ts, term)
>testABN_result <- testABN(diff(log(M2CD)), term=98) Time Series Length = 186Term = 98 1 1 0 1 0 0 3 1 0 4 14 19 15 16 18 9 9 12 12 9 7 4 1 2 1 1 1 4 4 3 2 1 3 4 8 16 12 11 5 6 7 5 5 3 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 2 6 35 103 84 81 88 89 77 66 77 59 60 50 55 87 84 96 88 51 43
Abnormality Test
• Test the “degree of breaks in a time series”
• Repeats Test(S) about subsequence TERM– The length of TERM is set to 98: Economical Data
• $TestABN returns a sequence of the Test(S) results– Detect the points N -> 0 (N: natural number)
24
Causality Test: $testCS
• testCS_result$shc[4,] expresses the Causality value of V3(n)
• term is the length of subsequence
$testCS(ts, term)
>testCS_result<- testCS(M2CD, 100)>testCS_result$shc [1] NA NA NA NA 0.8317427 0.8493772 0.8471115 [8] 0.8476854 0.8432251 0.8474183 0.8943973 0.8768367 0.8864003 0.8975523[15] 0.8943816 0.8876588 0.8841130 0.8554944 0.8761724 0.8838377 0.8857091[22] 0.8928515 0.8600705 0.8339162 0.8276518 0.8275824 0.8666339 0.8456843………………………………..(skipped)
Causality Test: $testCS
• Causality Test checks the linear causality relation from the time series X(n) to Y(n)
• X: e.g. GDP
• Y: e.g. M2CD
• $TestCS returns a data frame of 19 arrays of CR-values CR(SH)i(y)
26
Determinacy Test: $testD
• The test(D) checks linear causality from M2CD to GDP in Japan ( 1955q3 – 2001q4)
• testD_result$result[4,] denotes the Determinacy value of V3(n)
$testD(ts1, ts2, term)
> testD_result<-testD(GDP, M2CD, 183)> testD_result$result[4,] [1] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000…………………………………………………(skip)………………………………[162] 0.0000000 0.0000000 0.0000000 0.7482668 0.8520689 0.8561230 0.8512788[169] 0.8785611 0.8996980 0.9045383 0.8903662 0.8810984 0.8808611 0.8864593[176] 0.9035437 0.8871590 0.8862733 0.8848502 0.8751460 0.8882548 0.8878465[183] 0.8835506 0.8778081
Determinacy Test: $testD
• LN(6,2) determinacy property- Time series Z(n) hold the property if D(Z) >= CR(SH)
• $TestD returns java object TestD including 19 determinacy function values as to V(n)k . Fields of java class TestD
28
$result 19 determinacy function values
$index1,2 Index pair of (ψi(Z(f)), ψj(Z(f))
Visualization Example(1)
> full_result <-append (rep(NA,term), testABN_result) # fitting length to original data> ts.plot(ts(full_result , 1955.5,,frequency=4),gpars=list(xlab="year",ylab="",yaxt = "n"))> abline(v=c(1981.25),lty = 3)
$ts.plot, $abline
• $abline depicts vertical lines at time points specified by “v=“
Visualization Example(2)
> plotTestD(testD_result, 4, 1955.5)> abline(v=c(1981.25,1984,1996.5,1999),lty = 3)
$plotTestD(result, start)
Visualization Example(3)
>plotSample(testCS_result$shc, testD_result, 4 ,1955.5)>abline(v=c(1981.25,1984,1996.5,1999),lty = 3)
$plotSample(result, result, num, start)
Advanced Visualization: ggplot2
> abn<-read.table(file="plotABN.dat")>ggplot(abn , aes(x, y, fill=range)) + geom_area(alpha=0.3) + ggtitle("Stationary Graph")+xlab("year")+ylab("M2CD")
$ggplot2
Conclusion
• This is a provisional version– TestD and TestCS requires further refinement– TestS and TestABN are almost stable– We try to elaborate Visualization skills
• We hope these Tests will be commonly used.
33
References1. Yuji Nakano, Yasunori Okabe : A Time Series Analysis of Economical
Phenomena in Japan’s Lost Decade (1): Determinacy Property of the Velocity of Money and Equilibrium Solution ,Asia-Pacific Financial Markets November 2012, Volume 19, Issue 4, pp 371-389
34