functional data analysis t-61.6030 chapters 10,11,12 markus kuusisto
Post on 20-Dec-2015
225 views
TRANSCRIPT
![Page 1: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/1.jpg)
Functional Data Analysis T-61.6030
Chapters 10,11,12
Markus Kuusisto
![Page 2: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/2.jpg)
Topics
• 10 PCA of mixed data
• 11 Canonical Correlation Analysis
• 12 Functional linear models
![Page 3: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/3.jpg)
PCA of mixed data
• Both: functional part and vector part (xi ,yi)
• Canadian temperature: Registeration process finds suitable shift.
- Vector part is size of shift
- Functional part is shifted curve
![Page 4: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/4.jpg)
Canadian temperature
![Page 5: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/5.jpg)
Canadian temperature (shifted)
![Page 6: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/6.jpg)
Using PCA, vector part yi
• yi are nuisance parameters -> we ignore
• yi are marginal importance -> we ignore them when calculating PCA, but afterwards we investigate connections between PCA scores and yi
• yi are primary importance with functions xi -> we treat them as a hybrid data (xi ,yi)
![Page 7: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/7.jpg)
The PCA of hybrid data
• PCA weight function (,v)
• PCA score of particular observation:
i = xi (s) (s) ds + y’i v
• inner product: zi = (xi , yi )
z1 , z2 = x1 x2 + y’1 y2
• To find leading principal component maximize sample variance of the (,v) , zi when ||(,v)|| = 1
![Page 8: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/8.jpg)
Balance between functional and vector variation
• Measure units between functional and vector parts usually are not comparable
• z1 , z2 = x1 x2 + C2 y’1 y2
• Choice of C2
C2 = |T |, where T is interval of function xi
C2 = |T | / M, where M is length of y
C2 = Var(x) / Var(y)
![Page 9: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/9.jpg)
Incorporating smoothing
• Roughness of z = (x , y )
- D2z = (D2x, 0)
- || D2z ||2 = || D2x ||2
• Calculating like in chapter 9
![Page 10: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/10.jpg)
Canonical Correlation Analysis
• CCA is a way of measuring the linear relationship between two multidimensional variables
• Ordinary correlation analysis is dependent on the coordinate system in wich variables are described
• CCA finds the coordinate system where the correlation is maximized
![Page 11: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/11.jpg)
Definition of CCA• Consider the linear combination
x = xT wx y = yT wy
• Function to be maximized is
• The maximum of with respect to wx and wy is maximum canonical correlation
• The number of solutions are limited to the smallest dimensionality of x and y
yyyTyxxx
Tx
yxyTx
wCwwCw
wCw
yExE
xyE][][
][22
![Page 12: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/12.jpg)
Car marks example
![Page 13: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/13.jpg)
CCA of car marks
Correlation r1 = 0.9792 r2 = 0.8851
wx1 = wx2 =Price -0.4935 0.6887Value 0.8697 0.7251
wy1 = wy2 = Economy -0.5471 0.4693Service 0.2418 0.4496Design 0.0060 -0.0097Sport 0.5800 -0.0790Safety 0.2817 -0.0117Easy h. 0.4758 0.7558
![Page 14: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/14.jpg)
(xTwx2 , yTwy2 )
![Page 15: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/15.jpg)
Predicting by CCA
![Page 16: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/16.jpg)
Learning
• wx corresponds output (x)
• wy corresponds 52 previous datapoints (y)
• Learning
- Finding maximum canonical correlation and its weights wx , wy
- Linear line fitting
• Predicting output x is done by projecting y = yT wy
to fitted line.
![Page 17: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/17.jpg)
Predicting recursively next 50 data points
![Page 18: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/18.jpg)
Functional canonical correlation analysis
• Function to be maximized
subject to constraints
• It is possible allways to find perfect correlation• Maximization does not produce a meaningfult result
)(var)(var
)},{cov(),(
2
ii
ii
YX
YXccorsq
1)(var)var( ii YX
![Page 19: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/19.jpg)
Unsmoothed canonical variate weight function that attain perfect correlation.
• A standard condition for classical CCAn > p + q + 1,
- n is number of samples
- p is length of xi and q is lenght of yi
• In functional case p and q are infinite, no unique solution• Overfitting
![Page 20: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/20.jpg)
Smoothing
• Smoothing is essential
• Choice of can be done– subjectively
– by leave one out cross validation, maximazing squared correlation. (11.3.3)
• ccorsq calculated as above but with the observation (Xi ,Yi) omited
}||||)}{(var||||){(var
)},{cov(),(
2222
2
DYDX
YXccorsq
ii
ii
![Page 21: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/21.jpg)
Smoothed canonical weight functions
![Page 22: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/22.jpg)
Functional linear models
• Previous we have been exploring the variability of a functional variables
• Now we explore how much of variation is explained by other variables
• In calssical statistics we do that by linear regression and the general linear models.
• Now functional linear models
![Page 23: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/23.jpg)
Precipitation example
• Preciptitation (= total rainfall) of particular area
where i indexes the 35 weather stations
• Does the precipitation depend on temperature of that area
• Overfitting without smoothing
365
0
)(PrPr dttecectot ii
365
0
)()(Pr iii dsssTempectot
![Page 24: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/24.jpg)
A Functional response and a functional independent variable
• How does a precipitation profile depend on the associated temperature profile ?
• Concurrent: Precipitation now depends only on the temperature now
• Annual: Precipitation now depend on the temperature of the whole year
)()()()()(Pr tttTempttec iii
365
0
)(),()()()(Pr tdstssTempttec iii
![Page 25: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/25.jpg)
• Short-term feed-forward: For reasons of parsimony, precipitation now depends on the temperature over an interval back in time.
• Local influence: Precipitation now depends on the temperature over an interval back in time and the season (is it summer or winter ?)
t
t
iii tdstssTempttec
)(),()()()(Pr
t
tdstssTempttec iii )(),()()()(Pr
![Page 26: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/26.jpg)
Predicting derivatives
• Dynamic model: Model is designed to explain a derivative of some order
– homogenous first order linear differential equation
– -nonhomogenous• temperature in the equation is called forcing function
)()()(Pr)(Pr tttectecD iii
)()()()(Pr)(Pr 10 ttTempttectecD iiii
![Page 27: Functional Data Analysis T-61.6030 Chapters 10,11,12 Markus Kuusisto](https://reader035.vdocument.in/reader035/viewer/2022081513/56649d4b5503460f94a282e9/html5/thumbnails/27.jpg)
References• Book: Functional Data Analysis, J.O.Ramsay, B.W.Silverman
– http://www.functionaldata.org/• matlab toolbox for FDA
• http://www.imt.liu.se/~magnus/cca/– Classical Canonical Correlation Analysis– Method about solving blind source separation problem based on CCA– Matlab functions: cca.m and ccabss.m
• http://www.quantlet.com/mdstat/scripts/mva/htmlbook/mvahtmlnode95.html– Car marks example– You may get confused because results presented here differs from the site
above. Reason is that in that site the first and second canonical correlations are changed places.
• http://www.estsp2007.org/– Data of example ”Predicting by CCA”