first al-khawarezmi conference: qatar, december 6-8, 2010 ali hadi 0 0 the effects of centering and...

First Al-Khawarezmi Conference: Qatar, December 6-8, 2010 Ali Hadi

1

1

The Effects of Centering and Scaling the Rows of Multidimensional Data on

Their Graphical and Correlation Structures

Ali S. Hadi and Rida Moustafa

[email protected]@cornell.edu

www.aucegypt.edu/faculty/hadi

mailto:[email protected]

http://www.aucegypt.edu/faculty/hadi


2

2

Outline of the Talk

1. Introduction

2. Types of Centering and/or Scaling

3. Effects of Centering and/or Scaling

4. The Main Theoretical Results

5. Illustrative Examples

6. Conclusions


3

3

1. Introduction

Before performing certain statistical

analysis methods (e.g., principal

components and factor analyses), it may

be necessary to preprocess the data to

make them suitable for the analysis.


4

4

1. Introduction

Examples:

•Data Editing

• Imputation of missing values

• Transformation

• Identification of outliers

• Centering and/or Scaling (e.g., Rao,

2005)

• etc.


5

5

1. Introduction

Given an data matrix X, which

represents n multivariate observations on

p variables, the columns and/or the rows

of X may be centered and/or scaled before

applying a statistical method to the data

matrix X.

pn


6

6

2. Type of Centering and/or Scaling

1. Column (Variable) Centering:

The ij-th element of can be written as:

where is the mean of the i-th row.

Hence we have:

X11IX )( 1 Tnnn

c ncX

,. jij xx

pcT

n 0X1 jx.


7

7


2. Row (observations) Centering:


where is the mean of the i-th row.

Hence we have:

)( 1 Tppp

r p 11IXX rX

,.iij xx

.ixnp

r 01X


8

8


3. Column and Row Centering:


where is the mean of all elements of X.

)()( 11 Tppp

Tnnn

rc pn 11IX11IX

rcX

..,.. xxxx jiij

..x


9

9


4. Row Scaling (each row of X or :

This can be obtained by:

a. Scaling by the L1-norm

b. Scaling by the L2-norm

c. Scaling by the standard deviation (SD)

rX


10

10


4a.Scaling the rows of the matrix X or by

the L1-norm of its rows as follows:

where is diagonal matrix

with its i-th diagonal element equals to the

reciprocal of the L1-norm of the i-th row of X

or .

rrL

rLLL XSXXSX

1111 or

1LS

rX

rX

nn


11

11


4b.Scaling the rows of the matrix X or by

the L2-norm of its rows as follows:



reciprocal of the L2-norm of the i-th row of X

or .

rrL

rLLL XSXXSX

2222 or

2LS

rX

rX

nn


12

12


4c.Scaling the rows of the matrix X or by

the standard deviation (SD) of its rows:



reciprocal of the standard deviation of the i-th

row of X.

rSD

rSDSDSD XSXXSX or

SDS

rX

nn


13

13


5. Standardizing the variables (each column

of X:

where is diagonal matrix with

its i-th diagonal element equals to the

reciprocal of the standard deviation of the j-th

column of X.

pccs SXX

pS pp


14

14


6. Centering and standardizing both rows

and columns of X:

This is obtained by an iterative

standardization process of rows and

columns until the rows and columns are

approximately standardized.


15

15


The above row and column transformations

have been used by several authors in

practical applications: For example:

Holter et al. (2000) , Wen et al. (2007),

Pielou (1984), Jackson (1991), Pyle (1999),

van der Werf, Jellema, and Hankemeier

(2005), van den Berg et al. (2006).


16

16


We argue that of the 11 methods of

centering/scaling the data matrix mentioned

above only two (the which deal with the

columns of the data) are meaningful. The

other 9, which deal with the rows of the data,

are not generally recommended.

Why? Next slide.


17

17


There are several reasons why centering

and/or scaling the rows is not a good to do:

1. Centering and scaling the observations is

not always possible. For example, when

an observation consists of the same

numerical value on all dimensions,

centering will replace the observation by

zeros and scaling would not be possible.


18

18


2. Even when centering rows is possible, it

creates a perfect collinearity among the

variables even if the original variables are

orthogonal. This is because

for any data matrix X.

npr 01X


19

19


3. In addition, it alters the correlation

structure among the variables. For

example, two positively correlated

variables will turn into two perfectly

negatively correlated variables after row-

centering. This is because the two

variables in the row-centered data are

. hence , 2121rr

nrr XX0XX


20

20


Two positively correlated variables turn into

two negatively correlated variables


21

21


4. Centering and/or scaling observations are

not statistically meaningful because We

cannot attach a unit of measurement to

the mean or the standard deviation of the

observations because we may be adding

variables that have very different units of

measurements.


22

22


5. After row scaling, the observations on the

same variable would have different units

of measurements. Thus, the observations

on the same variable will have different

origin and/or different scale.


23

23


55

2020

3535

1020

545

575rXX

2/12/1

2/12/1

2/12/1rsX

Example:


24

24


6. Finally, perhaps the most damaging effect

of centering and/or scaling the

observations is that they distort the

graphical structure of the observations in

the multidimensional space and

substantially alters the correlation

structure among the variables.


25

25

Outline of the Talk

1. Introduction





6. Conclusions


26

26


Theorem 1. Centering the rows of X:

For simplicity of notation, let Y be the matrix

obtained by centering the rows of X or .

Then the columns of Y are linearly

dependent even if X and are of full-

column rank.

cX

cX


27

27


Theorem 2. Scaling the rows of X by the L1-

norm:


obtained by scaling the rows of X or by

the L1-norm. Then the rows of Y lie on the

surface of a parallelogram with sides.p2

rX


28

28


Theorem 3. Scaling the rows of X by the L2-

norm:


obtained by scaling the rows of X or by

the L2-norm. Then the rows of Y lie on the

surface of a sphere in p-dimensional space.

rX


29

29


Theorem 4. Scaling the rows of X using the

standard deviation:

Let be the matrix obtained by

centering and standardizing the rows of .

Then the rows of lie on the surface of a

dimensional ellipsoid, centered at the

origin.

rX

rSD

rSD

XSX

rSD

X

1p


30

30


Example 1. Two Dimensional Data


31

31




32

32




33

33


Example 2. DNA Microarrays Data

The DNA microarrays dataset consists of

genome-wide expression measurements,

where for each of n = 2467 genes, p=79

measurements have been taken resulting in

a data matrix X of 246 rows and 79

Columns (The National Academy of

Sciences Website www.pnas.org).


34

34


Example 2. DNA Microarrays Data

The dataset has been analyzed by many

authors. For example,

Schena et al. (1996), Shalon et al. (1996),

Cho et al. (1998), Eisen et al. (1998),

Spellman et al. (1998). The authors scale the

rows of X by the L2-norm.


35

35


Example 2. DNA Microarrays Data p = 3


36

36




37

37




38

38




39

39

6. Conclusions

1. Centering and/or scaling the rows of X

distorts the graphical structure of the

observations in the multi-dimensional

space and substantially alters the

correlation structure among the

variables.


40

40

6. Conclusions

2. Accordingly, analysts who use such row

centering and/or scaling should first

demonstrate that the process results in

a new, more appropriate structure for

their questions.


41

41

Outline of the Talk

1. Introduction





6. Conclusions


42

42

Thank You!!

first al-khawarezmi conference: qatar, december 6-8, 2010 ali hadi 0 0 the effects of centering and...

Documents

ali hadi

alkhawarezmi conference

row centering

ith row of x

row observations centering

rows of x

data matrix x

diagonal matrix