'&

$%

LAGO:AComputationallyEfficientMethod

forStatistica

lDetection

Mu

Zhu

Univ

ersity

ofW

ate

rlo

o

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-1

-

'&

$%

Acknowledgment

•Co-authors:

–WanhuaSu;

–HughA.Chipman.

•Resea

rchsupport:

–NSERC;

–MIT

ACS;

–CFI;

–Acadia

Centre

forMathem

atica

lModellin

gandComputatio

n.

•Others:

Willia

mJ.Welch

,R.Way

neOldford,Jerry

F.Law

less,

Mary

Thompson,S.Young.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-2

-

'&

$%

Agenda

1.Thesta

tisticaldetectio

nproblem

.

2.Avera

geprecisio

n.

3.Drugdiscov

eryandhighthroughputscreen

ing.

4.LAGO.

5.Radialbasis

functio

n(R

BF)netw

orks.

6.Support

vecto

rmachines

(SVMs).

7.Resu

lts.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-3

-

'&

$%

TheDetectionProblemH

its: h(t)

Detected

t

Relevant:

π

Collection

100%

Fig

ure

1:Illu

stratio

nofatypica

ldetectio

nopera

tion.A

smallfra

ction

πoftheentire

collectio

nCis

ofinterest

(relevant).

Analgorith

mdetects

afra

ctiontfro

mC,outofwhich

h(t)

isreleva

nt.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-4

-

'&

$%

TheTypicalParadigm

New

Data

Results

Ranking

Training D

ata

Model Supervised L

earning

Fig

ure

2:Illu

stratio

nofthetypica

lmodellin

gandpred

ictionprocess.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-5

-

'&

$%

TheHitCurve

0.00.2

0.40.6

0.81.0

0.00 0.01 0.02 0.03 0.04 0.05

t

h(t)

hA (t)

hB (t)

hR (t)

hP (t)

Fig

ure

3:Illu

stratio

nofsomehitcu

rves.

Note

thathA(t)

andhB(t)

cross

each

other;

hP(t)

isanidealcu

rveproduced

byaperfect

algorith

m;hR(t)

corresp

ondsto

thecase

ofrandom

detectio

n.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-6

-

'&

$%

TheAveragePrecisio

n

Let

h(t)

bethehit

curve;

let

r(t)=

h(t)

πand

p(t)

=h(t)

t.

Then

,

Avera

gePrecisio

n=

∫

p(t)d

r(t).(1)

Inpractice,

h(t)

takes

values

only

atafinite

number

ofpointsti=i/n,

i=

1,2,...,n

.Hen

ce,theinteg

ral(1)is

replaced

with

afinite

sum

∫

p(t)d

r(t)=

n∑i=1

p(ti )∆

r(ti )

(2)

where

∆r(t

i )=r(t

i )−r(t

i−1 ).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-7

-

'&

$%

ASimpleExample

Algorith

mA

Algorith

mB

Item(i)

Hit

p(ti )

∆r(t

i )Hit

p(ti )

∆r(t

i )

11

1/1

1/3

11/1

1/3

21

2/2

1/3

01/2

0

30

2/3

00

1/3

0

41

3/4

1/3

12/4

1/3

50

3/5

01

3/5

1/3

AP(A

)=

5∑i=1

p(ti )∆

r(ti )

=

(

11+

22+

34

)

×13≈

0.92.

AP(B

)=

5∑i=1

p(ti )∆

r(ti )

=

(

11+

24+

35

)

×13=

0.70.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-8

-

'&

$%

HighThroughputScreening(HTS)

HT

S

Com

pounds

chemistry

library

Y

p 1

X XC

omputational

Chem

ical

Fig

ure

4:Illu

stratio

nofthehighthroughputscreen

ingprocess.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-9

-

'&

$%

DrugDiscoveryData

Orig

inaldata

from

Natio

nalCancer

Institu

te(N

CI)

with

pred

ictors

calcu

lated

byGlaxoSmith

Klin

e,Inc.

1.n=

29,812chem

icalcompounds,

ofwhich

only

608are

activ

e

against

theHIV

viru

s.

2.d=

6chem

ometric

descrip

tors

ofthemolecu

larstru

cture,

know

nas

BCUT

numbers.

3.Usin

gstra

tified

samplin

g,randomly

split

ofthedata

toproduce

a

trainingset

andatest

set(ea

chwith

n=

14,906and304activ

e

compounds).

4.Tuningparameters

selectedusin

g5-fo

ldcro

ss-valid

atio

nonthe

trainingset,

andcompare

perfo

rmance

onthetest

set.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-10

-

'&

$%

RankingFunctions

1.Given

avecto

rofpred

ictors

x,theposterio

rprobability

g(x)≡P(y

=1|x)=

π1 p1 (x

)

π1 p1 (x

)+π0 p0 (x

)(3)

isarguably

agoodrankingfunctio

n,i.e.,

itemswith

ahigh

probability

ofbein

greleva

ntshould

beranked

first.

2.Asfarasrankingis

concern

ed,allmonotonic

transfo

rmatio

nsofg

are

clearly

equiva

lent,so

itsuffices

tofocu

sontheratio

functio

n

f(x)=

p1 (x

)

p0 (x

)(4)

since

thefunctio

ngis

oftheform

g(x)=

af(x)

af(x)+

1

forsomeconsta

ntanotdep

endingon

x,which

isamonotonic

transfo

rmatio

noff.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-11

-

'&

$%

TwoAssumptions

A1.Forallpractica

lpurposes

theden

sityfunctio

np1 (x

)canbeassu

med

tohavebounded

localsupport,

possib

lyover

anumber

of

disco

nnected

regions,Sγ⊂

Rd,γ=

1,2,...,Γ

,in

which

case

the

support

ofp1canbewritten

as

S=

Γ⋃

γ=1

Sγ⊂

Rd.

A2.Forevery

observa

tion

xi∈C1 ,

there

are

atlea

stacerta

innumber

of

observa

tions,

saym,fro

mC0in

itsim

med

iate

localneig

hborhood;

moreov

er,theden

sityfunctio

np0 (x

)in

thatneig

hborhoodcanbe

assu

med

toberela

tively

flatin

compariso

nwith

p1 (x

).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-12

-

'&

$%

Inordertobuildapredictive

modelfor

statisticaldetection

problems,

itsufficesto

☞estim

atep

1 (x)alon

eand

☞adjustp

1 (x)locally

dependingon

p0 (x)nearb

y.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-13

-

'&

$%

Assumex∈

Risascalar.

⇓

Generalize

tox∈

Rdfor

d>1.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-14

-

'&

$%

Step1:Estim

atingp

1

1.Use

anadaptiv

ebandwidth

kern

elestim

ator:

p1 (x

)=

1n1

∑

yi=1

K(x;x

i ,ri ).

(5)

2.Foreach

xi∈C1 ,

choose

riadaptiv

elyto

betheavera

gedista

nce

betw

eenxiandits

K-nearest

neig

hbors

from

C0 ,

i.e.,

ri=

1K

∑

wj∈N(x

i,K) |x

i−wj |.

(6)

Thenotatio

nN(x

i ,K)is

used

torefer

totheset

thatcontainsthe

K-nearest

class-0

neig

hbors

ofxi .

Thenumber

Kis

atuning

parameter

tobeselected

empirica

lly,e.g

.,with

cross-va

lidatio

n.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-15

-

'&

$%

OriginalInspiration

Fig

ure

5:Theancien

tChinese

gameofGoisagamein

which

each

play

er

triesto

claim

asmanyterrito

riesaspossib

leontheboard.Im

agetaken

from

http

://go.arad.ro

/Intro

ducere.h

tml.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-16

-

'&

$%

Step2:LocalAdjustmentofp

1

1.View

thekern

elden

sityestim

ate

(5)asamixture

andadjust

each

mixture

component(cen

teredatxi )

acco

rdingly.

2.Estim

ate

p0locally

aroundevery

xi∈C1 ,

sayp0 (x

;xi ),

anddivideit

intoK(x;x

i ,ri ).

Assu

mptio

nA2im

plies

thatwecansim

ply

estimate

p0 (x

;xi )

locally

asaconsta

nt,

sayci .

Hen

ce,weobtain

f(x)=

1n1

∑

yi=1

K(x;x

i ,ri )

ci

(7)

asanestim

ate

oftherankingfunctio

nf(x).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-17

-

'&

$%

AnIdealizedSituation

Instea

dofsay

ingp0 (x

;xi )≈ci ,weshallexplicitly

assu

methat,

forevery

xi∈C1 ,

there

exist

i.i.d.observa

tionsw1 ,w

2 ,...,wm

from

C0thatcanbe

taken

tobeunifo

rmly

distrib

uted

ontheinterva

l[x

i−

1/2ci ,x

i+

1/2ci ].

Theorem

1Let

x0be

afixed

observa

tionfro

mcla

ss1.Suppose

w1 ,w

2 ,...,wmare

i.i.d.observa

tionsfro

mcla

ss0thatare

uniform

ly

distribu

tedaroundx0 ,sayontheinterva

l[x0−

1/2c0 ,x

0+

1/2c0 ].

Ifr0

istheavera

gedista

nce

between

x0andits

Knearest

neigh

bors

from

class

0(K

<m),then

wehave

E(r0 )

=K

+1

4(m

+1)c0.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-18

-

'&

$%

ImplicationsoftheTheorem

•Canassu

methere

are

atlea

stm

observa

tionsfro

mC0distrib

uted

approxim

ately

unifo

rmly

aroundevery

xi∈C1 .

•ForK

<m,canconclu

deriisapproxim

ately

proportio

nalto

1/ci .

•Since

riis

alrea

dycomputed

,there

isnoneed

toestim

ate

ci ;we

simply

use

f(x)=

1n1

∑

yi=1

riK(x;x

i ,ri ).

(8)

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-19

-

'&

$%

AShortSummary

1.Estim

atio

nofp1 :

p1 (x

)=

1n1

∑

yi=1

K(x;x

i ,ri ).

2.Adjustm

entofp1acco

rdingto

p0nearby:

f(x)=

1n1

∑

yi=1

K(x;x

i ,ri )

ci

=⇒

f(x)=

1n1

∑

yi=1

riK(x;x

i ,ri ).

⇓

LAGO

=“Locally

Adjusted

GO-kern

elden

sityestim

ator.”

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-20

-

'&

$%

Extensionto

Rd

1.Forevery

trainingobserva

tionin

class

1,xi∈C1 ,

compute

aspecifi

c

bandwidth

vecto

rri=

(ri1 ,r

i2 ,...,rid )

T,where

rij

istheavera

ge

dista

nce

betw

eenxiandits

K-nearest

class-0

neig

hbors

inthejth

dim

ensio

n.

2.Forevery

new

observa

tion

x=

(x1 ,x

2 ,...,xd )

Twhere

apred

ictionis

required

,sco

reandrank

xacco

rdingto:

f(x)=

1n1

∑

yi=1

{

d∏

j=1

rij K

(xj ;x

ij ,rij )

}

,(9)

which

uses

theNaiveBayes

prin

ciple

(Hastie

etal.2001,Sectio

n

6.6.3).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-21

-

'&

$%

SomeKernelFunctions

Gau

ssianT

riang

ular

Un

iform

f(u)∝

exp

(

−u22

)

f(u)=

1−|u|

|u|≤

1

f(u)=

1

|u|≤

1

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-22

-

'&

$%

RadialBasisFunctionNetworks

Aradialbasis

functio

n(R

BF)netw

ork

hastheform

:

f(x)=

n∑i=1

βi K

(x;µ

i ,ri ),

(10)

where

K(x;µ,r)

isakern

elfunctio

ncen

teredatlocatio

nµ

with

radius

(orbandwidth)vecto

rr=

(r1 ,r

2 ,...,rd )

T.Clea

rly,in

order

toconstru

ct

anRBFnetw

ork

wemust

specify

thecen

tersµiandtheradii

rifor

i=

1,2,...,n

.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-23

-

'&

$%

GeneralParameterization

p1

p1

p0

p0 ’

f1

f1

f1 ’

f1 ’

β−effect: height

α−

effect: radius

Fig

ure

6:Illu

stratio

n.Left:

Den

sityfunctio

nsp0andp1 .

Right:

The

ratio

functio

nf(x).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-24

-

'&

$%

Parameterizin

gtheα-andβ-Effects

•Takeakern

elfunctio

nbelo

ngingto

alocatio

n-sca

lefamily

:

1ri K

(

x−xi

ri

)

.

Canexplicitly

parameterize

theα-andβ-eff

ectsasfollow

s:

rβ′

i

1αri K

(

x−xi

αri

)

∝rβ′−1

iK

(

x−xi

αri

)

≡rβiK

(

x−xi

αri

)

.

•In

constru

ctingtheLAGO

model,

wehavein

effect

argued

that

β=

0(orβ′=

1).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-25

-

'&

$%

TheLAGOModel

•Forrela

tively

largeK,canusually

obtain

amodel

with

very

simila

r

perfo

rmance

bysettin

gα>

1andusin

gamuch

smaller

K.

•Hen

cebykeep

ingα,canrestrict

ourselv

esto

amuch

narrow

errange

when

selectingK

bycro

ss-valid

atio

n.

•Thefinalform

oftheLAGO

model

is:

f(x)=

1n1

∑

yi=1

{

d∏

j=1

rij K

(xj ;x

ij ,αrij )

}

,(11)

with

twotuningparameters,

Kandα.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-26

-

'&

$%

SeparatingHyperplanes

•Given

xi∈

Rd,

ahyperp

lanein

Rdis

characterized

by

f(x)=βTx+β0=

0.

•Given

yi∈{−1,+

1}(tw

ocla

sses),ahyperp

laneis

asep

aratin

g

hyperp

laneifthere

exists

c>

0such

that

yi (β

Txi+β0 )≥c∀i.

•A

hyperp

lanecanberep

arameterized

bysca

ling,e.g

.,

βTx+β0=

0is

thesameas

s(βTx+β0 )

=0.

•A

separatin

ghyperp

lanesatisfy

ing

yi (β

Txi+β0 )≥

1∀i

(i.e.,sca

ledso

thatc=

1)is

sometim

escalled

acanonica

lsep

aratin

g

hyperp

lane(C

ristianiniandShaw

e-Tay

lor2000).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-27

-

'&

$%

SeparatingHyperplanesandMargins

Margin (W

orse)

Margin (B

etter)

Fig

ure

7:Twosep

aratin

ghyperp

lanes,

onewith

alarger

margin

thanthe

other.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-28

-

'&

$%

TheSupportVectorMachine

•It

canbecalcu

lated

thatacanonica

lsep

aratin

ghyperp

lanehas

margin

equalto

1

‖β‖.

•Thesupport

vecto

rmachine(SVM)findsa“best”

(maxim

al

margin)canonica

lsep

aratin

ghyperp

laneto

separate

thetw

ocla

sses

(labelled

+1and−1)bysolving

min

12‖β‖2+γ

n∑i=1

ξi

s.t.ξi≥

0and

yi (β

Txi+β0 )≥

1−ξi∀i.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-29

-

'&

$%

ASVMforUnbalancedClasses

Let

w0andw1becla

ssweig

hts;

exten

dtheoptim

izatio

nproblem

tobe:

min

12‖β‖2+γ1

∑

yi=1

ξi+γ0

∑

yi=0

ξi

s.t.ξi≥

0and

yi (β

Txi+β0 )≥

1−ξi∀i,

where

γ0=γw0andγ1=γw1 .

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-30

-

'&

$%

SVM:Characterizin

gtheSolution

•Thesolutio

nforβ

ischaracterized

by

β=∑

i∈SV

αi yi x

i ,

where

αi≥

0(i

=1,2,...,n

)are

solutio

nsto

thedualoptim

izatio

n

problem

andSV,theset

of“support

vecto

rs”with

αi>

0strictly

positiv

e.

•This

meanstheresu

ltinghyperp

lanecanbewritten

as

f(x)=βTx+β0=∑

i∈SV

αi yi x

Tix+β0=

0.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-31

-

'&

$%

SVMsandRBFNetworks

•Canrep

lace

theinner

product

xTixwith

akern

elfunctio

nK(x;x

i )

toget

anonlin

eardecisio

nboundary:

f(x)=∑

i∈SV

αi yi K

(x;x

i )+β0=

0.

Theboundary

islin

earin

thespace

ofh(x)where

h(·)

issuch

that

K(u

;v)=〈h(u

),h(v

)〉is

theinner

product

inthespace

ofh(x).

•Hen

ceSVM

canbeview

edasanautomatic

way

ofconstru

ctingan

RBFnetw

ork

(Scholkopfet

al.1997).

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-32

-

'&

$%

PerformanceResults:

DrugDiscoveryData

Index of Split

12

34

0.18 0.20 0.22 0.24 0.26G

aussianT

riangularU

niformK

NN

SV

MA

SV

M

Averag

e Precisio

n

Fig

ure

8:Theavera

geprecisio

nofallalgorith

mseva

luated

onthetest

data.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-33

-

'&

$%

PerformanceResults:

ANOVASet-up

Let

µK,µ

S,µ

A,µ

U,µ

TandµG

betheavera

geresu

ltofK-N

N,SVM,

ASVM,andLAGO

usin

gtheunifo

rmkern

el,thetria

ngularkern

eland

theGaussia

nkern

el,resp

ectively.

Contra

stExpressio

nEstim

ate

Cntr1

µT−µG

0.0027

Cntr2

µG−µA

0.0339

Cntr3

µA−µS

0.0230

Cntr4

µS−

(µK+µU)/2

0.0157

Cntr5

µU−µK

0.0014

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-34

-

'&

$%

PerformanceResults:

ANOVASummary

Source

SS(×

10−4)

df

MS(×

10−4)

F0

P-V

alue

Meth

ods

233.504

546.701

64.307

<0.0001

Cntr1

0.140

10.140

0.193

0.6664

Cntr2

22.916

122.916

31.556

<0.0001

Cntr3

10.534

110.534

14.505

0.0017

Cntr4

6.531

16.531

8.994

0.0090

Cntr5

0.036

10.036

0.050

0.8258

Splits

18.877

36.292

8.664

0.0014

Erro

r10.893

15

0.726

Total

263.274

23

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-35

-

'&

$%

HitCurves:DrugDiscoveryData

0100

200300

400500

0 20 40 60 80 100 120

Total N

umber D

etected: n

Actual Hits: h(n)

Gaussian

Triangular

Uniform

KN

NS

VM

AS

VM

Sp

lit 1

0100

200300

400500

0 20 40 60 80 100 120

Total N

umber D

etected: n

Actual Hits: h(n)

Gaussian

Triangular

Uniform

KN

NS

VM

AS

VM

Sp

lit 2

0100

200300

400500

0 20 40 60 80 100 120

Total N

umber D

etected: n

Actual Hits: h(n)

Gaussian

Triangular

Uniform

KN

NS

VM

AS

VM

Sp

lit 3

0100

200300

400500

0 20 40 60 80 100 120

Total N

umber D

etected: n

Actual Hits: h(n)

Gaussian

Triangular

Uniform

KN

NS

VM

AS

VM

Sp

lit 4

Fig

ure

9:Only

theinitia

lpart

ofthecu

rves

(upto

n=

500)are

show

n.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-36

-

'&

$%

MainConclusions

☞

(Tria

ngle

LAGO∼

Gaussia

nLAGO)Â

ÂASVMÂ

SVMÂ

Â(U

nifo

rmLAGO∼

KNN).

☞

Computatio

nally,

ASVM

isextrem

elyexpen

sive.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-37

-

'&

$%

TheNumberofSVs

SVM

ASVM

C0

C1

C0

C1

Split

111531

294

5927

291

Split

211419

303

3472

284

Split

311556

293

11706

290

Split

41863

293

6755

281

TotalPossib

le14602

304

14602

304

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-38

-

'&

$%

Empirica

lEvidence:β=0

Co

nto

ur o

f AP

: Sp

lit 1

α

β

01

23

45

−1.0 −0.5 0.0 0.5 1.0

Co

nto

ur o

f AP

: Sp

lit 2

α

β

01

23

45

−1.0 −0.5 0.0 0.5 1.0

Co

nto

ur o

f AP

: Sp

lit 3

α

β

01

23

45

−1.0 −0.5 0.0 0.5 1.0

Co

nto

ur o

f AP

: Sp

lit 4

α

β

01

23

45

−1.0 −0.5 0.0 0.5 1.0

Fig

ure

10:Choosin

gαandβ(w

hile

fixingK

=5)usin

g5-fo

ldCV.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-39

-

'&

$%

Refe

rence

s

Cristia

nini,N.andShaw

e-Tay

lor,

J.(2000).AnIntrod

uctio

nto

Support

Vecto

rMachines

andOther

Kern

el-based

LearningMeth

ods.

Cambrid

geUniversity

Press.

Hastie,

T.J.,Tibshira

ni,R.J.,andFried

man,J.H.(2001).The

Elem

entsofStatistica

lLearning:

Data-M

ining,Inferen

ceand

Pred

iction.Sprin

ger-V

erlag.

Scholkopf,B.,Sung,K.K.,Burges,

C.J.C.,Giro

si,F.,Niyogi,P.,

Poggio,T.,andVapnik,V.(1997).

Comparin

gsupport

vecto

r

machines

with

gaussia

nkern

elsto

radialbasis

functio

ncla

ssifiers.

IEEETransactio

nsonSign

alProcessin

g,45(11),2758–2765.

Zhu,M.,Su,W

.,andChipman,H.A.(2005).

LAGO:A

computatio

nally

efficien

tapproach

forsta

tisticaldetectio

n.WorkingPaper

2005-01,

Dep

artm

entofStatistics

andActu

aria

lScien

ce,University

of

Waterlo

o.Toappearin

Tech

nometrics.

LA

GO

Copyrig

ht

c©2003–2005

by

Mu

Zhu

-40

-