detecting influential observations in dea wilson

8/2/2019 Detecting Influential Observations in DEA WILSON

1/19

T h e J o u r n a l o f P r o d u c t i v i t y A n a l y s i s , 6 , 2 7 - 4 5 ( 1 9 9 5 )9 1 9 9 5 K l u w e r A c a d e m i c P u b l i s h e r s , B o s t o n . M a n u f a c t u r e d i n t h e N e t h e r l a n d s .

Detecting Influential Observations in DataEnvelopment AnalysisP A U L W . W I L S O N *Department of Economics, University of Texas, Austin, Texas 78712 [email protected], utexas.edu

A b s t r a c tT h i s p a p e r p r o v i d e s d i a g n o s t i c to o l s f o r e x a m i n i n g t h e r o l e o f i n f l u e n t ia l o b s e r v a t i o n s i n D a t a E n v e l o p m e n t A n a l y s i s( D E A ) a p p l i c a t io n s . O b s e r v a t i o n s m a y b e p r i o r i ti z e d f o r f u r t h e r s c r u t in y t o s e e i f t h e y a r e c o n t a m i n a t e d b y d a t ae r r o r s ; t h i s p r i o r i t iz a t J o n i s i m p o r t a n t i n s i tu a t i o n s w h e r e d a t a - c h e c k i n g i s c o s t l y a n d r e s o u r c e s a r e l i m i t e d . S e v e r a le m p i r i c a l e x a m p l e s a r e p r o v i d e d u s i n g d a t a f r o m p r e v i o u s l y p u b l i s h e d s t u d i e s .Keywords:D E A , i n f l u e n t ia l o b s e r v a t i o n s , o u t l i e rs ,

1 . I n t r o d u c t i o n

D e t e r m i n i s t i c f r o n t i e r m e t h o d s h a v e b e c o m e w i d e l y u s e d i n m e a s u r i n g e f f ic i e n c y i n p r o -d u c t i o n [ se e L o v e l l , ( 1 9 9 3 ) f o r e x a m p l e s o f a p p li c a ti o n s ]. T h e s e m e t h o d s t y p i c a l l y i n v o lv ec o n s t r u c t i n g a d e t e r m i n i s t ic f r o n t i e r a n d m e a s u r i n g e f f ic i e n c y i n t e r m s o f d i s ta n c e s i n i n -pu t /ou tpu t space f ro m the f ron t ie r . The d e te rmin is t i c f ron t ie r i s o f ten cons t ruc ted v ia l inea rp r o g r a m m i n g ( L P ) t e c h n i q u e s ( t h e se a p p r o a c h e s h a v e b e e n t e r m e d D a t a E n v e l o p m e n tA n a l y s i s , o r D E A ) , a l t h o u g h o t h e r t e c h n i q u e s s u c h a s t h e F r e e D i s p o s a l H u l l ( F D H ) o fD e p r i n s e t a l . ( 1 9 8 4 ) a r e s o m e t i m e s u s e d . U n f o r t u n a te l y , th e d e t e r m i n i s t ic n a t u r e o f th ef ron t ie rs means tha t da ta e r ro r s ( r esu l t ing f rom m easurem ent e r ro r s , cod ing e r ror s , o r o th e rprob lem s) in obse rva t ions on dec i s ion-making un i t s (DM Us) tha t suppor t the de te rmin is t i cf r o n t i e r m a y s e v e r e l y d i s t o rt m e a s u r e d e f f ic i e n c y sc o r e s f o r s o m e ( o r p e r h a p s a l l) o f t h er e m a i n i n g D M U s . T h i s i s a n a lo g o u s t o t h e p r o b l e m o f o u t li e r s i n c l as s ic a l l in e a r r e g r e s -s i o n ( C L R ) m o d e l s .

Out l i e r s a r e obse rva t ions tha t d o n o t f i t i n w i t h t h e p a t t e r n o f t h e r e m a i n i n g d a t a p o i n t sa n d a r e n o t a t a l l t y p i c a l o f th e r e st o f t h e d a t a ( G u n s t an d M a s o n , 1 9 8 0 ). R o u s s e e u wa n d v a n Z o m e r e n ( 1 9 9 0 ) r e f e r t o c e r t a in t y p e s o f o u f l ie r s a s l e v e ra g e p o in t s ; t h e s e o b s e r v a -t i o n s h a v e a d i s p r o p o r t i o n a t e e f fe c t o n t h e e s t i m a t e d s l o p e o f th e r e g r e s s io n l i n e i n C L Rm o d e l s , a n d t h u s m a y p o s e a m o r e s e r i o u s p r o b l e m t h a n o t h e r o u t li e r s w h i c h m i g h t o n l ya f f ec t the e s t ima ted in te r cep t t e rm . Out l i e rs ( inc lud ing leve rage po in t s ) a r e somet im es ca l ledinf luen t ia l obse rva t ions becau se they have a d i spropor t iona te e f f ec t in de te rmin ing the es t i-m a t e d r e su l ts i n C L R m o d e l s . I n t h i s p a p e r, in f luen t ia l observa t ions a r e d e f i n e d a s t h o s e* T h is r e s e a r c h w a s p e r fo r m e d w h i l e u n d e r c o n t r a c t w i t h th e M a n a g e m e n t S c i e n c e G r o u p , U . S . D e p a r t m e n t o fV e t e r a n s A f f a i r s , B e d f o r d , M A 0 1 7 3 0 . S h a w n a G r o s s k o p f a n d R i c h a r d G r a b o w s k i g r a c i o u s l y p r o v i d e d d a ta u s e di n t w o o f th e e m p i r i c a l e x a m p l e s .


2/19

28 P .W. WI L SO N

samp le obse rva t ion s which p lay a re la t ive ly l a rge ro le in de te rm in ing es t ima ted e f f i c iencyscores fo r a t l eas t som e o the r o bse rva t ions in the obse rved sam ple . Thu s , in f luen t ia l obse r -va t ions in the con tex t o f th i s pape r wi l l typ ica l ly be ou t l i e r s which suppo r t the de te rmin i s t i cf r o n t i e r u s e d t o m e a s u r e e f f i c i e n c y ( i n f lu e n t i a l o b s e rv a t i o n s o w e t h e i r i n f l u e n c e t o t h e f a c tt h a t t h e y a r e o u t l i e r s ) . S o m e o u t l i e r s m a y r e s u l t f r o m r e c o r d i n g o r m e a s u r e m e n t e r r o r sa n d s h o u l d b e c o r r e c t e d , i f p o s s i b l e , o r d e l e t e d f r o m t h e d a ta . H o w e v e r , i f d a t a a r e v i e w e da s h a v i n g c o m e f r o m s o m e p r o b a b i l i t y d i s t r i b u t i o n , t h e n i t i s q u i t e p o s s i b l e t o o b s e r v ep o i n t s w i t h l o w p r o b a b i l i ty ; o n e w o u l d n o t e x p e c t t o o b s e r v e m a n y s u c h p o i n ts , g i v e n t h e i rl o w p r o b a b il i ty , a n d h e n c e t h e y a p p e a r a s o u t l ie r s . C o o k a n d W e i sb e r g ( 1 9 8 2 ) o b s e r v e dt h a t o u t l ie r s o f th i s t y p e m a y l e a d t o t h e r e c o g n i t i o n o f i m p o r t a n t p h e n o m e n a t h a t m i g h to t h e r w i s e g o u n n o t i c e d . O u t l i e r s o f th i s t y p e m i g h t r e p r e s e n t t h e m o s t i n t e r e s ti n g p a r t o fthe da ta .W h i l e a p l e t h o r a o f d i a g n o s t i c t o o l s e x i s t w i t h w h i c h t o e x a m i n e a n d e v a l u a t e r e s u l t sf r o m C L R m o d e l s, 1 t h e r e i s a d i s t i n c t l a c k o f su c h t o o l s s u i t ab l e f o r a p p l i c a ti o n s w h e r ee f f i c i e n c y i s m e a s u r e d r e l a t iv e t o a d e t e r m i n i s ti c f r o n t i er . T h i s p a p e r r e d r e s s e s p a r t o f t h isd e f i c i e n c y b y s u g g e st in g a m e t h o d o l o g y t o d e t e c t i n f lu e n t i a l o b s e r v a t i o n s a m o n g a s a m p l eo f D M U s . N o t e , h o w e v e r, th a t i t w i l l r e m a i n f o r t h e a n a l y s t t o d e c i d e w h a t t o d o w i t hany in f luen t ia l obse rva t ions tha t a re foun d . Th e use fu lnes s o f the t echn iques p resen ted be lowis in iden t i fy ing obse rva t ions wh ich dese rv e c lose r s c ru t iny to de te rmine w he the r the obse r -va t ions a re rea l , o r wh e the r they re su l t f rom da ta e r ro rs . S ince da ta -check ing i s o f ten cos tly ,p a r t i c u l a r l y i n l a r g e s a m p l e s , i d e n t i f y in g a n d p r i o r i t iz i n g o b s e r v a t i o n s f o r c h e c k i n g m a ys a ve c o n s i d e r a b l e r e s o u r c e s .

S t a n d a r d m e t h o d s f o r d e t e c t i n g o u t li e r s in l i n e a r r e g r e s s i o n m o d e l s u s i n g o r d i n a r y l e a s ts q u a r e s ( O L S ) r e s i d u a l s c a n n o t b e a d a p t e d t o D E A o r F D H m o d e l s d u e t o t h e l a c k o fs t o c h a s ti c s t r u c t u r e [ d e t e rm i n i s t ic p a r a m e t r i c m o d e l s e s t i m a t e d b y c o r r e c t e d O L S ( C O L S )s u c h a s th o s e e s t i m a t e d b y G r e e n e , ( 1 9 8 0 ) , M a r t i n a n d P a g e , ( 1 9 8 3 ) , S e a v e r a n d T r i a n ti s ,( 1 9 8 9 ) , a n d o t h e r s d o n o t h a v e t h is p r o b l e m ) . S e x t o n ( 1 9 8 6 ) i l lu s t ra t e s th e p r o b l e m c a u s e db y o u t l i e r s i n t h e m e a s u r e m e n t o f t e c h n i c a l e ff i c i e n c y b y e s t i m a t in g a D E A m o d e l u s i n gda ta repor te d in Ch arnes e t a l . (1981) an d showing wh a t happens to the e s t ima ted e f f i c iencys c o r e s w h e n o n e o b s e r v a t i o n i s d e li b e r a t e l y m i s c o d e d . S e x t o n c o u n s e l s t h e u s e o f p r e p r o c -e s s in g e r r o r d e t e c t i o n r o u t in e s w h e n e v e r p o s s i b le , b u t m a k e s n o s p e c i f ic s u g g es t io n s o t h e rt h a n n o t i n g t h a t th e s t a n d a r d t e c h n i q u e s b a s e d o n O L S r e s id u a l s c a n n o t b e u s e d . I n t h ec a s e o f l a r g e s a m p l e s, i t m a y n o t b e p o s s i b le t o c h e c k e a c h i n d iv i d u a l o b s e r v a ti o n f o r a c c u -r a cy , a n d s o s o m e m e t h o d o f p r i o r i ti z i n g o b s e r v a t i o n s f o r c h e c k i n g is r e q u i r e d .

Seve ra l D EA s tud ies (e .g . , C harne s and Nera l id , 1990) have used sens i t iv i ty ana lys i st o g a u g e th e r o b u s t n e s s o f re s u l ts f r o m D E A m o d e l s . C h a r n e s e t al . ( 1 9 9 2 ) n o t e t h a t s u c ha n a l y s e s u s u a l l y h i n g e o n c o n d i t i o n s w h i c h p r e s e r v e t h e f e a s ib i l it y a n d o p t i m a l i t y c o n d i -t i o n s o f a n o p t i m a l b a s i c s o l u t i o n o b t a in e d v i a a n e x t r e m e p o i n t a l g o r i t h m , a n d p r o v i d ea m o r e c o m p r e h e n s i v e a p p r o a c h w h e r e i n f o r e a c h D M U a r e g i o n o f s ta b i li ty is c a l c u l a te ds u c h t h a t a ll p e r t u r b a t i o n s w i t h in t h e c e l l p r e s e r v e t h e D M U ' s c l a s s if i c a ti o n a s e f f ic i e n tv e r s u s i n e f f i c i e n t. S i m i l a rl y , G r o s s k o p f a n d V a ld m a n i s ( 1 9 8 7 ) e x a m i n e e f f i c ie n c y a m o n gh o s p i ta l s u s in g D E A a n d p e r f o r m a s e n s it i v it y a n a ly s i s b y u si n g a l t e r n a ti v e m e a s u r e s f o rs o m e i n p u ts a n d e x a m i n i n g t h e e f f e c t o n t h e m e a s u r e d e f f i c i e n c y s c o r e s. T h e s e t e c h n i q u e sa r e n o t t o b e c o n f u s e d w i t h a t te m p t s t o i d e n t i f y o u t li e r s. S e n s i t iv i t y a n a ly s e s c h e c k t h er o b u s t n e s s o f e s ti m a t e d e f f i c i e n c y s c o r e s w i t h r e s p e c t t o d e v i a t io n s o f o b s er v a t i o n s f r o m


3/19

DETECTING INFLUEN TIALOBSERVATIONS N DATA ENV ELOP ME NTANALYSIS 29

the i r in i t i a l loca t ion in the inpu t -ou tpu t space , whereas ou t l i e r de tec t ion invo lves f ind ingo b s e r v a t i o n s w h o s e l o c a t i o n i n t h e i n p u t o u t p u t s p a c e i s a t y p i c a l .

As sugges ted above , the l it e ra tu re on o u t l i e r de tec t ion in the con tex t o f de te rm in i s t i c f ron-t i e r mo de l s i s spa rse . W i l son and J ad low (1982) f i t a de te rmin i s t i c pa ram e t r i c f ron t i e r us ingLP techn iques , and then de le te obse rva t ions ly ing on the e s t ima ted f ron t i e r un t i l the pa ram -e te r e s t ima te s s t ab i li ze . De le t ion o f obse rva t ions on the f ron t i e r which a re ou t l i e r s wi l ll i k e ly p r o d u c e l a r g e c h a n g e s i n t h e p a r a m e t e r e s t i m a t e s, w h i l e d e l e t in g o b s e r v a t i o n s o nt h e f r o n t i e r t h a t a r e n o t o u t l ie r s w i l l l i k e l y h av e o n l y a s m a l l e f f e c t o n p a r a m e t e r e s t i m a t e s.W i l so n a n d J a d l o w d o n o t s t a te w h e t h e r o b s e r v a t i o n s a r e d e l e t e d s e q u e n ti a ll y , p a i r w i s e ,o r o t h e r w i s e . T h e i r m e t h o d i s a v a r i a ti o n o f t h a t u s e d b y T i m m e r ( 1 9 7 1 ) , w h o d e l e t e s af ixed perce ntage of the observat ions ly ing on the in i tial f ront ie r . Du sansky and W ilson (1994,1 9 9 5 ) u s e s i m i l a r a p p r o a c h e s i n D E A m o d e l s .S e a v e r a n d T r i an t i s ( 1 9 8 9 ) p e r f o r m a n e x t e n s i v e a n al y s is o f o u tl i e r s in e s t i m a t i n g a f r o n -t i e r p r o d u c t i o n m o d e l b y C O L S , a n d s ta t e t h a t th e m e t h o d s t h e y u s e a r e a p p l i ca b l e in d e t e r -m i n i s t i c n o n p a r a m e t r i c D E A m o d e l s a s w e l l. H o w e v e r , t h is i s n o t p o s s i b l e ( w i t h t h e e x -cep t io n o f the AP s ta ti s ti c a s d i s cus sed by W i l son , 1993) , s ince OL S re s idua l s a re no tavai lable . W ilson (1993) p rovides a d iagnos t ic s tat is tic wh ich m ay be u sed to ident i fy out l ie rsin de te rmin i s t i c f ron t i e r m ode l s w i th mul t ip le inpu ts and mu l t ip le ou tpu ts , bu t th i s approachb e c o m e s c o m p u t a t i o n a l l y i n f e a s ib l e a s t h e n u m b e r o f o b s e r v a t io n s a n d t h e d i m e n s i o n o fthe input-outpu t space increase (e .g . , w i th 7 d im ens ions in the input output space , the s tat is ticb e c o m e s c o m p u t a t i o n a l ly i n f e a s ib l e f o r m o r e t h a n a b o u t 1 0 0 o b s e r v a ti o n s ) . T h e m e t h o d -o l o g y p r e s e n t e d b e l o w i s m u c h l e s s c o m p u t a t i o n a l l y i n t e n s i v e , a n d t h u s m a y b e a p p l i e di n m u c h l a r g e r s a m p l e s t h a n t h e s t a t i s t i c p r o p o s e d b y W i l s o n ( 1 9 9 3 ) .

T h e n e x t s e c t io n p r e s e n ts a m o d i f i c a t i o n o f t h e s t a n d a r d D E A m o d e l w h i c h a v o i d s ac e n s o r i n g p r o b l e m i n h e r e n t in c o n v e n t i o n a l D E A f o r m u l a t i o n s . T h e t h i r d s e c t i o n s u g ge s tsa s t ra t e g y f o r d e te c t in g i n f lu e n t ia l o b s e rv a t i o n s u s i n g t h e m o d i f i e d D E A m o d e l . T h e f o u r t hsec t ion p re sen t s s eve ra l em pi r i ca l exam ples ; conc lus ions a re d i s cus sed in the f ina l s ec t ion .

2 . A Mod i f i e d D E A m e th od o l ogyTo i l lus tra te the de tec t io n o f in f luen t i a l obse rv a t ions in DE A app l ica t ions , con s ide r thei n p u t w e a k e f f i c i e n c y ( I W E ) s c o r e d i s c u s s e d b y F f i re e t a l . ( 1 9 8 5 ) [ t he I W E s c o r e i s th ei n v e r s e o f t h e S h e p h a r d ( 1 9 7 0 ) i n p u t d i s ta n c e f u n c t i o n ] . T h i s p a r t i c u l a r D E A m o d e l h a sb e e n w i d e l y a p p l i e d , a n d t h e m e t h o d o l o g y to b e p r e s e n t e d b e l o w m a y b e e a s i l y e x t e n d e dt o o t h e r D E A m o d e l s o r t h e F D H m e t h o d . G i v e n a s a m p le o f N D M U s , t h e I W E s c o r ef o r t h e i t h D M U i s c o m p u t e d b y s o l v i n g t h e l i n e a r p r o g r a m

m i n { X i l Y i < - Yq i , X i x i > - Xq i , T q i = 1 , q i E 6 t N } (1 )w h e r e qi i s a (N x 1 ) vec to r o f we igh t s to be com puted , 1 i s a (1 N) vec to r o f ones ,Xi is a scalar, x i i s a (K x 1) vec to r o f inpu t s fo r the i th D M U, Yi i s a (M 1) ve c to ro f o u tp u t s f o r t h e it h D M U , X = [ X l , . . . , XN ] i s a ( K x N ) m a t r i x o f o b s e r v e d in p u t s,a n d Y = [Y b . . . , YN ] i s a ( M x N ) m a t r i x o f o b s e r v e d o u t p u ts . T h e m i n i m a n d X im e a s u r e s t h e i n p u t w e a k e f f i c i e n c y o f th e i t h D M U ; t y p i ca l ly , o n e w o u l d s o l v e ( 1) N


4/19

3 0 P.W. WILSON

t im e s , o n c e f o r e a c h D M U . T h e c o n s t r a i n t 1 q i = 1 r esu l t s in a va r iab le r e tu rns to s ca let e c h n o l o g y ; c o n s t a n t re t u r n s t o s c a l e m a y b e i m p o s e d b y o m i t ti n g t h is c o n s t r a i n t f r o m t h eL P p r o b l e m i n ( 1 ) . T h e o b s e r v a t i o n s t h a t a p p e a r i n t h e c o n s tr a in t s o f t h e L P p r o b l e m i n( 1 ) c o m p r i s e t h e r e f e r ence s e t , and in the cas e o f (1 ) inc lude a l l obse rva t ions in the samp le .Clea r ly , Xi < 1 , wi th Xi = 1 ind ica t ing an e f f ic ien t D M U ly ing on the measu red boun-d a r y o f t h e p r o d u c t io n s e t. D M U s f o r w h i c h ) k < 1 a r e r ega rded as t echn ica l ly ine ff ic ien t .

T o i l lu s t ra t e t h e IW E s c o r e , c o n s i d e r 7 D M U s , i = 1 , . . . , 7 , l a b e l e d A , B , C , D , E , F,a n d G r e s p e c t i v e l y w h i c h e a c h p r o d u c e t h e s a m e l e v e l o f o u t p u t f r o m t w o i n p u t q u a n t i ti e sX l, x 2. I n p u t a n d o u t p u t q u a n ti t ie s f o r t h e s e i l lu s t ra t iv e D M U s a p p e a r i n c o l u m n s 2 - 4 o fT a b l e 1 , a n d a r e p l o t t e d in ( x l , x 2 ) -s p a c e i n F i g u r e 1. F o r D M U s C a n d D , I W E i s c o m -p u t e d b y O C / O C a n d O D ' / O D , r e spec t ive ly . He nc e C i s os tens ib ly e f f ic ien t a s ind ica tedby i t s IW E score equ a l to 1 , wh i le D i s ine f f ic ien t since i t l i es wi th in the in te r io r o f thep r o d u c t i o n s e t b o u n d e d b y t h e p i e c e w i s e - li n e a r p r o d u c t i o n - s e t b o u n d a r y p a s s in g t h r o u g hA, C , and G . In add i t ion to C , DM U s A and G a r e a l so e f f ic ien t ; B , D, E , and F a r e con-s i d e r e d i n e f fi c i e n t b y t h e I W E m e a s u r e . I n a t y p i c a l d a t a s e t , t h e r e m a y b e m a n y o b s e r v a -t ions such as A, C , and G wh ich l i e on the com pute d f ron t ie r . Conseq uent ly , the d i s t r ibu-t i o n o f e f f i c ie n c y sc o r e s w i l l o f t e n i n c lu d e a m a s s a t o n e i n t h e c a s e o f t h e 1 W E s c o re .

C o m p u t i n g t h e l i n e a r p r o g r a m i n ( 1 ) f o r e a c h o f N D M U s y i e l d a s e t o f e ff i c ie n c y s c o r e s{~ , i l i = 1 . . . . N } . T h e s e t ~ = { i l ~ , i - - 1 , i = 1 , . . . , N} is t e rm ed the e f f ic i e n t s u b s e ta n d i s th e s e t o f D M U s w h i c h s u p p o r t t h e m e a s u r e d b o u n d a r y o f t h e p r o d u c t io n s e t.Ko ros te lev e t a l . (19 94) show that for a f ini te sample o f ident ically , indepen dent ly dis t r ibutedo b s e r v a t i o n s (xi, Yi) d r a w n f r o m a s e t g , t h e p r o d u c t i o n s e t b o u n d a r y i m p l i e d b y (1 ) an do t h e r D E A m o d e l s i s b i a se d d o w n w a r d r el a ti v e t o t h e t r u e b o u n d a r y o f 9 . H e n c e , t h e o b s e r -va t ions in ~ a r e no t necessa r i ly e f f ic ien t in the t r ue s e n s e , e v e n t h o u g h t h e i r m e a s u r e de f f i c ie n c y s c o re s e q u a l u n i ty . T h e r e f o r e th e o b s e r v a ti o n s i n g a r e r e g a r d e d a s o s t e n s i b l ye f f ic ien t .

A nde r sen and Pe te r sen (1989) and Love l l e t a l . ( 1993) sugges t a modi f ica t ion to the DE Am o d e l t o a l lo w r a n k i n g o f o s t e n s i b l y e f f i c i e n t o b s e r v a t io n s , z T h i s m o d i f i c a t io n i s a l s ouseful for ident i fying inf luent ia l observat ions . In (1) , ef f ic iency for the i th D M U is me asuredr e l a ti v e t o a ll D M U s i n t h e s a m p l e , i n c l u d in g t h e i t h D M U . T h e m o d i f i c a t i o n u s e db y A n d e r s e n a n d P e t e r s e n a n d b y L o v e l l e t a l. i n v o l v e s r e m o v i n g t h e i t h D M U f r o m

T a b l e 1 . Illustrative case for Figure 1.DMU x 1 x 2 y X

A 0.50 8.00 2.00 4.0000B 2.00 6.00 2.00 0.8519C 3.00 2.00 2.00 1.6000D 4.00 6.00 2.00 0.5897E 5.00 3.00 2.00 0.6500F 7.00 4.00 2.00 0.4815G 8.00 1.00 2.00 2.0000


5/19

DETEC TING INFLU ENTIAL OBSERVATIONS N DATA ENV ELOPM ENT ANALYSIS 3 1

371

O

AA

tllB /911D

/ / // / e F

t l E

t / / // / G/ / v/ 10 ) X 2

Figure 1. Inp ut w eak technical efficiency.t h e c o n s t r a in t s e t w h e n e f f i c i e n c y f o r t h e i t h D M U i s c o m p u t e d . F o r t h e I W E s c o r e , t h el i n e a r p r o g r a m b e c o m e s

m i n { h * l y i _-_ y ( i ) q , , h . x i >_ x ( i ) q . , T q * ~ (it { } (2)w h e r e X * i s t h e m o d i f i e d IW E s c o r e f o r t h e i t h D M U , q * i s a v e c t o r o f w e i g h t s, X ( i ) =[ x j ] V j ~ i , y ( i ) = [ y j ] V j ~ i , a n d o t h e r v a r i a b l e s a r e d e f i n e d a s b e f o r e . H e n c e X ( 0a n d y ( i ) have dim ensio ns K (N - 1) and M (iV - 1), respect ively, and q* has dim en-s i o n s 1 x ( N - 1 ).

U n l i k e h i i n ( 1 ), X * is n o t b o u n d e d f r o m a b o v e a n d h e n c e X * > 0 . T h e m o d i f i e d s c o r eX * m e a s u r e s t e c h n i c a l e f fi c i e n c y f o r th e i t h D M U r e l a ti v e to a l l o t h e r D M U s i n th e s an >ple . F or X* < 1 , the in te rpre ta t ion i s the sam e as fo r h i com pu ted f ro m (1) . How ever ,fo r X* __- 1 , X* i s in te rpre ted as the amount by which the inpu t vec tor f o r DMU i mayb e p r o p o r t i o n a t e l y in c r e a s e d w i t h o u t b e in g d o m i n a t e d b y a l i n e a r c o m b i n a t i o n o f th e r e -m a i n i n g D M U s i n t h e s a m p l e .

F o r D M U C i n th e a b o v e e x a m p le , t h e m o d i f ie d I W E s c o r e is c o m p u t e d a s O C ' / O Ca s s h o w n i n F i g u r e 2 , a n d c l e a r l y e x c e e d s u n i ty . I n F i g u r e 2 , e f f i c i e n c y f o r D M U C i s


6/19

32 P.W. WILSON

X l

0

A

B D

F

r , / / ~

J/ .

) X2

Figure 2. Modified nput weak echnicalefficiency.me asured re la tive to f ront ier formed whe n C i s dropped f rom the sample . The m odi f ica-t ion only affects eff iciency m easu red for DM Us A , C, and G , wh ich formed the ef f ic ien tsubset in Figure 1. For D M Us B, D, E, a nd F, k* = ~. H enc e the front ier shown in Figure2 i s only relevant for m easur ing the technical ef f ic iency of DM U C; for each o f the D M Usin the efficient subset 8 , efficiency is me asu red relative to a different frontier in the mod ifiedapproach. Thus, w hile the mo dified approach in (2) is useful in detecting influential obser-vations in the n ext section, the efficiency scores fro m (1) and the fron t ier implied by (1)m ay have a m ore intui tive eco nom ic meaning. Us e of (2) for diagnostic pu rposes does n otincre ase computational requirements, however; on e ca n simply co m pute k~' fro m (2) andthen se t

) , i = ~ i * i f k * < 1;o therwise .

(3)

This is equivalent to computing hi directly from (1).


7/19

DETECTING INFLUENTIALOBSERVATIONS N DATA ENV ELOPM ENT ANALYSIS 33

N o t e t h a t it is p o s s i b l e t h a t X * c a n n o t b e c o m p u t e d f o r s o m e o b s e r v a t i o n s i f d e l e ti o no f o b s e r v a t i o n i f r o m t h e r e f e r e n c e s e t r e s u lt s i n a n i n f e a s i b l e c o n s t r a i n t s e t in ( 2 ). T h ei n f e a s ib i l it y p r o b l e m r e s u l t s w h e n a n o b s e r v a t i o n l ie s a b o v e t h e f r o n t i e r s u p p o r t e d b y t h eo the r obse rva t ions in t he samp le , so t ha t ne i t he r a r ad i a l con t r ac t i on nor ex pans ion o f i npu t s,ho ld ing ou tpu t cons t an t , w i l l r each t he f ron t i e r . Th i s s i t ua t i on i s i l l us t r a t ed i n F igure 3f o r th e c a s e o f a s i n g l e i n p u t ( x) m a p p e d i n t o a s in g l e o u t p u t ( y ) . T h i s p r o b l e m c o u l d b eavo ided by us ing t he cons t an t r e tu rns t o sca l e fo rmu la t i on (by d ropp ing the cons t r a in t -~ q*= 1 i n (2 ) ) . Th e i n feas ib i l i t y o f t he cons t r a in t s e t fo r ob se rva t ion i m ean s t ha t X* i su n d e f i n e d . T h i s p r o b l e m d o e s n o t o c c u r f o r X i c o m p u t e d f r o m ( 1 ), s i n c e o b s e r v a t i o n i i si n c l u d e d i n t h e r e f e r e n c e s e t . N o t e t h a t f o r a n y o b s e r v a t i o n s i s u c h t h a t X * i s u n d e f i n e ddue to an in feas ib l e cons t r a in t s e t , X i = 1 .

G i v e n a s a m p l e o f o b s e r v a t i o n s o n N D M U s , (1 ) a n d ( 2) m a y b e u s e d t o c o m p u t e { X il i= 1 , . . . , N } a n d { X * ] i = 1, . . . , N } , r e s p e c t i v e ly . A s n o t e d e a r li e r, t h e d i s tr i b u ti o nof t he Xi w i l l t yp i ca l l y i nc lude a ma ss a t un i ty ; t h i s i s o f t en i nd i ca t i ve o f a cen sore dd i s t ri bu t ion . 3 Th i s i s co nf i rm ed by the d i s t r i bu t ion o f the X* which , i f censored f rom abovea t un i ty a s i n (3 ) , be co m es i den t i ca l to t he d i s t r i bu t ion o f X . I n add i t i on , t he v i ew tha t

Y

O

C'? - - - - e - - - -

--)

Figure 3. Infeasible constraint sets.


8/19

3 4 P.W. WILSO N

X is cen sore d is cons i s ten t wi th s tud ies such as M cC ar ty and Ya isawarng (1993) , D usan-sky and W i l son (1994) , and o the rs which have used censo red regres s ion (e .g . , Tob it ) mod e lsto regres s e f f i c iency scores f ro m m ode ls such as (1 ) on fac to rs tha t m igh t in f luen ce measure dtechnica l eff ic iency. Ce nsorin g involves a loss of informat ion ; c lear ly , {X* l i = 1 . . . , N }conta ins m ore in form at ion about the N D M Us in the obse rved sample than {Xil i = 1, . . . ,N } . T h e a d d it i o n a l i n f o r m a t i o n c o n t a in e d in {X * [ i = 1, . . . , N } i s u s e f u l i n d e t e c ti n ginf luen t ia l obse rva t ions in the s ample .

3 . D e t e c t i n g I n f l u e n t i a l O b s e r v a t i o n s

D E A m o d e l s s u c h a s ( 1) a n d ( 2 ) m e a s u r e e f f i c i e n c y r e l a t iv e t o a d e t e r m i n i s t i c f r o n t i e r s u p-por ted by the s ample obse rva t ions . Consequen t ly , the re su l t ing e f f i c iency scores incorpora tes ta t is ti c al n o i s e a s w e l l a s i n e f f ic i e n cy . F u r t h e r m o r e , a n y m e a s u r e m e n t e r r o r , c o d i n g e r -r o r s , o r o t h e r d a t a e r r o r s w i l l a f f e c t t h e m e a s u r e d e f f i c ie n c y s c o r e s. F o r o b s e r v a t io n s i nt h e e f f i c i e n t s u b s e t, t h is p r o b l e m i s a c u te s i n c e a n y e r r o r s w i ll n o t o n l y a f f e c t e ff i c i e n c ym e a s u r e d f o r th e D M U w i t h t h e d at a e r r o r , b u t p o s s i b ly o t h e r D M U s a s w e ll . T h e s e o b s e r -va t ions sugges t th ree ques t ions rega rd ing obse rva t ions in the e f f i c ien t subse t :( i) H o w c o n f i d e n t c a n w e b e th a t a n o b s e r v a t io n i n t h e e f f ic i e n t s u b s e t i s r e a ll y e f fi c i e n t

r e l a t iv e t o o t h e r D M U s i n th e s a m p l e ; i . e ., h o w m u c h s u p p o r t i s t h e r e f o r t h e f r o n t i e ri n t h e n e i g h b o r h o o d o f s u c h a n o b s e r v a t i o n ?

( ii ) H o w m a n y e f f i c ie n c y s c o r e s f o r o t h e r D M U s a r e a f f e c t e d b y th e p r e s e n c e o f a n o b s e r-v a t i o n ( p o s s i b l y m e a s u r e d w i t h e r r o r ) i n t h e e f f i c i e n t s u b s e t ?

( ii i) H o w m u c h d o e s t h e p r e s e n c e o f a n o b s e r v a t i o n i n th e e f f i c i e n t s u b s e t a f f e c t t h e m e a s -u r e d e f f ic i e n cy o f o t h e r D M U s ?

Ins igh t rega rd ing the answer to ques t ion ( i ) i s ob ta ined by examin ing the ad jus ted e f f i -c i e n c y s c o r e s c o m p u t e d f r o m ( 2 ) . F o r t h e e x a m p l e p i c tu r e d i n F i g u r e s 1 - 2 , t h e s e s c o r e sa re sh own in Tab le 1 . As no ted ea r l ie r , D M U s for which X* >_ 1 a re in the e f f i c ien t subse t ,w h i c h i n t h is c a s e i n c l u d e D M U s A , C , a n d G . F o r D M U C , t h e m o d i f i e d e f fi c i e n c y s c o r ei n d i ca t e s t h a t t h i s D M U ' s i n p u ts w o u l d h a v e to b e i n c r e a s e d b y 6 0 p e r c e n t t o r e a c h t h ef ron t ie r suppo r ted by the o the r o bse rva t ions in the s ample . T h is i s a re la t ive ly la rge amoun t ,a n d i n d i c at e s t h a t t h e r e i s l it tl e s u p p o r t f o r t h e f r o n t i e r i n t h e n e i g h b o r h o o d o f D M U C. 4Fo r D M U s A an d G, the ad jus ted e f f i c iency score i s even l a rge r, aga in sugges ting tha t the rei s li tt le s u p p o r t f o r th e f r o n t i e r i n t h e n e i g h b o r h o o d s o f th e s e D M U s . T h e v a l u e s o f X *f o r D M U s A, C, and G ind ica te tha t these obse rva t ions a re a ty p ica l ( i . e . , ouf l i e r s ) . In th i sexample , on e shou ld thus be susp ic ious o f any o f the D M Us in the measure d e f f i c ien t subse tbe ing e f f i c ien t in a t rue s ense . S in ce the s ample s ize is ve ry sm a l l he re , th is sho u ld no tb e a s u r p r i s i n g c o n c l u s i o n ( t h e n e x t s e c t i o n g i v e s s o m e e m p i r i c a l e x a m p l e s w i t h l a r g e rsample s izes ) .

To examine ques t ions ( i i) and ( i ii ) , cons ide r on ce aga in the exam ple represen ted in F igures1 - 2 . I n p a r t ic u l a r , c o n s i d e r D M U C . W h e n t h e c o n v e n t i o n a l D E A e f f i c ie n c y s c o r e i n (1)i s u s e d , t h e e x i s t e n c e o f C a f fe c t s m e a s u r e d e f f i c i e n c y f o r D M U s B , D , E , a n d E W h e nt h e m o d i f i e d e f f i c i e n c y s c o r e i n ( 2 ) is u s e d , t h e e x i s t e n c e o f C a f f e ct s m e a s u r e d e f f i c ie n c y


9/19

DETECTING INFLUEN TIALOBSERVATIONS N DATA ENVE LOPM ENTANALYSIS 35

f o r th e s e s a m e D M U s , b u t a l so a f f e ct s m e a s u r e d e f fi c ie n c y f o r D M U G - - in f o r m a t io n w h ic his lo s t when the conven t iona l sco re i s u sed due to the cen so r ing p rob lem . 5 A l though theresearc her m ay u l t imate ly w ish to u se X i com pu ted f ro m (1) to charac te r ize the techno logyand any ine f f ic iency , the in fo rma t ion r egard ing D M U G may s t il l be u se fu l s ince the shapeo f t h e f r o n t ie r a ro u n d D M U G i s a f f e c t e d b y th e p r e s e n c e o f D M U C . T h i s w o u ld b e im -p o r t a n t i f t h e r e s e a r c h e r w a n t e d t o m e a s u r e s h a d ow p r i c e s o r m a r g in a l r at e s o f t e c h n ic a lsubst i tu t ion .

T o e x am in e t h e q u e s t io n o f h o w m u c h t h e p r e s e n c e o f D M U C a f f e c ts th e e f f i c ie n c ym e a s u r e d f o r D MU s B , D , E , F , a n d G , ( 2 ) c o u ld b e r e c o m p u te d f o r e a c h D MU in t h es a m p le w h i l e d e l e ti n g D M U C f r o m th e r e f e r e n c e s e t. L e t th e i n d e x i = 1, . . . , 7 r e f e rt o t h e D M U s A . . . . . G , r e s pe c t iv e ly. T h e n ~ = {1, 3 , 7 } . F o r e a ch j E ~ , c o m p u te

min{),~-lyi _< y ( i j ) q , j , X ~ j x i > _ x ( i j ) q ~ , T q ~ = 1, q~j E 6IN+} (4)fo r i = 1 . . . . 7 , i # j where X ( i j ) = [Xm] u m ; e i , j , y ( i j ) = [ Y m ] q m ~ i , j , a n do ther va r iab les a r e de f ined as be fo re (no te X ( i ) a n d y ( i j ) have d imens ions K x (N - 2 )and M x (N - 2 ) , r espec t ive ly , and q* has d ime ns ions 1 x (N - 2 ) ) . Th is y ie ld s , fo re a c h j E g , a s e t { X * li = 1 . . . . N , i ~ j } ( n o te t h a t w h e n o b s e r v a ti o n j i s d e l e t e d ,X~ ma y be unde f ined due to an in feas ib le cons t r a in t se t in (4 ) , a s d i scussed above in con -nec t ion wi th (2 ) ) .

Nex t , de f ine n j , j E g , a s the num ber o f cases whe re X* ;e )~ and ~* Xg- a r e de f ined ,and def ine n* , as n j p lus the num ber o f cases w here 3 ,* i s de f ined and ),~ j i s undef ined .T h e n t h e a v e r a g e c h a n g e i n m e a s u r e d e f f i c ie n c y r e s u lt i n g f r o m d e le t i n g a D M U j f i g i s~ j ~ nj 1 ~iEe (X~j -- ~k*), w he re e = { i [ i = 1 . . . . N , i r j , ; k * an d X~j are de fine d} .

Per fo rming th i s exerc i se fo r the da ta in Tab le 1 y ie ld s :

1 2 0 .347 4 0 .69483 5 0 .413 6 2 .068 07 2 0 . 0 1 7 6 0 . 0 3 5 2

Thus , ob serva t ions 1 and 7 (DM Us A and G , r espec t ive ly ) each in f luence measu red e f f i -c i e n c y f o r tw o o th e r D M U s u s in g t h e m o d i f i e d m e a s u r e i n (2 ) . O b s e r v a t i o n 3 ( D M U C)in f lu e n c e s m e a s u r e d e f f ic i e n c y f o r f iv e o th e r D M U s .

The average change in measu red e f f ic iency , 6 / fo r DMUs A and C a re qu i te l a rge( r e la t ive to the average am oun t o f ine f f ic iency tha t i s typ ica l ly found in em p i r ica l app l ica -t i on s ) . H o w e v e r, D M U C a f fe c t s m e a s u r e d e f f i c i e n c y f o r 2 . 5 t im e s t h e n u m b e r o f D M U saf fec ted by DM U A. H enc e the quan t i ty nj* x 6 j p rov ides a mea su re o f the r e la t ive im-p o r t a n c e o f t h e e f f ec t s o f D M U s A a n d C . I f d a t a - c h ec k in g i s c o s t ly a n d r e s o u r c e s a r es c a r c e ( a c o m m o n s c e n a r io ) , t h e c a r e f u l re s e a r c h e r w o u ld w a n t t o f ir s t e x a m in e w h e th e rth e o b s e rv a t i on o n D M U C h a s b e e n m e a s u r e d c o r r e c t l y ; i f n o t , t h e r e s e a r c h e r w o u ld w a n tto e i the r co r r ec t the observa t ion i f poss ib le , o r de le te i t o therwise , s ince i ts p resenc e has


10/19

36 P.W. WILSON

s u c h a l a rg e e f f e c t o n e f f i c i e n c y m e a s u r e d f o r o t h e r D M U s i n th e s a m p l e . T h e r e s e a r c h e rm i g h t n e x t f o c u s o n t h e o b s e r v a t i o n f o r D M U A , i f s u f f i c i e n t r e s o u r c e s w e r e a v a i l a b l e .

And rews and P reg ibon (1978) , R ousseeuw and Basse t t (1991) , Wi l son (1993) , and o the rsh a v e d i s c u s se d t h e m a s k i n g p r o b l e m i n t h e c o n t e x t o f o u t l i e r d e t e c t i o n , w h e r e t h e e f f e c t so f a n o u t l ie r m a y b e m a s k e d b y t h e p r e s e n c e o f o n e o r m o r e o t h e r n e a r b y o u t li e rs i n t h espace con ta in ing the da ta. Th e m e thodo logy proposed above cou ld be ex tended to an i te ra tiveanalys is a long the l ines of D usansk y and W ilson (1994) w her e pairs , t r ip le ts , etc. a re d ele ted,b u t t h i s w o u l d g r e a t ly i n c r e a s e t h e c o m p u t a t io n a l b u r d e n a n d w o u l d m a k e t h e m e t h o d o l o g yi n t r a c ta b l e i n t h e c a s e o f l a r g e s a m p l es . T h e e m p i r i c a l e x a m p l e s i n t h e n e x t s e c t io ndem ons t ra te tha t h igh ly in f luen t ia l obse rva t ions can be found w i thou t cons ide r ing the mask-i n g p r o b l e m . F u r t h e r m o r e , i f t h e e f f e c ts o f a n o u t l i e r a r e m a s k e d b y a d j a c e n t o b s e r v a t io n s ,t h i s w o u l d s e e m t o i m p l y a l o w e r p r o b a b i l i t y t h a t t h e o b s e r v a t i o n s a r e c o n t a m i n a t e d b yda ta e r ro rs . S eaver and Tr ian t i s (1989) and W elsch (1982) sugges t tha t i t ma y be des i rab leto use m ore than one o u t l i e r de tec t ion scheme ; the s am e cavea t app l ie s he re , and th i s a r t ic lep r e s e n t s o n l y o n e s u c h s c h e m e .

T h e m e t h o d o l o g y d e s c r i b e d a b o v e al lo w s t h e r e s e a r c h e r t o p r i o r i t i z e o b s e rv a t i o n s in t h ee f f i c i e n t s u b s e t f o r f u r t h e r s c r u ti n y . T h e p r i o r i t iz a t i o n d e p e n d s u p o n s e v e ra l f a c to r s : h o wma ny o the r obse rva t ions ' e f f i c iency scores a re in f luenced by a g iven obse rva t ion , how m uchm e a s u r e d e f f i c i e n c y w o u l d c h a n g e f o r t h e s e o t h e r o b s e r v a t io n s i f t h e g i v e n o b s e r v a t io nw e r e d e l e te d , a n d h o w m u c h s u p p o r t e x i st s f o r t h e f r o n t i e r in t h e n e i g h b o r h o o d o f t h e g i v e no b s e r v a t i o n . R a t h e r t h a n r e l y i n g o n a s t a ti s ti c al t e st t o d e t e r m i n e w h e t h e r a g i v e n o b s e r v a -t i o n i s a n o u t li e r , th e r e s e a r c h e r m u s t u s e h i s k n o w l e d g e o f t h e u n d e r l y i n g p r o d u c t i o n p r o -c e s s to d e t e r m i n e w h e t h e r t h e i n f l u e n c e o f a p a r t i c u l a r o b s e r v a t i o n is l a rg e e n o u g h t o w a r -r a n t f u r t h e r s c r u t i n y ( w h i c h m a y b e c o s t l y ) , a n d t o d e c i d e h o w m a n y o f th e p r i o r i t i z e do b s e r v a t io n s s h o u l d b e s c r u t i n i z ed f o r e r r o r s . 6 N o t e t h a t t h e a p p r o a c h h e r e i n v o lv e s th es a m e c a l c u l u s o f c o m p a r i n g m a r g i n a l b e n e f i t s a n d m a r g i n a l c o s t s t h a t e c o n o m i s t s u s ew h e n e v e r r e s o u r c e s a r e s c a r c e . I f t h e c o s t o f d a t a - c h e c k i n g i s c o n s t a n t a c r o s s o b s e r v a -t i o n s , o n e s h o u l d f i r s t c h e c k o b s e r v a t io n s w h i c h , i f th e y c o n t a i n a d a t a e r r o r , m i g h t c a u s et h e m o s t d a m a g e t o t h e e n t i r e a n al y si s . T h e n e x t s e c t i o n p re s e n t s s o m e e m p i r i c a l e x a m p l e sus ing da ta f rom prev ious ly pub l i shed s tud ies to i l lus t ra te (1 ) how the ideas f rom th i s s ec -t i o n c a n b e a p p l i e d i n l a r g e r s a m p l e s t h a n t h e e x a m p l e c o n s i d e r e d a b o v e , a n d ( 2 ) t h ep r e v a l e n c e o f t h e o u t l i e r p r o b l e m i n r e a l d a t a s e t s u s e d i n t y p i c a l D E A a p p l i c a t i o n s .

4. Some Empirical ExamplesF o r e a c h e x a m p l e b e lo w , e f f i c i e n c y i s m e a s u r e d i n t e rm s o f t h e m o d i f i e d I W E s c o r e c o m -p u t e d v i a ( 2 ). A l l c o m p u t a t i o n s w e r e d o n e o n a S U N S P A R C s t at io n 1 0 M o d e l 3 0 d e s k to pw o r k s t a t i o n u s i n g o p t i m i z e d F o r t r a n c o d e . 7

Example 4.1.C h a r n e s e t a l. ( 1 9 8 1 ) r e p o r t d a ta o n r e su l t s f r o m P r o g r a m F o l l o w T h r o u g h ; t h e i r d a t a c o n -ta in 70 observat ions on f ive inputs and thre e outputs . M easurin g eff ic iency us ing the m odif ied


11/19

D E T E C T I N G I N F L U E N T I A L O B SE R V AT IO N S I N D AT A E N V E L O P M E N T A N A L Y S IS 37

I W E s c o r e i n ( 2 ) i nd i c a te s 2 7 o s t e ns i b l y e f f i c i e nt o bs e r v a t io ns w hi c h a r e l i s te d i n Ta ble 2 .F o r e a c h o s t e n s ib l y e f f i c i en t o b s e r v a ti o n , T a b le 2 s h o w s th e m o d i f i e d I W E s c o r e X * , t h en u m b e r o f D M U s f o r w h i c h X * is a lt er ed w h e n t h e o st e n s i b ly e f f i c i en t o b s e r va t io n i sde l e t e d ( n j* ), the a v e ra g e c ha ng e i n me a s ur e d e f f ic i e nc y o c c ur r i ng w he n t he o s t e ns ib l y e f f i -c i e nt o bs e r v a t i o n i s o m i t te d ( t j ) , a nd the we i g h t e d m e a s ur e n j* x 6 j.

A m on g the os tens ibly e f f i c ient observations l i s ted in Table 2 , obervations 4 4 and 54 s tando ut a f t e r i ns pe c t i o n o f t he v a l ue s X* ; o bs e r v a t i o n 4 4 ha s t he l a r g e s t v a l ue o f X* , whi l eX~9 i s un def in ed. M oreover , o bserv at ion 44 h as the high es t va lue for nj* x 6 j, indica t inga g r e a t de a l o f in f l ue n c e o n o t h e r o bs e r v a t io ns i n the s a m pl e . O bs e r v a t i o n 5 2 h a s t he ne x th i g he s t v a l ue f o r n j* x 6 j, f o l l o w e d by o bs e r v a t i o ns 6 9 , 6 2 , 5 9 , 5 6 , 1 5 , 5 8 , e t c . Ob s e r v a -t i o ns 1 7 , 4 7 , a nd 4 9 e a c h i n f l ue nc e 1 1 o r mo r e o t he r o bs e r v a t i o ns , but r a nk r a t he r l o wi n te r ms o f t he ir a ve r ag e l e v e l o f in f l ue nc e a s me a s u r e d by n j* x 6 j.

Table 2 . C h a r n e s e t a l . ( 1 9 8 1 ) d a t a .jEg X~ nj* 4 nj* ~

5 1 . 0 5 0 4 3 0 . 0 4 0 5 0 . 1 2 1 51 1 1 . 0 5 7 7 0 0 . 0 0 0 0 0 . 0 0 0 01 2 1 . 0 4 8 7 4 0 . 0 0 9 7 0 . 0 3 8 81 5 1 . 2 8 6 8 5 0 . 0 7 4 2 0 . 3 7 0 91 7 1 . 2 3 6 0 1 l 0 . 0 1 8 6 0 . 2 0 5 11 8 1 . 0 3 9 3 0 0 . 0 0 0 0 0 . 0 0 0 02 0 1 . 1 42 1 4 0 . 0 2 0 3 0 .0 8 1 22 1 1 . 1 1 2 2 6 0 . 0 0 3 7 0 . 0 2 2 22 2 1 . 0 1 58 7 0 . 0 0 0 8 0 .0 0 5 62 4 1 . 1 05 5 5 0 . 0 2 6 0 0 .1 2 9 82 7 1 . 0 6 3 0 7 0 . 0 0 6 4 0 .0 4 5 13 2 1 . 0 6 1 5 3 0 . 0 0 5 5 0 . 0 1 6 53 5 1 . 0 2 9 9 1 0 . 0 4 6 9 0 .0 4 6 93 8 1 . 1 45 5 3 0 . 0 1 2 0 0 .0 3 6 04 4 2 . 0 8 1 6 3 2 0 . 0 4 1 2 1 . 3 1 9 04 5 1 . 0 1 2 0 4 0 . 0 5 1 8 0 . 2 0 7 14 7 1 . 1 0 8 9 1 8 0 . 0 0 8 9 0 . 1 5 9 64 8 1 . 3 0 1 8 4 0 . 0 1 3 9 0 . 0 5 5 64 9 1 . 0 6 9 0 2 2 0 . 0 0 6 2 0 . 1 3 6 25 2 1 . 1 8 63 3 7 0 . 0 2 0 5 0 . 7 5 6 85 4 1 . 2 1 8 6 2 0 . 0 0 5 4 0 . 0 1 0 85 6 1 . 0 9 2 1 7 0 . 0 5 5 5 0 . 3 8 8 35 8 1 . 3 5 1 4 1 1 0 . 0 1 9 8 0 . 2 1 8 35 9 - - 4 0 . 1 3 7 0 0 . 5 4 7 96 2 1 . 5 5 4 1 3 2 0 . 0 2 0 3 0 . 6 4 8 76 8 1 . 1 8 9 7 3 0 . 0 1 0 6 0 .0 3 1 96 9 1 . 6 4 4 8 1 8 0 . 0 3 8 6 0 . 6 9 4 1


12/19

38 P.W. WILSO N

W ilson (1993) repor t s r e su l ts f rom an ou t l ie r ana lys i s on the same d a ta used in th i s ex-amp le ; obse rva t ions 35 , 44 , 54 , and 59 w ere found to be ou t l ie r s , a s we l l a s 5 o the r obse r -va t ions tha t a re no t in the e f f i c i en t subse t . O bse rva t ions 44 an d 59 w ere foun d to be them ost obv ious ou t l i e r s , cons i s t en t wi th the re su l ts above . The re su l ts he re ind ica te o the rimp or tan t in f luen t i a l obse rva t ions such a s obse rva t ion 52 , w hich in f luences meas ured e f -f i c i ency fo r 53 .6 pe rcen t o f the remain ing obse rva t ions in the sample .

Example 4.2Ff i re e t a l . (1986) m easure e f f i c iency am ong 100 s t eam e lec t r i c gene ra t ing p lan ts in 1975u s i n g 5 i n p u ts a n d 5 o u t p u t s. C o m p u t i n g th e m o d i f i e d I W E sc o r e i n (2 ) f o r ea c h D M Uin the sam ple reveals 83 ostens ibly eff ic ient obse rvat ions as l i s ted in Table 3 . For 18 ofthese obse rva t ions , ~* i s undef ined due to in feas ib le cons t ra in t set s. O f the rem ain ing 65obse rva t ions , 12 have va lues ~* > 2 , and 4 have va lues X* > 25 . F or obse rva t ion 68 ,~ 8 = 1586 .4 sugges t ing tha t t h i s p l an t s ' inpu t s would have to be sca led up by more than150 ,000 p e rcen t to reach the f ron t i e r suppor t ed by the rem ain ing obse rva t ions ; t h is seemsqui t e un l ike ly . Moreover , t h i s obse rva t ion in f luences e f f i c iency m easu rem ent fo r 13 o the rp lan t s represen ted in the sample .Fo r obse rva t ion 94 , X~'4 i s unde f ined . Th i s obse rva t ion in f luences e f f i c i ency meas ure -m ent fo r 11 o the r obse rva t ions in the sam ple , and to an ex t raord ina ry degree a s ind ica tedby ~j and nj* x 6j. Observ at ions 1 , 20, 4 9, 66, and 69 a lso exer t re la t ive ly large inf luenc eon m easu red eff ic iency for other observat ions in the sample . W hi le other observat ions l is tedin Tab le 3 m ay con ta in da ta e r ro rs tha t a f fec t e f f ic i ency fo r o the r obse rva t ions , t he obse r -va t ions l is t ed above seem to be the mo st l i ke ly to have da ta e r ro rs , and po ten t i a lly havethe l a rges t imp ac t on e f f i c i ency m easu rem ent wi th in the sample .

Example 4.3A ly e t a l . (1990) exam ine e ff i c i ency us ing the IW E score in (1) fo r a samp le o f 322 in -depen dent banks in 1986 . Th e i r da ta con ta in obse rva t ions fo r 3 inpu t s and 5 ou tpu t s fo reach of the 322 banks in the i r sample . C om put ing the mod i f i ed IW E score in (2 ) fo r eachD M U in the sam ple revea ls 90 os t ens ib ly e f fi c i en t obse rva t ions a s l is t ed in Tab le 4 ; o fthese , X* is un def in ed for two observ at ions, and X* > 2 for e ight observat ions. In addi-t ion , seve ra l obse rva t ions in f luence m easured e f f i c iency fo r a la rge num ber o f DM U s inthe sam ple ; e .g . , obse rva t ion 298 a f fec ts m easure d e f f i c iency fo r mo re than on e th i rd o fthe D M U s in the sample ; 18 o f the obse rva t ions l is t ed in Tab le 4 a f fec t me asured e f f i -c iency_for a t l eas t 10 pe rcen t o f the sam ple . In t e rms of ove ra ll i n f luence a s mea sured bynj* x 6j, obse rvat ions 26, 2 59, and 298 app ear to have the largest inf luen ce and thus aregood candida te s fo r fu r the r sc ru t iny .As no ted prev ious ly , t he ques t ion of wh e the r n* x 6 j i s l a rge ma y depend upon theunde r ly ing produc t ion process . Aly e t a l . (1990) repor t m ean pr i ces , P_=_ [23 .7 0 .43 0 .07] .M ean inpu t s a re g iven by X = ]32 889000 35680000] , and hen ce XP' = $2 ,880 ,628 .Thu s , on ave rage , a change in e f f i c iency of 0 .01 i s wo r th approx ima te ly $28 ,806 , wh ichgives a guid el ine for interpre t ing the m agn i tudes of nj* x 3j in Table 4 .


13/19

DETECTING I NFLU ENTI AL OBSERVATIONS IN DATA ENVELOPMENT ANALYSIS 39

T a b l e 3 . F~re et al. (1986) data.n 4 n Y

1 -- 6 3.2687 19.61222 1.2858 9 0.0294 0.26483 1.0070 1 0.0000 0.00004 1.1253 5 0.0025 0.01265 1.2126 0 0.0000 0.00006 1.0000 0 0.0000 0.00007 1.0165 0 0.0000 0.00008 -- 0 0.0000 0.00009 -- 7 0.0910 0.6367

10 25.1167 13 0.0604 0.784911 1.0741 4 0.0036 0.014312 1.0009 0 0.0000 0.000014 1.4722 0 0.0000 0.000015 - - 1 0.0000 0.000016 1.1768 2 0.0027 0.005417 1.5523 8 0.0069 0.054818 1.2107 3 0.0113 0.033819 1.0489 4 0.0056 0.022220 -- 9 3.2077 28.869121 3.0195 5 0.0403 0.201522 1.0263 2 0.0004 0.000824 1.0228 2 0.0000 0.000026 1.0003 0 0.0000 0.000027 1.0098 0 0.0000 0.000028 1.9247 8 0.0294 0.234929 2.9746 11 0.1867 2.054030 -- 9 0.1078 0.970131 216.6826 0 0.0000 0.000033 4.0362 4 0.0126 0.050634 1.1227 0 0.0000 0.000035 1.4428 3 0.0180 0.053936 2.3032 5 1.0111 5.055637 -- 21 0,0588 1.233838 1.0669 0 0,0000 0.000039 4.5989 2 1,0800 2.159940 1.8467 4 0.0145 0.057941 1.0163 0 0,0000 0.000042 -- 9 0,0382 0.344143 -- 1 0.0165 0.016545 1.1201 0 0.0000 0.000047 1.2973 5 0.0111 0.055749 2.6866 8 7.1677 57.341751 1.5500 6 0.0770 0,462152 1.2459 2 0.1983 0,396653 1.8893 0 0.0000 0,000055 1.0136 0 0.0000 0.000057 1.0384 1 0.0000 0.000058 1.0765 0 0.0000 0.000059 1.5729 0 0.0000 0.000060 1.4589 10 0.0330 0,3303


14/19

4 0 P.W. WILSON

T able 3. Continued.jE g X~ nj* 4 nJ* x 4

63 1.1121 0 0.0000 0.000064 1.0004 1 0.0001 0.000166 -- 7 1.8124 12.686967 1.3901 0 0.0000 0.000068 1586.4194 13 4.6366 60.276169 1.0001 2 9.7569 19.513770 1.1262 4 0.0160 0.064071 1.0129 1 0.0058 0.005872 1.1154 3 0.0245 0.073473 1.3392 3 0.0237 0.071275 5.1170 4 0.4378 1.751376 1.0122 10 0.0012 0.012177 -- 1 0.0000 0.000078 1.1102 2 0.0077 0.015379 1.5068 12 0.0479 0.574680 1.0086 0 0.0000 0.000081 -- 3 0.0510 0.153182 -- 5 0.1505 0.752483 1.0466 1 0.1357 0.135784 -- 0 0.0000 0.000086 -- 2 0.0051 0.010187 -- 4 0.9660 3.864088 3.8671 15 0.1239 1.858489 1.1034 1 0.0176 0.017690 1.2555 2 0.0016 0.003191 1.0079 1 0.2205 0.220593 25.3920 20 0.0677 1.354194 - - 11 116.2929 1279.222395 3.5466 5 0.0572 0.286196 1.0344 0 0.0000 0.000097 1.0383 0 0.0000 0.000098 - - 27 0.0311 0.8399

100 1.0258 0 0.0000 0.0000

Table 4 . A l y e t a l . ( 1 9 9 0 ) d a t a .

2 1.7363 40 0.0281 1.12473 1.3160 72 0.0140 1,00524 1.0144 0 0.0000 0,00005 1.5773 36 0.0106 0,38046 2.1902 93 0.0227 2,11548 1.5017 38 0.0171 0.647913 1.1138 12 0.0079 0.0951

19 1.5556 60 0.0212 1.273920 1.2830 53 0.0113 0.596821 1.0638 0 0.0000 0.000023 1.0454 3 0.0051 0.0154


15/19

DETECTING INFLUENTIAL OBSERVATIONS IN DATA ENVELOPMENT ANALYSIS 4 ]

Tab le 4 . Continued.

25 1.2256 1 0.1417 0.141726 4.3915 83 0.0416 3.452928 1.0217 1 0.0538 0.053829 1.0827 10 0.0043 0.043530 1.6906 8 0.0724 0.579631 1.1668 6 0.0166 0.099735 1.0598 0 0.0000 0.000036 1.2734 27 0.0169 0.455537 1.0171 2 0.0011 0.002338 1.2880 21 0.0129 0.270642 1.1048 25 0.0049 0.123553 1.0311 1 0.0065 0.006554 1.0615 9 0.0117 0.105558 1.0216 0 0.0000 0.000061 1.4589 5 0.0126 0.062963 1.2025 3 0.0123 0.036868 1.0339 0 0.0000 0.000075 1.0547 2 0.0022 0.004583 1.0696 1 0.1146 0.114686 1.0352 5 0.0051 0.025594 1.3163 19 0.0738 1.402299 1.0245 1 0.0013 0.0013100 1.1401 5 0.0098 0,0491

104 1.6791 4 0.0289 0.1158106 1.3516 19 0.0119 0,2264107 3.0921 24 0.0575 1,3811110 1.1414 5 0.0025 0.0127112 1.0572 2 0.0030 0.0060119 1.0470 1 0.0071 0.0071121 3.5218 18 0.1159 2.0865122 1.3072 11 0.0795 0.8744132 1.1386 4 0.0041 0.0166135 1.2068 88 0.0110 0.9655136 1.0543 1 0.0107 0.0107139 1.0793 10 0.0040 0.0402147 1.0851 19 0.0006 0.0105151 1.5369 92 0.0230 2.1177153 1.3229 19 0.0267 0.5076168 1.0638 4 0.0018 0.0071173 1.1333 14 0.0346 0.4843180 1.1306 4 0.0050 0.0200184 1.1516 8 0.0061 0.0492197 1.1538 6 0.0048 0.0289205 1.4875 3 0.0232 0.0696213 1.2189 25 0.0121 0.3024228 1.1566 8 0.0065 0.0519235 1.2619 20 0.0118 0.2359238 1.0144 1 0.0004 0.0004239 1.1758 38 0.0075 0.2856240 1.0822 32 0.0035 0.1118


16/19

42 P.W. WILSON

Tab le 4 . Continued.

245 1.3169 22 0.0120 0.2647246 1.1742 28 0.0088 0.2461251 1.0717 3 0.0004 0.0011254 1.1294 12 0.0049 0.0587259 4.5327 81 0.0699 5.6625261 1.1942 9 0.0006 0.0050267 1.3726 22 0.0059 0.1288268 1.9270 22 0.0497 1.0931285 1.1049 1 0.0353 0.0353292 1.4966 50 0.0229 1.1430295 1.4098 61 0.0253 1.5458296 1.2013 31 0.0099 0.3084298 1.5483 111 0.0305 3.3900299 2.7110 47 0.0573 2.6917302 1.2529 3 0.0627 0.1881304 1.1736 3 0.0129 0.0386305 1.7552 20 0.0511 1.0222306 1.2354 9 0.0170 0.1530308 1.2392 7 0.0134 0.0940309 1.6534 42 0.0236 0.9913311 1.6011 2 0.1249 0.2498312 1.4201 34 0.0151 0.5141313 2.2224 16 0.0646 1.0332316 1.4847 5 0.0177 0.0883317 1.2780 1 0.0041 0.0041319 1.1721 2 0.0271 0.0542320 2.4940 22 0.1358 2.9883321 -- 11 0.0326 0.3582322 -- 19 0.1532 2.9105

5 . C o n c l u s i o n sAlthough ne ither of the three studies cited as examples in the previous section make anymen tio n of attempts to meas ure the influen ce of particular observations or to test for thepresence of oufliers in the data, the examples are not intended to suggest that the authorsof the studies were careless or o therwis e negli gent in their analyses. No good outli er diag-nostics were availab le at the times of their studies. Rather, the examples me rel y illustratethe need for such diagnostics, and the potential for the present diagnostic to find influen-tial observati ons in data used to estimate technical efficiency scores. T he examples suggestthat substantial numbers of influential observations exist in typical settings where D EAhas been used. Furthermore, the diagnostics that have been proposed in this paper givean in dication of the consequ ences of a data error in an ostensibly efficient observation,and a s cheme for prioritizi ng observation s for further scrutiny is proposed. This prioritiza-tion is particularly import ant in situations where data-checking is costly and resources arelimited.


17/19

D E T E C T I N G I N F L U E N T I A L O B SE RV A TIO N S I N D AT A E N V E L O P M E N T A N A L Y S IS 4 3

A s n o t e d i n t h e i n t r o d u c t io n , i d e n t i fi c a t io n o f in f l u e n ti a l o b s e r v a t i o n s o r o u t l ie r s i s o n l ya f i r s t s t e p . I t r e m a i n s f o r t h e r e s e a r c h e r t o s c r u t i n i z e s u s p i c i o u s o b s e r v a t i o n s t o e n s u r et h a t th e y d o n o t c o n t a i n s o m e t y p e o f m e a s u r e m e n t e r r o r . I f a d a t a e r r o r i s f o u n d in a p a r -t i c u l a r o b s e r v a t i o n , i t sh o u l d b e c o r r e c t e d i f p o s s i b l e ; o t h e r w i s e , t h e r e s e a r c h e r m a y w i s ht o d e l e te t h e o b s e r v a t i o n w i t h t h e e r r o r . I f n o d a t a e r r o r i s fo u n d , t h e n t h e o b s e r v a t i o ni s m e r e l y a t y p i c a l o f th e r e m a i n i n g d a t a , a n d m a y c o n t a i n u s e fu l i n f o r m a t i o n .

Notes1 . A w i d e l i te r a t u re e x i s t s d e s c r i b i n g t h e d e t e c t i o n o f o u f l ie r s in t h e c o n t e x t o f t h e C LR m o d e l . S e e C h a t t e r j e e

a n d H a d i ( 1 9 8 6 ) a n d G r a y ( 1 98 8 ) f o r s u m m a r i e s o f th i s l it e r a tu r e .2 . C h a r n e s e t a l . ( 1 9 8 6 ) a p p e a r t o b e t h e f i r s t t o c o n s i d e r m o r e t h a n o n e c l a s s o f e f f i c i e n t o b s e r v a t i o n s . Th e

m o d i f i c a t i o n u s e d b y A n d e r s e n a n d P e t e r s e n a n d b y Lo v e l l e t a l. i s r e l a t e d t o t h e c l a s s i f i c a t io n o f o s t e n s i b l ye f f i c i e n t o b s e r v a t i o n s d i s c u s s e d b y C h a r n e s e t a l . ( 1 9 9 1 ), Th r a l l ( 1 9 9 3 ) , a n d e l s e w h e r e .

3 . T h e d i s t i n c t i o n b e t w e e n censor ing a n d t runca t ion h a s s o m e t i m e s b e e n c o n f u s e d i n t h e l i t e r a t u r e . I n e i t h e rc a s e , a v a r i a b l e z i s o b s e r v e d , w h i l e a v a r i a b l e z * i s t h e u n d e r l y i n g v a r i a b l e o f i n t e r e s t. I f z i s c e n s o r e d f r o mabove a t a va lue c , then z i = z* fo r a l l i such tha t z* -< c , a nd z i = c fo r a l l i such tha t z* > c . I f z i s t run -c a t e d f r o m a b o v e a t a v a l u e c , t h e n z i = z* f o r a l l i s u c h t h a t z * < c , a n d z * d o e s n o t e x is t ( a n d h e n c e zii s u n o b s e r v e d ) o t h e r w i s e .

4 . I n m a n y e m p i r i c a l a p p l i c a ti o n s , a v e r a ge i n e f f ic i e n c y m e a s u r e d b y ( 1) h a s o f t e n b e e n f o u n d t o r a n g e f r o m a b o u t0 . 7 5 t o 1 . 0 0; e . g . , s e e A l y e t al . ( 1 9 9 0 ) , B u r g e s s a n d W i l s o n ( 1 9 9 3) , a n d D u s a n s k y a n d W i l s o n ( 1 9 9 4 , 1 9 95 ).

5 . N o t e t h a t in t h e c a s e o f th e m o d i f i e d s c o r e c o m p u t e d f r o m ( 2 ), t h e p r e s e n c e o f D M U C a f f e ct s m e a s u r e de f f i c i e n c y f o r D M U G e v e n t h o u g h C d o e s n o t d o m i n a t e G i n t h e s e n s e o f u s in g l e s s o f b o t h i n p u t s t o p r o d u c et h e s a m e o r g r e a t e r o u t p u t .

6 . Th e s i g n i f ic a n c e o f e f f i c ie n c y s c o r e c h a n g e s r e s u l t in g f r o m d e l e t i o n o f a n o s t e n s i b l y e f f i c ie n t o b s e r v a t i o n i sm e a n i n g f u l o n l y i n t h e c o n t e x t o f t h e u n d e r l y i n g p r o d u c t i o n p r o c e s s ; i . e . , a g i v e n c h a n g e i n e f f i c i e n c y m a ybe im por tan t fo r one indus t ry and un imp or tan t fo r ano ther indust ry , depend ing upon the na tu re o f the t echno logya n d i n p u t / o u t p u t m a p p i n g u s e d i n e a c h i n d u s t r y . I n a d d i t i o n , u s e o f a s t a t is t ic a l te s t i n t h e m e t h o d o l o g y o fth i s sec t ion wo u ld requ i re d i s t r ibu t iona l as sumpt ions regard ing the e f f i c iency sco res ; o ne o f the a t t rac t ive fea tu reso f D E A a n d F D H m e t h o d s i s t h a t th e y a v o id d i s tr i b u t io n a l a s s u m p t i o n s .

7 . P e r f o r m a n c e o f t h i s m a c h i n e i s r a te d b y th e m a n u f a c t u r e r a t 8 6 . 1 m i l l i o n i n s t r u c t io n s p e r s e c o n d , a n d 1 0 . 6mi l l ion f loa t ing -po in t opera t ions per second . In the l a rges t sam ple inves t iga ted in th i s sec t ion (Aly e t a l . , 1990,w i t h 3 2 2 o b s e r v a t i o n s ) , a ll c o m p u t a t i o n s w e r e p e r f o r m e d i n l e s s th a n 6 9 m i n u t e s o f e l a p s e d t i m e .

ReferencesA l y , H .Y . , R . G . G r a b o w s k i , C . P a s u r k a , a n d N . R a n g a n . ( 1 9 9 0 ). " Te c h n i c a l , S c a l e , a n d A l l o c a t i v e E f f i c i e n c i e s

in U.S . Bank ing : an Empi r ica l Inves t iga t ion ." R e v i e w o f E c o n o m i c s a n d S t a ti s ti c s 7 2 , 2 1 1 - 2 1 8 .A n d e r s e n , E , a n d N . C . P e t e r s e n . ( 1 98 9) . " A P r o c e d u r e f o r R a n k i ng E f f i c i e n t U n i t s in D a t a En v e l o p m e n t A n a l -

y s is . " U n p u b l i s h e d w o r k i n g p a p er , D e p a r t m e n t o f M a n a g e m e n t , O d e n s e U n i v er s it y , O d e n s e , D e n m a r k .A n d r e w s , D . E , a n d D . P r e g i b o n . ( 1 9 7 8 ) . " F i n d i n g t h e O u t l i e r s t h a t Ma t t e r . " Journa l o f the Roya l S ta t i s t i ca l

Soc ie ty S e r i e s B , 4 0 , 8 5 - 9 3 .Burg ess , J .E J r . , and P .W. W i lson . (1993) . "Tec hn ica l Eff ic ienc y in VA Hos p i ta l s , " in The M e a s u r e m e n t o f P r o -

ductive Efficiency: Techniques and App licat ion s, ( e d s . H a r o l d O . F r i e d , C . A . K n o x Lo v e l t , a n d S h e l t o n S .S c h m i d t ) , O x f o r d : O x f o r d U n i v e r s i t y P r e s s , 3 3 5 - 3 5 1 .

C h a r n e s , A . , W . W . C o o p e r , a n d E . R h o d e s . ( 1 9 8 1 ). " Ev a l u a t i n g P r o g r a m a n d Ma n a g e r i a l E f f i c i e n c y : A n A p p l i -c a t i on o f D a t a E n v e l o p m e n t A n a l y si s t o P r o g r a m F o l l ow T h r o u g h ." M a n a g e m e n t S c i e n c e 2 7 , 6 6 8 - 6 9 7 .

C h a r n e s , A . , W . W . C o o p e r , a n d R . M . Th r a l l . ( 1 9 8 6 ) . " I d e n t i f y i n g a n d C l a s s i f y i n g S c a l e a n d Te c h n i c a l E f f i c i e n -c i e s i n D a t a E n v e l o p m e n t A n a l y s i s . " Ope rations Research Le t ters 5 , 105-116 .


18/19

44 P.W. WILSON

Ch arnes, A ., W .W. Cooper , and R.M . Thrall . (1991) . "A Structure for Classifying and Ch aracter izing Eff iciencyand Ineff ic iency in D ata Enve lop ment Analys is? ' Journal of Productivity Analysis 2, 197-237.

Ch arnes , A., S. Haag, P. Jaska, and J. Semple. (1992). "Se nsit ivi ty of Eff iciency Classif icat ions in the AdditiveMode l o f D a t a Enve l opm e n t A na l ys is ? ' International Journal o f Systems Science 2 3 , 7 8 9 - 7 9 8 .

Ch arnes , A., and L. Nerali6. (1990). Sensit ivi ty Ana lysis of the Addit ive M odel in Data Env elopm ent Analysis."European Journal of Operational Research 48 , 332- 341 .

Cha tter jee, S. and Hadi, A.S. (1986). "Inf lu ential Obs ervations, High Leverage Points, an d Outl iers in Lin earRegression." Statistical Science, 1, 379-392.

Cook, R.D., and S. Weisberg. (1982) . Residuals and Influence in Regression, New York: Chap man an d Hall .Dep rins, D., L. Simar , and H. Tulkens. (1984). "M easu ring Labo r Ineff iciency in Post Off ices, ' in The Perfor-

mance of Public Enterprises: Concepts and Measurements , (ed. M. M archa nd , P. Pestieau, and H. Tulkens),A ms t e r da m: N or t h - H ol la nd , 243 - 267 .

Dusansky, R . and P.W. W ilson. (1994) . "M easu ring Efficiency in the Care of the Developmentally Disabled ."Review o f Economics and Statis tics 76 , 340- 345 .

Dusansky, R . an d P.W. W ilson, (1995) . "O n the Relative Eff iciency of Alternative M ode s of Produ cing Pub licSector Output: T he Case of the Developmentally Disab led? ' European Journal of O perational Research 80 ,6 0 8 - 6 2 8 .

F~ire, R., S. Grosskopf, an d C.A .K. Lovell. (1985). The Measurement of Efficiency of Production, Boston:Kluwer-Nijhoff.

F~ire, R., S. Grosskopf, an d C. Pasurka. (1986). "E ffects on Relative Efficiency in Ele ctric Power Gen erat ionDue to Envi ronmenta l Cont rols ." Resources and Energy 8, 167-184.

Gray, J.B. (1988) . '~ Class i f ica t ion of Inf luence Measures? ' Journal o f Statistical C ompu tation and Sim ulation,30, 159-171.

Greene , W.H. (1980). "M axim um Likel ihood Est imat ion of Econom etr ic Front ie r Func t ions? ' Journal of Econo-metrics 13 , 27 - 56 .

Grosskopf , S . , and V. Valdman is . (1987). "M easur ing Hospital Per formance ." Journal o f Health Economics6, 89-107.G uns t , R .E , a nd R .L . Ma son . ( 1980). Regression Analysis and its Application. New York: M arce l Dekker .Korostelev, A.P. , L . Simar , and A.B. Tsybakov. (1994). "E ff icien t Estim ation of M ono tone Bou ndaries." Annals

of Statistics, forthcoming.Love ll , C .A.K. (1993). "Prod uc t ion Front ie r s and Produc t ive Ef f ic iency, ' in The Measurement of Productive

Efficiency: T echniques and Ap plications, (eds. H. F r ied, C .A.K. Lovell, and S. Schmidt) , Oxford: OxfordUniversity.

Love ll , C .A.K. , L.C. Wai te rs , and L.L . Wood. (1993). "S t ra t i f ied Models of Educa t ion Produc t ion Using DEAand Regression Analysis?' in Data E nvelopm ent Analysis: T heory, Method s, and Applications, (eds. A. Charnes,W.W. Cooper , A .Y. Lew in, and L.M . Seiford), New Y ork: Qu orom Books.

M artin, J .P. , and J.M. Page. (1983). "T he Imp act of Subsidies o n X-eff iciency in LD C Ind ustry: The ory andan Em pir ica l Tes t. " Review of Economics and Statis tics , 65 , 608- 617 .McC arty, T .A., and S. Yaisawarng. (1993). "Technical Efficiency in New Jersey School Districts?' in T he Measure-

ment of Productive Efficiency: Techniqu es and Applications, (eds. H. Fr ied, C.A.K. Lovell , and S. Schmidt) ,Oxford: Oxford University.

Rousseeuw, P.J., an d G.W. Bassett . (1991). "Ro bustn ess of the p-Sub set Alg ori thm with High Breakdown Poin t? 'in Directions in Robust Statistics and Diagnostics: Part II, (eds. W. Stahel and S. Weisberg), New York:Springer-Verlag.

Rousseeuw, P.J., an d B.C. van Zom eren. (1990). "U nm ask ing Mu lt ivar iate Outl iers an d Leverage Points," Jour-nal of the American Statistical Association, 85 , 633- 639 .

Seaver, B.L ., an d K.P. Triantis. (1989). "T he Implications of Us ing M essy Data to Estimate Production-Frontier-Based Technical Eff iciency M eas ure s, ' Journal o f Business and Economic Statis tics 7, 49-59.Sexton, T.R., R .H. Silkm an, and A.J. Hogan. (1986). "D ata Envelo pm ent Analysis: C ri t ique and Extensions,"in Measuring E fficiency: An Assessment of Data Envelopment A nalysis , R.H . Silkm an (ed.) , San Francisco:Jossey-Bass Co.

Shephard, Ronald W. (1970) . Theory of Cost and Production Functions, Princeton: Pr inceton Universi ty.Thra l l , R.M . (1993). "Duali ty , Class i f ica t ion, and S lacks in DE A " Unp ubl i shed working paper, Jones Gradua te

School of Administrat ion, Rice Universi ty.


19/19

D E T E C T I N G I N F L U E N T I A L O BS E RVA T IO N S I N D A T A E N V E L O P ME N T A N A L Y SI S 45

Timmer, C.E (1971). "U sin g a Probab il ist ic Fro ntier Produc tion Fun ction to Meas ure Technical Eff iciency."Journ al o f Pol i t ica l Economy 79 , 776- 794 .

Welsch, R.E. (1982). "Inf lu enc e Fun ctions and Reg ression Diagnostics." in Mo dern Data Analys is , (ed. by R.L.Launer and A.E Siegel) , New York: Academic Press.W ilson, G.W., and J.M . Jadlow. (1982) . "C om peti t io n, Prof i t Incentives, and Technical Eff iciency in the Provi-

sion of Nu clear Me dicine Services." The Bel l Jo u r n a l o f Eco n o mi cs 1 3 , 4 7 2 - 4 8 2 .Wilson, P .W. (1993). "Detec t ing Out l ie r s in D ete rminis t ic Non parametr ic Front ie r Mod els wi th M ul t ip le Out -

puts." Journ al o f Bus iness and Econo mic S ta t i s t ics 11, 319-323.

detecting influential observations in dea wilson

Documents