usaid response to 1980 impact study

8/9/2019 USAID Response to 1980 Impact Study

1/9


2/9

~~mThis paper reviews the findings and critiques the Lnpact studyof theJh. Kennedy -Yedicdl Center (Project No. 669-0054) dated-1980. ' I h e C e n t e r w a s b u i l t i n ~ , ~ ,ndwasintendedt o m a t b e c p r e o f a h e a l * c a r a U ~ p y s t e n f m a l l o fU i a . * pct study kss b e y a Libexhn managetent amsdt ing

'Qd research finnf amsmf mc.. whichwas ccntrarted byum-tD amduct a -te specia l evaluatim series.

IheJPRMedicalCenterimpaCtstrdy~tbefirstdmeintbeseries. Zh e s e a n d s h r l y i n t h e series. theimpact study on the~ ~ f a C o u n t y ~ D e v e l ~ t P m j e c t h a s a l sby PBC/E/S recently. Ccat for the series was SU. 00. of a c h $3.000W a s S p e n t t o m d t h e J E K i m p ~ ~ .

With t h i s s as wiL5 the Upper Lofa Caunty impact shrly, thecontext far Mettzing ara d t i q u i n g is as follcws:

- w *rll the w r t defines the pmject's scope andobjectives;- ther the r e p r t finds that these objectives wse

achieved and substantiates these E W g s ;- hether the report has an adequate resear& design(including identificatian of the m t r s bjectives.and identificaticm and -iderati= of majordeveloprent issues and major indiotors of eEect ie -ness and imp& ) ;--whether the z e p t used a viable metkdologicah appmachin the -text of its pu~ose objective) nd.- t h e r the r ep * achieved its objective.

-


3/9

F i n d i n ~ s f the 2ev5.e~Like i t s successor , the impact study on 3ppes LofaCounty Development, this study is mediocre, However,

it succeeds in many ways t ha t - t he rural development b p a c tstudy does not, a&&= i t s e l f more c r e d i t a b l e i n the process'!h?re t h e rw a l development iapact s tudy f a i l e d t o s a t i s f a c -t o r i l y desucceeds.i n this rfine p r o j e c t o b j e c t i v e s and scope, t h innd i sg a c t u a l p r o je c t achievementse p o s t is easier because they ar- more. -presentzd -

.s t udyand failureshonestlyT h i s study l a c k s th e s t y l e that evidences a goodresearch design as does i t s successor , Xowever, t h e weaknesi n methodology t h a t appeared i n t h e rural developmentstudy is sawwhat lessened here.I n consider ing t h e d i f f e rences i n t h e two reports,consider a l s o t h a t t h e JFK Xedica l Center ivas a siwlefunct ion p r o j e c t where t h e r u r a l develoyment project wasmult i fuf tc t ional , The l o c a l e f o r the former i s urban; f o rt h e l a t t e r rural, T h i s would effect a v a i l a ' P i U t y anda c c e s s ~ b i l i t y f i n f r a s t r u c t u r e and de l ive ry systems --~ o s i t i v e l yn the forner case, negatively i n the l a t t e r .

A l l these f a c t o r s nay account for a f a c i l i t y i n -2iqleraentiaga b i n i s t e r i n g , mollitoriog, and t h e r e f o r e evaluating theJZ! project t h a t was lacking i n t h e rural develo~ment rojec

YIhile revfewing this study, cons ide ra t i on mas e v e n t h ef a c t t h a t this study comes tno years a f t e r t h e phase out o fAID a c t i v i t i e s , 17 years after t h e p r o j e c t ' s i n p l e n e n t a t i o n ,and nine years a f t e r National $Iedical Centes ( a l s o h o r n asthe JFK Center or MMC) became f u l l y operat ional , Sufficientt i n e had lagsed t o conduct r igorous evaluat ive research intoreplicability, as w e l l as e f fec t i venes s and i npac t , rathert han the "quick a d d i r t y" study of which t b i s i s morer ep resen t a t i ve ,',?]ere he ob.iectives and scobe o f the broject defined?

Yes, According t o th e impact study, USAID involvementin the John F. Kelmedy I~fedicalCenter mas i n i t i a t e d I n 1961,with implementation o f A ID activities in 1963 and phase o u tin 1978, During t h a t period a total o f $6.8 million inU.S. capFtal loans and $9.2 million in U.S. govermenttechnical assistance grants helpe0 t o construct t h e


4/9


5/9

T k mpact study magnizes shortccMings but mineins thatduring their research the team mcmered a "story of steady progressciting the follmdng general indicators of the project's success:- ncrease in the anount of the national hdgehealth =vices, spcif ical ly in the ammt=;

- ncrease in the numbers of qualified LiberianMicians anddentists an record in the country, fran f- than 30 in I371to nearly 100 in 1979 (Therewas, -, no indication ofhaJmanyof t h e s e ~ ~ c a l d c c t o r s a s a p p a s e d t o d e n t i sNor was there an indication of how the and dentistsw re deplcyed, i . e . , who benefited mst fran their &ces.);

-- the gmckatian of 61 doctors fran the Medical College of theWversity of Liberia since its first class in 1973;- he evolution of the as a teaching hospital c ~ n ~ l e x ;nd,- he cans-le transfer of hkstern ttschmlagy eddied in

ti. enter's wamdmusing system, rrrodern -tion center,isolathn ward, cancer +search center, diagraostic laboratorieAnm~gthe problems clearly defined in the study- he recurrencosts of the Center whichUmre than 40 percent of t o t a l recurrenna- health costs. Also, the reprtmtions that Ixl- c m -s tm in t s an qerating f m d s and salaries cause high turn- inmdisgnmtlement among t b s e wfio stay, and disincentives for staff to

prfonn a t aptimal levels. These cmst rah ts in the face of risingp r diem patient costs, excessive patient loads (hospitalsand stationary bu3qet.s are likely, the evaluators feel, to lead to adeterbzatian of pat-t care and health serJice delivery quality.!I&i sw of the study is that the thexmding of hospitals iscaused in part by the many dddmn who are treated at hospital faci l ities w i t h o u t coot. !BE stam that the "situation is Likely

a eteriorate unless a separate children's hospital iscreated or present facilities at JFk( are eaq?andsd.Anotkr problem defined by the study was that of the cultural-rqriataiess of the JFR Hospital. "Key informnts" suggest thatthe h0gpiIz.l is largely an -can ~~t whose design and layuut

. do not a~ i f c~ rno the African cul tural context. -, the studyreveals that, equipnent and supplies are amsidered toos ; o ~ h i s t i c a t e d or both staff and patients, 1 to o c c a ~ i \ d '


6/9

The cultural problem is not given as much weight asthe others by the study team. Perhaps, f 2 should not beso quickly dismissed. Questions of cultural appropriate-ness dovetail with questions of whether a particular typeof intervention is what the project beneficiaries want,need, or will adopt.The high death rate at the JFK Hospital (which has

earned for it among Monrovians the soubriquet, Just ForKilling) is explained by the study as a result sf h s use-of the hospital by the populace as a court of last resort.By GOL policy, we are told, the emphasis on the servicesat the Hospital is curative rather than preventive. Furththe study states, the majority of those who die at JFKare poor children. They die, according to the study, ofpreventable illnesses. This suggested to the evaluationteam the priority of focusing the health care system onpreventive medicine,The report could have delved more deeply intoeffectiveness and impact issues of preventive versuscurative medical systems at this point; but did not.Such a treatment would raise questions whether or notinversion of the pyramid (as conceptualized in the JFK. 'project) .s necessary. It would also require the evaluator. . to study more thoroughly the effectiveness and impact of .the delivery system linkages between the core (NMC) andthe periphery (village health care centers)

Impact of the project was broken out in three categorieconomic, social and policy. This study makes what amountsto a non-statement about the economic impact of the projectleaving the reader unsure whether the impact on the nationaeconomy was more negative, more positive, or non-existent.Reference is made to multiplier effect of four which madeannual economic impact about $28 million in 1979. What doethis really say about income? It says nothing about actualper capita increase from prior years (specifically, from prproject baselines). Even if one accepts, without question,the szudy's statement that income increased, who enjoyedthis increase; what indicators does the study use to showthat this increase affected consumption or improved thequality of life. The study makes mention of 100 U.S.-trainLiberian professionals, "most" of whom, we are told, are stemployed in the national health care system. But, what doe"most" constitute? And, what did ''brain drain" cost theeconomy in real terms?


7/9

In addition, no attenyt -as made by the study team tomeasure the incErec t benefits of increased productivity dueto a healthier population because, as the stusij states, suchf i a d i n g s are difficult to measure quantitativePy. F tna l l y ,me are told, "on the negative side, the annual c o s t ofoperating the %IC %s . . . ore than 4Q percent o f thetotal health budge t , " creating a serious drain on nationalpublic finance not of f se t by income from pa t i en t fees whichare i n s ign i f i c an t .h o n g outcomes a t h o s i t i v e social impact, t h e s%uincludes-- major contribution t owa rd the inrprovement, over tiof basfc h e a l t h i nd i c a t o r s (including l i f e expectanat birth, infant mortality rate, child mortality rapopulatron per physician, population per nurse,-crude death rate) ;

hospital serves as ttvillage quare patients wait-ing in halls and vaitiag rooms converse atkt acquaint-ances ( ! ) ;hospital has l l ~ e l P a r spects in that the poor mhocome there a r e given d, cloth* and/o% medicalcare, according t o their needs,

Scoiai drawbacks ?qere described as overcrowding, long-maitipmiods f o r gatients, t h e aon-central IscatLon o f t h e hospi~ h i c h equires up-country patients to travel great distanct o get treatment, and a perception among the general gubl it h a t the hospital is a death house vhich makes some peaplreluctant to seek medical. attention there. Xot includedthe social irrpacts sectfon of the report, but an outcone t;ri,.s i gn i f i c an t social, and some seonomic, ramifications is thee , q l o i t a t l on of t h e h o g p i t a by those who can afford to pay,by p ~ i v a t e onganies, and by t h e local government, Samevell-to-do use the hospital as a nursing home for terminalelderly, and p~ivate omganLes send their workers (at noc o s t ) to JrT I~Iemorial. o r treatment when they could a f fo rdt s send them t o pr iva t e cPinries o r hospitals. The localgoverment joins these axphoilers , using JFH facilities asthe official city morgue, The depredations by these threegroups has added t~ aLready critical space problems.

The greatest impact an GOL pol icy s t a t e d by the reporta s itscomitment to the f stitutionalf ation of the FiMC.The project has f a i l e d t o exert the anticipated b p a c ton GCL i n n regard to f a n f l y planning policy: that I t s purposeo f l f inprovingna%i nal f n l l g planning servlc stl has had~ r y l i n b t e d dapta t ion by the GCL, and has n o t led to3oXcg chmgz, Ths has had little effect, as well, inchmeeg GOL preference for curative services as opposedts preventive sezvices according ta tfe eprt.


8/9

G e n e a l y , a c h i m t of project objectives a t the p u p s l e ~w a s successful despite pmb l a ~ l ~hat need to be address&. H a e v e r ,success at the goal l eve l is not as apparent and seans m i k e l y unlessGaL poclicyard camifment, in the areas of family plarnring and p.revmtimikine , m e s .Coes the repcat have a c l e ~e=& design?

In this -, as in the U m fa Gounty rural develqment studypmpa: research design is questicnahle. ?he M y oes not state itslxupoJe or objectives. Although, f m n the cover sheet,one acpectst o ~ ~ t i n p a c t . t h e r e a d e t i s m t m l d i n w f i a t ~Wi.+Ue presented. What are the major m s e s b u t Wth care indew lap ing countries tbat need tD be addressed in terms of this projectW t re the najor*mpact issues: institutionalizatim, effectiveness,benefit to the -tian, or replicabili* of the prqiect? This shdydoes Plot-say.

lh is sMy , unlike the Upper Lofa Caunty m y , ees have a c011~1wfiich states the general arceps of the project tut sees a need tom-te the Maternity Wspital and teaching cmpaents, r m t e certaSecticns of the JFR Bospital, and amstmct a c h i l d r e d s hospital. -,~ o l i c ymplications for AID or other donors rat &e clear.

As indicated earlier in t h i s paper, questims raised by the outcarof the project's intemention as to ODL policy no t given adequateaddress. For instance, the tradeoff I=etTieen preventive and curativemedical systans was not welle,ay be a'o s t criticalquestion facing the U i a n health planners at t h i s point. The re-also fails to ex=mine the issue of family planning as tfiorcuqhly asit s h d d , considering that establishing family planning in the nationwas amajor goal which has not been acMeve3, ard for which GOr,c u m i ~ ts lacking."Was the -logy viable?

A report shDuld indicate its sources of infomtion: thep ak l v e d in research; the type of research (i-e.. aqerimental or imar.-al, case study, or o b m t i a n ) and har t h esearch wasm n d u ( i -e . , intensively or ecknsively) ; where research was done(e.g. , in the f ie ld, or at a desk); the d e v d c p m t of research strategthe mjor issues imolmd and haw Lhey w r e determined. Also S c a t &in the rnethcdcIogical statemnt skpl ld k he amunt of t i n e spent inresearch. This study fulfills sane, h t not all of these -ts.


9/9

The "key informant1' y was used to canvas pdz ic ipan t opinionof the project, Eeyond this source of i n f o m t i a n , it WEIwhat the ather sources of data wsre. The "key informant" approachto eliciting unbiased impressions is suspect. In this stdy, forinstance, anly urban W c i p a n t s canvassed. Yet, tbe project wahav e impact on all of the nation. So, thy w e ival participantsnot sumeyed to e l i c i t their opinions about the outreach effectivenessof the health care delivery s y s t em headed up by theW?

F k a r e t o l d t h a t t b r e s e a r & ~ s ~ b y a p r i v a t e L i b e r *~~nagerrwtonsulting and research firm, but given no specificinfonmtim Wt the Wv i d u a l qualifications of the researcherskavoIVt?d in the Lnpart study. Although wz are told ttat interviewsof informants took place in the field ( tbmovh), the type ofresearch -is ot defined, W are not told haw it= onducted.

To sane extent ( in terms of the questicmnaize) M o p n e n t ofresearch stmtegywas discussed. hFe are to ld in- No. 3, t h a tthe total evaluation effort took ninew. No mention is made ofwhat the study's majo r issues are, or I.low they -e determined,Conclusions

The report canvinvingly cancludes that the project has beensuccessful in pmviding hspital facilities, and teachbq facilities,h . ic2 . t carprise the K. However, budgetary m t s , dinhistrati=problems, as well as problarrs w i t h staff norale and turn-bver raiseserious questions a b u t the future a b i l i t y of the WE to meet thehealth needs of even Mommvhn citizens (who appear to be the chiefbeneficiaries of the project a t th is p i n t ) . y/Jhole sections of "Lhep p d a t i o n appear to be in need of outreach of basic health caresemices, and the o-ty cost of an urban hospital 40percmt of the country's nedical W g e t is high. Discussion ofpreventive -cine and family planning mud have been paxticularlyd c e m e in this regard.

% find the quali.ty of the i r p c t study deficient."quick a d dirty" evaluation it is not up to par. (Bywith the s tud ies dam in the "Bennett' xercise,unprofessional job. A need for guidance, t~ lfissicm-host-country evaluators, in designing a d od-g hipa&research, is indicated,

usaid response to 1980 impact study

Documents