the journal of financial transformation #11

journal 09/2

00

4/#

11

the journaloffinancialtransformation

Economic

Financial

Enterprise

Data

Recipient of the APEX Awards for Publication Excellence 2002-2004

bmw williamsf1 team

©2003 Hewlett-Packard Development Company, L.P.

The BMW WilliamsF1 Team chose HP to provide the supercomputer used to design the car and to conduct thousands of race simulations. And before the car even hits the track, HP servers and notebooks are used to analyze research data that enables the team to make precise suspension and engine adjustments. It’s mission-critical computing for fast-moving enterprises, and then some. www.hp.com/plus_bmwwilliamsf1

Our 200 mph laboratory.

Editor

Shahin Shojai,DirectorofStrategicResearch,Capco

Advisory Editors

Predrag Dizdarevic,Partner,Capco

Bill Irving,President,Capco

John Owen,Partner,Capco

Editorial BoardFranklin Allen,NipponLifeProfessorofFinance,TheWhartonSchool,UniversityofPennsylvaniaJoe Anastasio,CEO,CrossBorderExchange,andPartner,CapcoPhilippe d’Arvisenet,GroupChiefEconomist,BNPParibasJacques Attali,Chairman,PlaNetFinanceRudi Bogni,FormerChiefExecutiveOfficer,UBSPrivateBankingBruno Bonati,MemberoftheExecutiveBoardandDivisionHeadTechnology&Operations,CreditSuisseFinancialServicesDavid Clark,NEDontheboardoffinancialinstitutionsandaformersenioradvisortotheFSAGéry Daeninck,formerCEO,RobecoDouglas W. Diamond,MertonH.MillerDistinguishedServiceProfessorofFinance,GraduateSchoolofBusiness,UniversityofChicagoElroy Dimson,ProfessorofFinance,LondonBusinessSchoolNicholas Economides,ProfessorofEconomics,LeonardN.SternSchoolofBusiness,NewYorkUniversityMichael Enthoven,ChiefExecutiveOfficer,NIBCapitalBankN.V.José Luis Escrivá,GroupChiefEconomist,GrupoBBVAGeorge Feiger,ExecutiveVicePresidentandHeadofWealthManagement,ZionsBancorporationGregorio de Felice,GroupChiefEconomist,BancaIntesaWilfried Hauck,ChiefExecutiveOfficer,AllianzDresdnerAssetManagementInternationalGmbHThomas Kloet,SeniorExecutiveVice-President&ChiefOperatingOfficer,FimatUSA,Inc.Herwig Langohr,ProfessorofFinanceandBanking,INSEADMitchel Lenson,GlobalHeadofOperations&Technology,DeutscheBankGroupDavid Lester,ChiefInformationOfficer,TheLondonStockExchangeDonald A. Marchand,ProfessorofStrategyandInformationManagement,IMDandChairmanandPresidentofenterpriseIQ®

Colin Mayer,PeterMooresProfessorofManagementStudies,SaïdBusinessSchool,OxfordUniversityRobert J. McGrail,ChairmanoftheBoard,OmgeoJeremy Peat,GroupChiefEconomist,TheRoyalBankofScotlandJos Schmitt,Partner,CapcoKate Sullivan,ChiefOperatingOfficer,e-CitiJohn Taysom,Founder&JointCEO,TheReutersGreenhouseFundGraham Vickery,HeadofInformationEconomyUnit,OECDNorbert Walter,GroupChiefEconomist,DeutscheBankGroupDavid Weymouth,ChiefInformationOfficer,BarclaysPlc

TABlE oF conTEnTs

THE noBEl lAuREATE ViEw

6 The world of economic data – A discussion with Prof. Paul A. samuelsonPaulA.Samuelson,InstituteProfessor,Emeritus;ProfessorofEconomics,Emeritus;GordonY.BillardFellow,MassachusettsInstituteofTechnology;WinnerofTheBankofSwedenPrizeinEconomicSciencesinMemoryofAlfredNobel1970

Economic

10 opinion: The impossibility of accurate macro-economic forecastingPaulOrmerod,Director,VolterraConsulting

15 Revisions to GDP and related estimatesDennisFixler,ChiefStatistician,BureauofEconomicAnalysisBruceGrimm,Economist,BureauofEconomicAnalysis

23 The reliability of quarterly national accounts in seven major countries: A user’s perspectiveRobertYork,SeniorEconomist,InternationalMonetaryFundPaulAtkinson,DeputyDirector,theDirectorateforScience,TechnologyandIndustry,OECD

31 The effect of telecom density data on growth, efficiencies, and distributions in global economiesLallRamrattan,Lecturer,UniversityofCalifornia,BerkeleyFrankDiMeglio,Consultant,MacrosoftCompanyMichaelSzenberg,DistinguishedProfessorofEconomics,PaceUniversity

FinAnciAl

44 opinion: corporate action processing: complexity and riskJamesFemia,ManagingDirector,AssetServices,DTCCGunnarNiels,Economist,OxeraConsultingLtd.

48 opinion: what lies beneathLarsHamich,ManagingDirector,STOXXLtd.

52 opinion: Hedge fund indicesJamesR.Hedges,IV,President&ChiefInvestmentOfficer,LJHGlobalInvestments

58 opinion: Data management in financial services 2004 and beyondAndyDilkes,ChiefTechnologyOfficer,UnitySystems

62 opinion: integrated data architecture – The end gamePredragDizdarevic,Partner,CapcoShahinShojai,DirectorofStrategicResearch,Capco

67 Reference data primerMarilynHignett,Partner,Capco

75 Data in financial institutionsRichardMcLaughlin,Solicitor,Technology,MediaandTelecommunicationsDepartment,NabarroNathansonsolicitors

81 Data mining in finance: From extremes to realismBorisKovalerchuk,Professor,DepartmentofComputerScience,CentralWashingtonUniversityEvgeniiVityaev,SeniorScientist,InstituteofMathematics,RussianAcademyofSciences

EnTERPRisE

92 opinion: The legal assault against marketingKirkHerath,ChiefPrivacyOfficer,AssociateGeneralCounsel,NationwideInsuranceCompanies

96 opinion: who owns the customer? who owns the data?KeithMacDonald,Partner,CapcoMarkDynes,ManagingPrincipal,Capco

100 opinion: Privacy challengesRayEverett-Church,PrincipalandChiefPrivacyOfficer,ePrivacyGroup,LLC

102 opinion: Data quality management: How to produce high quality reports for risk managementBarbaraBoos,ITManager,RiskApplications,EuropeanInvestmentBank

108 opinion: The fourth leg of the stool – Data protectionJohnP.Rosato,CEO,CSTechnology

110 opinion: steady progress – But could do betterPeymanMestchian,Director,RiskManagementPractice,SASUK

112 opinion: The shift to web servicesKurtGilman,Partner,PricewaterhouseCoopersShawnConnors,SeniorManager,PricewaterhouseCoopers

117 A user-centric approach to effective enterprise data servicesGopiChelliah,ChiefInformationOfficer,Cross-BusinessTechnology&Operations,DeutscheBankAG

125 Extracting the business value of iT: it is usage, not just deployment that counts!DonaldA.Marchand,ProfessorofStrategyandInformationManagement,IMDandChairmanandPresidentofenterpriseIQ®

133 Taking snapshots of the internet: new database of insider transactions and liquidityStevenM.Benveniste,Researcher,TheHaroldPriceCenterforEntrepreneurialStudies,TheAndersonSchool,UCLADukeK.Bristow,FinancialEconomist,TheHaroldPriceCenterforEntrepreneurialStudies,TheAndersonSchool,UCLAAlfredE.Osborne,Jr,DirectorofTheHaroldPriceCenterforEntrepreneurialStudiesandAssociateProfessorofBusinessEconomics,TheJohnE.AndersonGraduateSchoolofManagement,UCLA

143 The informational role of financial analysts: interpreting public disclosuresDonalByard,AssistantProfessor,ZicklinSchoolofBusiness,BaruchCollege,CityUniversityofNewYorkKennethW.Shaw,AssociateProfessor,JosephA.SilvosoFacultyFellow,CollegeofBusiness,UniversityofMissouri

Dedicating an issue of the journal to data nicely highlights the ambiguities inherent in the subject. Few

executives,forexample,understanditscomplexitiesbutallrecognizeitsimportanceasarawmaterialtothe

financialservicesindustry.Mostfinancialinstitutionsspendmillionstryingtomanageandexploitdata,but

fewareclearexactlyhowmanymillionsorwhethertheyaregettingapaybackforallthisspending.

The dichotomy is perhaps most stark in the distinction between attitudes to transaction data — the data

requiredtoexecutetransactions—anddecisiondata—dataonwhicheconomicandbusinessdecisionsare

based.Whenitcomestotransactiondatathefocusisonvolumes,efficiency,andcost.Incompletecontrast,

decisiondataisprizedforitsmeaning,fortheinsightsthatitprovidesandforitspowerasapotentialsource

ofcompetitiveadvantage.

ThisisacontrastofwhichIampersonallywellawarebecausebothaspectsofthedataissuearereflectedin

ourownbusiness.Weprovideandmanagetransactiondataasrawmaterialonbehalfofourclientsandwe

areobsessedwiththeaccuracy,efficiency,andtimelinessofthedataweprovide.Incontrast,inourconsulting

work,itisthemeaningthatdataholdsthatissovaluabletousandourclients.

Inmyview,bothaspectsofdataareequallyimportanttoourindustry.Transactiondataisasimportantto

financial services businesses as air is to those who manage them. At the same time, few organizations

thatignoretheimportanceofgooddatatoqualitydecisionmakingsurvive.Thiseditionofthejournalthere-

foreaddressesnotonlytheeffectivemanagementofdataanddataprocessesbutalsotheeconomicand

commercialvaluethatcanbederivedfromit.

Ofcourse,inshowcasingcontributionstothinkingonbothaspectsofthedataissue,Ihopethatthisjournal

itselfprovesusefultoyourownthinkinganddecisionmaking.Aboveall,though,Ihopethatyouenjoythis

issueandfindinitbothinsightandmeaningforyourowndatachallenges.

RobHeyvaert,

Founder,ChairmanandCEO,Capco

why data?

Theword‘data’isbeingusedquitealotthesedays.Fromthereliabilityofintelligencedatagatheredpriorto

thewarinIraqtothedatausedindeterminingpossiblerisesinU.S.interestrates,datahasbecomeavery

popularterm.Ithas, ifyouwill,madeacomebackandissurpassingwordpeerssuchas‘information’and

‘knowledge’.

Inthesamewaythatpeoplearechallengingthedataemployedtojustifythewar,questionsarebeingraised

aboutthedatausedtodefendchangesinmonetarypolicy.Forexample,canexpertscopeeffectivelywiththe

hosepipeofinformationpointedatthem,especiallywhenmostofthisdataisbackward-looking?Cancentral

banksthenusethisdatatotrulyinfluencetheeconomy?Experienceextrapolatedfromtheintelligenceworld

hasshown,atleastwiththetragiceventsinNYCandMadrid,howdifficultthisis.

Mostofushavebeeninterestedinthesequestionsforyearsbuthaveyettohearconvincinganswers.This

issueofthejournalaimstoprovidethem,aswepresenttheviewsofsomeoftheworld’smostrespected

economists,includingthedistinguishedNobelLaureate,PaulSamuelson,onthesubjectofdata.

Globalmacro-economicandsocio-politicalenvironmentsareveryimportanttobusiness,astheyimpactrisks

andreturns.Butfinancialexecutivesareevenmoreinterestedinhowtheycanimprovethemanagementof

thisrawmaterial—data—withintheirownorganizations.Howdataiscaptured,disseminated,shared,and

analyzedacrossthewholeenterprisecanhavesignificantimplicationsonefficiencyandprofitability.Thatis

whywehavededicatedmostofthecurrentissuetothistopic.

Insectiontwo,wefocusonhowfinancialinstitutionsgoaboutcollectingdatafrommanyinternalandexter-

nalsourcestoimprovethewaytheydealwithclients,regulators,andcompetitors.Theaccumulateddatais

whereallthisinformationisconcealed.Whatarethenneededaretoolstoextractthatinformation.

Sectionthreefocusesonhowthetruebusinessvalueofdatacanbecaptured.Itdescribesthequantitative

andqualitativetoolsavailabletomanagementtomakethebestpossibleuseofdata.Thisisespeciallyuseful

becausefinancialinstitutionsdonotseemtobeaswellplacedasotherindustriestotakeadvantageofthese

tools.Consequently,wefeatureexpertswhounderstandnotonlyourindustrybutothersaswell.Here,they

giveguidanceonhowtotransformdataintoinformationthatcanhelpusimprovethereturnsfromcurrent

clients,attractandretainnewcustomers,andensure thatweachievetheseobjectiveswhilemeeting the

ever-increasingnumberofregulationsimposedonourindustry.

Wehopethatthisissueachievesitsobjective.Wealsohopethatyouenjoyreadingthisissueandcontinueto

supportthejournalbysubmittingyourbestideastous.

Onbehalfoftheboardofeditors

Renaissance of data

8 - The Journal of financial transformation

THE NOBEL LAuREATE VIEW

The world of economic data

A discussion with Prof. Paul A. samuelsonInstitute Professor, Emeritus; Professor of Economics, Emeritus; Gordon Y. Billard Fellow, Massachusetts Institute of TechnologyWinner of The Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel 1970

WeareveryhonoredthatProf.Samuelson,oneofthe

greatest living economists, took time from his very

busyscheduletoansweranumberquestionsthatwe

had posed to him. Below, you will find the questions

posed and the answers kindly provided by Prof.

Samuelson.

Q: Many economists base their judgments about the

future potential of an emerging economy on the data

that is published by both the local governments and

the many international bodies. How reliable do you

believe these data are? And how helpful are they in

helping us assess the future of these countries?

Prof. Samuelson:Mostmixed-marketnationsproduce

meaningful macro-economic data. Changes in these

aremoremeaningful thanabsolute levels.Forplaces

like the USSR or Mao’s China such data are less

reliable.Asidefromoutrightcheatingorsimple igno-

rance, the reason for their unreliability is that prices

and price ratios in such places lack much corres-

pondencetohumanneedsanddesiresandlackmuch

correspondence to technological trade-offs numeri-

cally.GarbageInproducesGarbageOut.(Example:CIA

figuresforEastGermanybeforeunificationwereesti-

matedtobeaboutthree-quartersofWestGermany’s

Net National Product per capita. After unification,

expertscametolearnthatmaybeone-thirdwouldbe

nearerthemarkthanthree-quarters.)

Q: The experience of the Asian countries proved that

multilateral organizations, such as IMF, do not always

get it right when it comes to emerging markets. Do

you put that down to inadequate data or lack of cor-

rect analytical skills? Kindly explain.

Prof. Samuelson:InAsia(andelsewhere)neitherlocal

governments nor IMF’s nor large foreign lenders

‘alwaysget it right.’After the 1997Thailandfinancial

crash,IMFadviceworkedbetterforsomeplacesthan

forothers.(SouthKoreaversusMalaysia?TheIMFwas

not involved inJapan’sstagnationbut its 1990-2000

recordwasperhapsworstofall.)

Q: With regards to the more developed countries, we

are aware that most economic data is looking back.

How valuable is such data to economists for predict-

ing the future?

Prof. Samuelson:Goodpredictionaboutthefutureis

not yet possible. However, disregarding the past will

usuallygenerateespeciallyinaccurateforecasts.

Q: Many question the predictions given by govern-

ments of the major industrial countries due to the

fact that they are basing their analyses on data that

is certainly not valid when initially issued and cer-

tainly not much more useful when revised. What is

your view?

Prof. Samuelson:Onemustweighcarefullyconflicting

databasesandmakeguessesaboutwhichwillturnout

tobemostusefulontheaverage.

Q: With so much data being funneled at decision-

makers, how do they know which data is actually

useful and which is not? For example, many countries

no longer monitor the money supply as much as they

used to. Is that a wise decision?

Prof. Samuelson:2004knowledgeaboutmoneysup-

plies in 150 countries is, if anything, more accurate

thanitwasin1994,1964,or1944.Butthebestecono-

9

mists no longer agree with Milton Friedman of the

1970s,whoseviewwasthatcentralbanksshouldfocus

prettyexclusivelyonanydeviationsingrowthrateof

currencypluscheckablebankdeposits.AttheBankof

England or the U.S. Federal Reserve, other variables

thandM/dtenteralsointotheirstabilizationactivities.

Q: Economic forecasting in general has been found to

be a not so accurate a science. What do you put this

down to, lack of useful data or inadequate analytical

capabilities?

Prof. Samuelson:Economistsdohavemoreandbetter

present and past data than they used to have. And

policymakershavehelpedtoreducetheamplitudesof,

say, 1970-2004businesscyclesascompared to 1910-

1944businesscycles.However,thereisnovalidreason

tothinkthatifwewereonlyalittlemoreknowledge-

ableandalittlemoreenergetic,wecouldconvergeon

highly accurate macro forecasts. Mass behaviors

answertonosimplediscoverablesetofrules.

Economic

The impossibility of accurate macro-economic forecasting

Revisions to GDP and related estimates

The reliability of quarterly national accounts in seven major countries: A user’s perspective

The effect of telecom density data on growth, efficiencies, and distributions in global economies

The impossibility of accurate mac-ro-economic forecastingPaul OrmerodDirector, Volterra Consulting

ber of economics professors of unexceptional conservatism.

ThepurposeoftheSocietywastohelpbusinessmenforetell

the future.Forecastsweremadeseveral timesamonthand

undoubtedlygainedinstaturefromtheirassociationwiththe

augustnameoftheuniversity’.Galbraithrelatesthatinearly

1929theSocietydecidedthatamildrecessionwasdue.‘Week

byweek,theyforetoldaslightsetbackinbusiness.When,by

the summer of 1929, the setback had not appeared, the

Societygaveupandconfessederror.Businessmightbegood

afterall.’AslateasNovemberofthatyear,theSocietyargued

that ‘aseveredepression isoutsidetherangeofprobability.

Wearenotfacingprotractedliquidation.’InGalbraith’slaconic

phrase,‘...thisviewtheSocietyreiterateduntilitwasliquidat-

ed.’

ThebasicproblemwasidentifiedbyFishermanydecadesago.

Therearesomanyfactorswhichcausetheeconomicupsand

downsofthebusinesscycleandtheorderandcombinations

inwhichtheyappeararesovariedthatitisvirtuallyimpossi-

bletodistinguishbetweenthistypeofdataandthosewhich

weregenuinelyrandomintermsoftheirpredictability.There

aretoomanyfactorsandnotenoughdatawithwhichtoiden-

tifytheirseparateimpacts.

Themoderntrackrecordofforecastingiscertainlycompati-

blewiththisview.Indeed,thereareexamplesofspectacular

failures, which caught policymakers completely by surprise.

The East Asian crisis of 1997/98 is a notorious example. In

1998,therewasamajoreconomicrecessioninthisarea,with

output in Indonesia, for example, falling by 15 percent. Yet

noneofthiswaspredictedatall.

InMayof1997,forexample,theInternationalMonetaryFund

(IMF)forecastedacontinuationoftheenormousgrowthrates

which those economies had experienced for a number of

years:7percentgrowthwasprojectedforThailandin1998,7.5

percentforIndonesia,and8percentforMalaysia.ByOctober,

thesehadbeenreviseddownto3.5,6,and6.5percentrespec-

tively.ButbyDecember1997theIMFwasforecasting‘only’3

percent growth for Malaysia and Indonesia, and zero for

Thailand.Inotherwords,justonemonthbefore1998began,

theIMFhadnoinklingofthespectacularrecessionwhichwas

abouttotakeplace.

Inthedevelopedworld,thefailuresofforecastingarenotas

dramatic,buttheyarepersistent.Wenowhaveatrackrecord

of published forecasts going back at least 30 years. These

coverboththepublicandprivatesector.Andtheycoverdif-

ferentschoolsofeconomists.Allareuniformlybadbygenuine

scientificstandards.

Asexamplesoftheone-yearaheadforecastingrecordforGDP

growth, for the U.S. economy recessions have not generally

been forecast prior to their occurrence, and the recessions

followingthe1974and1981peaksinthelevelofoutputwere

not recognized even as they took place1 [Stekler and Fildes

(1999)]. Furthermore, growth has generally been overesti-

mated during slowdowns and recessions whilst underesti-

matesoccurredduringrecoveriesandbooms[Zarnowitzand

Braun (1992)]. For the U.K., the predictions of the Treasury

overthe1971-1996periodhavebeenatleastasgoodasthose

ofother forecasters,but themeanabsoluteannual forecast

errorfortheseone-yearaheadpredictionswas1.45%ofGDP,

comparedtoanactualmeanabsolutechangeof2.10%[Mellis

andWhittaker(1998)].In13Europeancountries,overthe1971-

1995 period, the average absolute error was 1.43% of GDP,

comparedtotheaverageannualchangeof2.91%[Öllerand

Barot(1999)].

Ingeneral,theforecastingrecordexhibitsacertaindegreeof

accuracyinthattheaverageerrorovertimeissmallerthan

thesizeofthevariablebeingpredicted.Buttheerrorisstill

largecomparedtotheactualdata,andmostoftheaccurate

forecasts were made when economic conditions were rela-

tivelystable.

Areasonwhichisoftengivenforthepoorforecastingrecord

isthateconomistsmaynotknowexactlywheretheeconomy

isatthetimewhenaforecastismade.Outsidefinancialmar-

kets,economicdataappearswitha lag.Equally importantly,


13

Generations of economic policymakers have been raised in

themechanisticviewoftheworld,withthechecklistmentali-

ty:toachieveaparticularsetofaims,drawupalistofpolicies,

and simply tick them off. It is a comforting environment in

which to live, being seemingly dependable, predictable, and

controllable.TheplannersoftheSovietUnionbelievedthisto

bethecase.Buttheireconomyultimatelycouldnotcompete

withthemoredisorderedworldofcapitalism,notasitispor-

trayedinconventionaleconomics,butasitactuallyexists.The

intricateinteractionsofmillionsofindividualagentsgiverise

tocomplicatedbehaviorofthesystemasawhole.

Central banks and treasuries around the world continue to

operateas if futuremovements intheeconomycanbepre-

dictedaccurately,andasifdecisionstheytakenowwillmake

thefuturebetter.Butitisaworldofillusion.Realityisnotlike

thatatall.Alongtrackrecordhasnowbeenbuiltuponjust

how(in!)accurateeconomicforecastsare,andhowlittlethe

authoritiesknowabouthowtheeconomyactuallybehaves.

In order to exercise successful short-term control over the

economy, in the sense of being able to bring about a more

desirableoutcomethanotherwisewouldhaveoccurred, the

authoritiesneedtobeabletobothmakeforecastswhichare

reasonablyaccurateinasystematicwayovertimeandunder-

standwithreasonableaccuracytheeffectofchangesinpolicy

instrumentsontheeconomy.

Unlesstheauthoritiesknowwithreasonableconfidencewhat

thestateoftheeconomywillbein,say,oneyear’stime,itis

not possible to say what action is required now in order to

bringaboutamoredesirableoutcome.Andunlesstheauthor-

itiesunderstandtheimpactoftheiractions,itisnotpossible

to know what should be done in order to bring about any

desiredoutcome.

Governmentsofallideologicalpersuasionsspendagreatdeal

oftimeworryingabouthowtheeconomywilldevelopinthe

shortterm,overthenextcoupleofyears.Iftheanxietylevels

ofpoliticiansweretheonlyissue,fewwouldbeconcerned.But

our representatives do not merely contemplate the short-

termfuture,theyseektoinfluenceit.Elaborateforecastsare

prepared,notjustbygovernmentsbutalsobyacademicinsti-

tutionsandcommercialcompanies.Adviceisfreelyofferedas

tohowtheprospectsfortheeconomycanbeimproved,byan

alterationtoincome-taxrateshere,oratouchofpublicexpen-

diturethere.Butthecontrolwhichgovernmentsbelievethey

have, intheirabilitytobothmakereasonablyaccuratefore-

casts and understand the consequences of policy changes

designedtoaltertheoutcome,islargelyillusory.

Theideathatshort-termfluctuationsintheoveralleconomy,

theboomsandrecessionsofwhatiscalledthebusinesscycle,

areintrinsicallyunpredictableisnotnewineconomics.Milton

Friedmanargued intheearly 1950sthatshort-termgovern-

mentinterventionwasjustaslikelytoaccentuatethefluctua-

tions of the business cycle as it was to dampen them. In

essence,hewasveryskepticalthatgovernmentscouldantici-

pateeventswithsufficientaccuracy.Byluck,someindividual

governmentswouldgetthetimingoftheirinterventionsright

andsucceedincontainingthestrengthofboomsandslumps,

buttheirunluckycounterpartswouldonlysucceedinintensi-

fyingthefluctuationsintheireconomies.

ThesameconclusionwasreachedevenearlierbyIrvingFisher

of Yale, the most distinguished American economist of the

earlydecadesofthe20thcentury,usingamoresophisticated

argument. He argued that the business cycle is inherently

unpredictable.Hebelievedthatmovementsovertime inthe

volumeofoutputwere‘acompositeofnumerouselementary

fluctuations, both cyclical and non-cyclical,’ and quoted

approvingly from his contemporary Moore, who wrote that

‘businesscyclesdifferwidely induration, in intensity, in the

sequence of their phases and in the relative prominence of

theirvariousphenomena’.

Fisher’s contemporaries at Harvard soon provided a memo-

rableexampleofforecastingfailure.JKGalbraith,inhisbook

TheGreatCrash1929,tellsthestoryoftheHarvardEconomic

Society:‘…anextracurricularenterpriseconductedbyanum-

1 Economicdata,exceptinfinancialmarkets,donotappearimmediately,anditcan

beseveralmonthsbeforeapreliminaryestimateofthelevelofoutputinagiven

periodbecomesavailable.

theofficialestimatesofthestateoftheeconomyatanypoint

mayalterastimegoesby.Estimatingthesizeoftheeconomy,

forexample, requirescollectingdata fromahugevarietyof

sources,suchasInternalRevenueinformationonhowmuch

taxwepay,salestaxreceipts,andmanymore.Wholebooks

havebeenwrittenonthis.Bookswhichmakeeventheworks

ofProustreadlikeabestsellingairportnovel.

Buttheoperativewordinallofthisis‘estimate’.Theeconomy

is not something which can be put on a pair of scales and

weighed. All the bits and pieces of information need to be

juggledtogether,ratherlikeagiantjigsawpuzzle,inthehope

thatareasonablycoherentpictureemergesattheend.

Inevitably,acertainamountof judgment isrequired.But,as

timegoesby,newinformationisreceivedabouttheeconomy.

Thisisnotnewinthesenseoftellinguswhathashappened

atamorerecentdate.Itisnewinthesenseoftellingusmore

aboutwhatwashappeningataparticularperiodinthepast.

Onoccasions,thiscanbeseriouslymisleading.Forexample,in

the 1992 U.S. Presidential election, the American economy

was emerging from the 1990-91 recession. At the time, the

recovery was thought to be slow and hesitant. Bill Clinton’s

phrase‘it’stheeconomy,stupid’seemednevermoreapposite.

Yetnow, itappearsthatatthetimeoftheelectionAmerica

wasgrowingverynicely,atarateslightlyfasterthanthepost-

waraverage.

But there are even deeper problems with the nature of the

data with which economic forecasters are obliged to work.

Powerfulmodernmathematicaltechniquesenableustoshow

thatFisher’sinsightsofthe1920sandFriedman’softhe1950s

were essentially correct. The poor forecasting record by

economistsappears tobedue to inherentcharacteristicsof

the data, and cannot be improved substantially no matter

whateconomictheoryorstatisticaltechniqueisusedtogen-

eratethem.

Imaginebeinginaremotepartofthecountrytryingtolisten

toaradioprogram.Thestrengthofthesignalmaybeweak,

andthereceptiondominatedbyinterference.Thismaybeso

strongthatitishardeventotellwhattheprogramisabout.In

ananalogousway,wecananalyzetheproportionsof‘signal’,

orgenuineinformation,and‘noise’,orrandominterference,in

adataseriessuchasGDPgrowthorthechangeininflation.

Thehigher is the ratioofnoise tosignal, theharder it is to

makesenseoftheseries,nomatterwhattechniqueortheory

isusedtoanalyzeit.Thisisexactlythecasewithmostmacro-

economicdataseries.Apaperin‘PhysicaA’,theworld’slead-

ingjournalofstatisticalphysics,setsouttherelevantmathe-

matics[OrmerodandMounfield(2000)],whichisnoteasyto

conveyinashortarticle.

Butwhetheritistheactualforecastingrecordortheapplica-

tionofmodernmathematicstoeconomicdata,theconclusion

isthesame.Thetrackrecordofeconomicforecastingispoor,

andshowsnosignofgettingbetterovertime.

Aclearimplicationoftheaboveisthatanapproachtopolicy

whichisbaseduponanticipatingtheimmediatefuturestate

of the economy in the business cycle, and taking decisions

nowtotrytoaltertheoutcome,isessentiallymistaken.This

doesnotmeanthatactionshouldnotbetakenoncetheposi-

tionoftheeconomyinthebusinesscyclebecomesclear.But

attempts to anticipate events are unlikely to be successful

overtime,nomatterwhatmethodologyisusedforprediction.

Thestandard instrumentsofmacroeconomicpolicy—policy

designed to influence the behavior of the economy at the

aggregate level — have come to be seen since the Second

WorldWarasvariables,suchaspublicexpenditure,taxation,

and interest rates.Theviewthatgovernments,ormonetary

authorities,cansettheseinordertocontrolthecoursewhich

theeconomyfollowsisstillwidespread.

Separatefromthequestionastowhetherfuturechangecan

be anticipated with any reasonable degree of accuracy is

whethertheimpactofchangesinthesepolicyvariablesiswell

understood. Changes in, say, tax or public expenditure

undoubtedly have an effect on the course of the economy.


15

However,despiteasubstantialresearchcampaignspanningat

leastthreedecades,appliedeconomistsarebynomeanscer-

tainoftheimpactofsuchmeasures.Theuncertaintycaneven

extendtothesignoftheeffect.

Theconventionalapproachtothecontroloftheeconomyat

theaggregate level requires theability tomake reasonably

accuratepredictionsofwhatwillhappeninthefutureinthe

absenceofpolicychangesandhavingareasonablyaccurate

understandingoftheimpactofpolicychangesontheecono-

my.

Neitheroftheseisthecase.Thereareinherentreasonswhy

theabilitytoforecastwithanyreasonabledegreeofaccuracy

over time is severely limited, and why the ability to extract

informationfromaggregatetime-seriesdataabouttheways

inwhicheconomicvariablesinteractisalsorestricted.

The implication is not that governments should do nothing.

Theactionsofgovernmentsclearlydohaveconsequences,for

better or for worse. But the conventional way of thinking,

which requires a world which behaves like a dependable

machine,needstobeabandoned.

References• Stekler,H.,andR.Fildes,1999,“Thestateofmacroeconomicforecasting,”George

WashingtonUniversity,CenterforEconomicResearchDiscussionPaper,99-04

• Zarnowitz,V.,andP.Braun,1992,“Twenty-twoyearsoftheNBER-ASAQuarterly

OutlookSurveys:aspectsandcomparisonsofforecastingperformance,”NBER

WorkingPaper3965

• Mellis,C.,andR.Whittaker,1998,“TheTreasuryforecastingrecord:somenew

results,”NationalInstituteEconomicReview,164,65-79

• Öller,L-E.,andB.Barot,1999,“ComparingtheaccuracyofEuropeanGDPforecasts,”

NationalInstituteofEconomicResearch,Stockholm,Sweden

• Ormerod,P.,andC.Mounfield,2000,“RandomMatrixTheoryandtheFailureof

Macro-economicForecasting,”PhysicaA,280,497-504

Economic


Dennis Fixler1Chief Statistician,

Bureau of Economic Analysis

Bruce GrimmEconomist, Bureau of Economic Analysis

Abstract

AmajorgoaloftheBureauofEconomicAnalysisistoprovide

a timely, comprehensive, and reliable picture of the U.S.

economy.TheBureau’sestimatesofgrossdomesticproduct

(GDP)andgrossdomesticincome(GDI)areitsfeaturedsum-

marymeasuresofeconomicactivity.The reliabilityof these

estimates, which is taken to mean the ability of successive

vintages of estimates to provide a consistent picture of the

economy, is an important issue for economic policy making

andbusinessdecisions. In thispaperwereportsomeof the

resultsofourearlierworkthatstudiedtherevisionstothese

estimates.

171 Viewsexpressedarethoseoftheauthorsanddonotnecessarilyreflectthoseof

theBureauofEconomicAnalysis.

2 BEAproducesthreesuccessivecurrentquarterlyestimatesofeachquarter’s

economyinthethreemonthsfollowingtheendofthequarter;thesearelabeled

the‘advance’,‘preliminary’,and‘final’estimates.EachJuly,itrevisestheesti-

matesforthequartersofthethreepreviousyears;thesearelabeledthe‘first’,

‘second’,and‘third’annualrevisionestimates.Inaddition,abouteveryfiveyears,

itrevisesalloftheestimatestoincorporatetheestimatesfromquinquennial

input-outputtablesthatarebasedoneconomiccensuses.Inadditiontousingnew

andbettersourcedata,thesuccessiveestimatesincorporatedefinitionalchanges

thatadapttheaccountstoachangingeconomy,correcterrorsinsourcedata,and

makeimprovementsinmethodology.Also,revisionstoseasonalfactors,whichare

inherentlyunforecastable,occurwiththepassageoftimeaswellaswiththe

incorporationofnewdata.Morecompletediscussionsofthesourcesofrevisions

maybefoundinFixlerandGrimm(2002)andFixler,Grimm,andLee(2003).

3 ‘Neartrend’isdefinedasbeingwithinonestandarddeviationoftrend;thatis,real

GDPgrowthratesarebetween2.5and4.3percent.

4 Here,thelatestavailableestimatesarethosethatwereavailableatthetimethat

thestatisticsforthisstudywerecalculated;thatis,thosethatwereavailablein

thefallof2002.Thus,theydonotincludethecomprehensiverevisionofthe

NIPAsthatwasreleasedinDecember2003.


AmajorgoaloftheBureauofEconomicAnalysis(BEA)isto

provide a timely, comprehensive, and reliable description of

the condition of the U.S. economy. These estimates of the

economy are contained in the national income and product

accounts (NIPAs), which feature two summary measures,

gross domestic product (GDP) and gross domestic income

(GDI).Thereliabilityof theestimatesofeconomicactivity is

highly important to a correct understanding of what the

economyisdoingandtherebyaffectsbotheconomicpolicy-

makingandbusinessdecisions.

In general, reliability refers to the ability of successive vin-

tages of estimates to present a consistent picture of the

economyastheestimatesarerevisedtoincorporateincreas-

inglycomprehensiveandimprovedsourcedata.2Inmorethan

adozenstudies,overthreeandahalfdecades,BEAhasfound

thatearlyestimatesofcurrent-dollarGDP,realGDP,andGDI

andtheircomponentsarereliableandpresentausefulpicture

ofeconomicactivity.Inparticular,overthelasttwodecades,

theestimateshavesuccessivelyindicatedthedirectionofreal

GDP 97 percent of the time, have successfully indicated

whether real GDP was accelerating or decelerating about

three-fourthsofthetime,andsuccessfullyindicatedwhether

real GDP growth was high relative to trend about three-

fourthsofthetimeandwhether itwas lowrelativetotrend

abouttwo-thirdsofthetime.3

However, Fixler and Grimm (2002) found that although the

currentquarterlyestimatesofrealGDPsuccessfullyindicated

thecyclicalpeak inthefourmostrecentrecessionspriorto

1990, they incorrectly indicated positive growth in the third

quarterof1990,whichisnowestimatedtobethefirstquarter

ofdeclineinthe1990-91recession,andtendedtounderstate

the strength of recoveries in the quarters around cyclical

troughs.Inaddition,thelatestestimatesputthebeginningof

the2001recession inthefirstquarter, ratherthanthethird

quarteraswasindicatedbythecurrentquarterlyestimates.

Theexistenceofvintagesofestimatescancreateatimegap

betweenpolicymakersandbusinessanalystsandresearchers.

Policymakersandbusinessanalystsreacttotheearlyvintag-

esofestimatesofGDPanditscomponents,mostlythefirst,or

advance estimates, while researchers and modelers look at

latervintagesofdatathatincluderevisions.Aspointedoutby

FairandShiller(1990)thetimegapaffectstheevaluationof

forecasting models and forecasts. In evaluating them, care

mustbetakentoexcludedataandinformationthatwouldnot

havebeenavailableatthetimeoftheforecasts.

Twofrequently-usedmeasuresofthetypicalsizesofrevisions

aremeanrevisions,thenumericalaverage,andmeanabsolute

revisions,numericalaveragewithoutregardtosign.Inorder

toavoidproblemswithmeasuringrevisionstoeconomicmea-

suresthataretrendedovertime,therevisionsarecalculated

asratesofchange,inpercentageterms,andareexpressedat

annual rates. The latest available estimates are used as the

standardsforsizesofrevisions,andtherevisionsarecalcu-

latedas the latestestimates less thecurrentquarterlyesti-

mates.4

Revisions to GDPThispaperonlyreportssomeoftheresultsofasetofstudies

ofrevisionstotheNIPAs.Additionalfindingsanddiscussions

maybefoundinFixlerandGrimm(2002,2003).

ThefirsttwocolumnsofFigure1showthemeanabsoluterevi-

sions for current quarterly current-dollar and real GDP and

theirmajorcomponents for theperiod 1983-2000.ForGDP,

thereisamodestdecreasefromtheadvancetotheprelimi-

naryestimates,butnofurtherdecreasetothefinalestimates.

Thethreevintagesofthecurrent-dollarestimatesofGDPall

have mean absolute revisions of slightly more than 1.0 per-

centage point, and the mean absolute revisions to the real

estimatesareabout0.2percentagepoints larger. Incompa-

rison, the rate of growth of current-dollar GDP averaged

18


19

6.3percentfromthefirstquarterof1983tothefourthquar-

terof2000andrangedfrom0.2percentto14.2percent;the

rateofgrowthofrealGDPaveraged3.6percentandranged

from–3.2percentto9.8percent.

The revisions toestimatesof themajorcomponentsofcur-

rent-dollar and real GDP are similar to those of these sum-

mary measures. From the advance to the preliminary esti-

mates,themeanabsoluterevisionsdecreaseforall17ofthe

current-dollarcomponentsandfor14oftherealcomponents.

However, from the preliminary to the final estimates, the

meanabsoluterevisionsdecreaseforonlysixofthecurrent-

dollarandsixoftherealcomponents.Withtheexceptionof

personal consumption expenditures, the components’ mean

absolute revisions are considerably larger than the corre-

spondingonesforGDP.Themeanabsoluterevisionsforthe

componentsoffixedinvestmentarelargerthanthosefortotal

fixedinvestment.Incontrast,themeanabsoluterevisionsfor

state and local government expenditures are much smaller

thanthosefortotalgovernmentexpenditures.5

Becausechangeinprivateinventoriesisfrequentlynegative,

itisnotpossibletocalculatepercentchangesorpercentage

pointrevisionsmeasures.However,theeffectsofrevisionsto

changeinprivateinventoriescanbeapproximatedbycompar-

ing the revisions measures for the three current quarterly

estimatesofgrossprivatedomesticinvestment(GPDI),which

isthesumofchange inprivate inventoriesandfixed invest-

ment, with those for fixed investment. The mean absolute

revisionsforcurrent-dollarandrealGPDIaremorethandou-

blethoseforfixedinvestment,indicatingthatrevisionstothe

estimatesof inventoriescontributesubstantiallytorevisions

intheestimatesofGPDI.

ThethirdandfourthcolumnsofFigure1showthemeanrevi-

sionstocurrent-dollarandrealGDPandtheirmajorcompo-

nentsfortheperiod1983-2000.ThemeanrevisionsforGDP

aresmallandpositive, indicatinga tendencytowardupward

revisions.ThemeanrevisionsforGDP,however,arenotstatis-

ticallysignificant.6Inaddition,muchoftheupwardrevisions

areduetodefinitionalrevisionsthattendtoincreaseboththe

levelandrateofgrowthofGDPandaremade toadapt the

measureofGDPtoachangingeconomy.Forexample inthe

1999 comprehensive revision, the recognition of business

purchasesofsoftwareasinvestmentraisedtheaveragerate

of growth of current-dollar GDP by 0.17 percentage point in

theperiod1995-99.Thus,themeanrevisionsgenerallydonot

indicate errors, but are in large part the results of changes

madetoimprovetheNIPAestimates.AsindicatedinFigure2,

the1999comprehensiverevisionincreasedthelevelsofcur-

rent-dollarGDPbyamountsrangingfromaboutU.S.$20bil-

lionin1983tonearlyU.S.$250billionin1998.

Fortheperiod1983-2000,themeanrevisionsfortheprelimi-

naryandfinalestimatesofbothcurrent-dollarandrealGDP

are about 0.1 percentage point smaller than those for the

advanceestimates.Themeanrevisionsforpersonalconsump-

tionexpendituresanditscomponentsarealsopositive.With

theexceptionofnonresidentialstructures,themeanrevisions

forGPDIandfixed investmentarenegative.Withtheexcep-

tionofnonresidentialstructures,themeanrevisionsofmost

componentsofinvestmentarealsonegative.Themeanrevi-

sions forcurrent-dollarand realexportsare largeandposi-

tive, whereas the mean revisions for final current-dollar

importsandforallthreevintagesofrealimportsarenegative.

Themeanrevisionsfortotalgovernmentexpendituresandfor

mostofitscomponentsarepositive.However,themeanrevi-

sions for current-dollar non-defense expenditures are large

and negative, whereas the mean revisions for real non-

defenseexpendituresarelargeandpositive.

Animportantquestioniswhetherarevisiontotheestimates

ofcurrent-dollarGDP is likely tobefollowedbysimilarrevi-

sionsinsucceedingvintagesofestimates.Figure3showsthe

5 Therevisionstototalandfederalgovernmentexpendituresincludea1991revision

inthetreatmentofCommodityCreditCorporationpurchasesofagriculturalgoods

thatwasoffsetone-for-oneinchangeinfarmprivateinventories;thisrevisiondid

notaffecttheestimatesofGDP.

6 Itispossibletotestthestatisticalsignificanceofthemeanrevisionsforboth

current-dollarandrealGDP—forallthreecurrentquarterlyvintages—because

Jarque-Berastatisticsfailtorejectthenullhypothesisofnormalityoftherevi-

sions,withp-valuesrangingfrom.31to.71.Thet-teststatisticsforthemean

revisionsrangefrom.25to.35,indicatingthatallthemeanrevisionsarefarfrom

statisticallysignificant.



Figure1:MeanabsoluterevisionsandmeanrevisionstoquarterlychangesinGDPanditsmajorcomponents,

latestestimateslesscurrentquarterlyestimates,1983-2000[percentagepoints]

mean absolute revisions mean revisions

Current- Current- dollar Real dollar Real estimates estimates estimates estimates

Gross domestic product

Advance 1.10 1.28 0.48 0.46 Preliminary 1.05 1.21 0.32 0.36 Final 1.05 1.23 0.34 0.38

Personal consumption expenditures


Durable goods


Nondurable goods


Services


Gross private domestic investment

Advance 7.99 8.01 -0.81 -1.05 Preliminary 7.98 7.95 -0.48 -0.68 Final 7.91 7.75 -0.82 -1.17

Fixed investment

Advance 2.75 3.25 0.17 -0.48 Preliminary 2.54 3.15 -0.32 -0.80 Final 2.56 3.28 -0.50 -1.11

Nonresidential


Structures


Equipment and software [1]


Residential

Advance 4.64 4.66 -0.10 -0.45 Preliminary 4.45 4.64 -0.09 0.03 Final 4.53 4.55 -0.11 -0.15Change in private

inventories [2] --- --- --- ---

[1] Followingthe1999comprehensiverevisionoftheNIPAs,thelatestestimates

includecomputersoftware.

[2] Negativevaluesinsomequartersmakethecalculationofpercentagechanges

impossible.

[3] Followingthe1996comprehensiverevisionoftheNIPAs,theestimatesinclude

consumptionoffixedcapital.


Current- Current- dollar Real dollar Real estimates estimates estimates estimates

net exports of goods and services [2] --- --- --- ---

Exports


Imports

Advance 5.92 7.00 0.87 -0.35 Preliminary 4.75 6.41 0.12 -1.31 Final 4.82 6.56 -0.36 -1.67

Government consumption expenditures and gross investment [3]


Federal

Advance 5.84 6.64 0.21 0.30 Preliminary 6.07 6.64 -0.18 -0.04 Final 6.03 6.70 0.18 0.47

Defense

Advance 3.43 4.38 0.18 -0.30 Preliminary 3.25 3.81 0.17 -0.38 Final 3.28 3.86 0.21 -0.49

Nondefense

Advance 21.77 25.12 -4.35 6.19 Preliminary 22.35 25.32 -5.98 7.92 Final 21.76 24.82 -4.47 6.13

State and local


Addendum: Final sales


correlationsofeachvintageofrevisionswitheachsuccessive

vintageofrevisions.Forexample,theentryattheupperleft

showsacorrelationof0.26betweentheadvance-to-prelimi-

naryrevisionandthepreliminary-to-finalrevision.Generally,

thecorrelationsarequitesmallandnegative.Inparticular,all

ofthecorrelationsinvolvingthevintagesofannualrevisions

arenegative.

The correlations reflect several factors. One is that there is

nearlyanequalchancethatarevisionfromonevintagetothe

nextwillbeeitherupordown.For theestimatesofcurrent

dollarGDP,theshareofupwardrevisionsisonlyslightlymore

thanhalf formostsuccessivepairsofrevisions,suchasthe

advance to preliminary or third annual-to-final. Overall, the

shareofupwardrevisionsforallofthesuccessivevintagesis

54percent.Thecombinationof thisresultandthenegative

correlations shown in Figure 3 suggests that the downward

revisionsaretypicallylargerthantheupwardrevisions.

Althoughanupward(ordownward)revisionfromtheadvance

tothepreliminaryestimateofcurrent-dollarGDPismodestly

morelikelytobefollowedbyanotherupward(ordownward)

revision to the final estimate, this result does not hold for

otherpairsofvintagesofestimates.Beginningwiththepre-

liminary estimates and going through the third annual esti-

mates,only39percentofupwardordownwardrevisionsare

followedbyanotherrevisionofthesamesign.

Additionally,thesizesofmeanabsoluterevisionsforcurrent-

dollarGDPvaryfromonevintageofestimatestothenext.The

meanabsoluterevisionfromtheadvancetothepreliminary

estimatesis0.55percentagepoint,butfromthepreliminary

to final estimates it is 0.28 percentage point. For later vin-

tagesofrevisions,themeanabsoluterevisionsfromonevin-

tagetothesucceedingvintageareeachroughlythree-fourths

ofapercentagepoint.


21

Figure2:Revisionstocurrent-dollarGDPinthe1999comprehensiverevision

250

200

150

100

50

0

Vintage of subsequent revision

Vintage of revision Preliminary to Final to First annual to Second annual Third annual

final first annual second annual to third annual to latest [1]

Advancetopreliminary 0.26 -0.03 -0.08 -0.15 -0.09

Preliminarytofinal 0.09 -0.15 -0.05 -0.24

Finaltofirstannual -0.20 -0.15 -0.26

Firstannualtosecondannual -0.05 -0.10

Secondannualtothirdannual -0.21

[1]1983-95only.Latestestimatesbeginningwith1996arethirdannualestimates.

Figure3:Correlationsofdifferentvintagesofrevisionstocurrent-dollarGDPestimates.1983-98

Estimatesfor1999-2000willundergofurtherannualrevisions.




Advance [1] Preliminary Final Advance [1] Preliminary Final

Gross domestic income --- 1.21[2] 1.20 --- 0.26[2] 0.25

national income --- 1.54[2] 1.44 --- 0.33[2] 0.23

Compensationofemployees 1.18 1.19 1.18 0.28 0.26 0.22

Proprietors’incomewithinventoryvaluation

andcapitalconsumptionadjustments 10.35 10.66 10.26 -0.92 -0.96 -0.84

Nonfarm 5.70 5.62 5.74 -0.72 -0.65 -0.55

Rentalincomeofpersonswithcapital

consumptionadjustment[2] --- --- --- --- --- ---

Corporateprofitswithinventoryvaluation

andcapitalconsumptionadjustments --- 11.47[2] 11.62 --- 0.47[2] -1.04

Netinterest --- 7.35[2] 7.41 --- 1.31[2] 1.37

[1]NoadvanceestimatesaremadeforseveralcomponentsofGDIduetoalackofsuitablesourcedata.

[2] Nopreliminaryestimatesweremadeforthefourthquartersof1995through2000.

[3] Negativevaluesinsomequartersmakethecalculationofpercentchangesimpossible.

Figure4:Meanabsoluterevisionsandmeanrevisionstoquarterlychangesingrossdomesticincome,nationalincome,anditsmajorcomponents,latestestimateslesscurrent

quarterlyestimates,1983-2000[percentagepoints]


Thus,arevisionofanygivenvintagecontainsverylittleinfor-

mationaboutanysuccessivevintageofrevision.Thatis,revi-

sions do not have momentum. There is no quarter in the

sampleperiodforwhichallfivevintagesofrevisionstocur-

rent-dollarGDPareinthesamedirection(notshown).

Revisions to GDiFigure4showsthemeanandmeanabsoluterevisionsforGDI

and itsmajorcomponentsfortheperiod1983to2000.The

mean absolute revisions for GDI are somewhat larger than

thoseforcurrent-dollarGDP,andthemeanabsoluterevisions

fornationalincomeareevenlarger.Thelargermeanabsolute

revisionsfornationalincomereflectsubstantialandnotfully

offsetting mean absolute revisions for the components that

areaddedandsubtractedfromGDItoobtainnationalincome.

Among the major components of GDI and national income,

onlycompensationofemployeeshasmeanabsoluterevisions

similarinmagnitudetothoseformostmajorcomponentsof

GDP.Theothercomponentshavemuchlargermeanabsolute

revisions, primarily reflecting the very limited availability of

current quarterly source data. For the annual revision esti-

matesofthecomponents,thesecondannualestimatesincor-

poratethefinalrevisionsofsomeannual-frequencydata.The

largemeanabsoluterevisionsforproprietors’incomereflect

largerevisionstofarmproprietors’income;themeanabsolute

revisionsfornon-farmproprietors’incomeareonlyabouthalf

as large as those for total proprietors’ income. As with the

product-sideestimates,thereislittletendencyforreductions

inmeanabsoluterevisionswhenprogressingfromadvanceto

preliminarytofinalestimates.

MeanrevisionsforGDI,nationalincome,andtheirmajorcom-

ponentsaresimilarinsizetothoseforcurrent-dollarGDPand

itsmajorcomponents.Infact,themeanrevisionsforGDIand

national income are smaller than those for GDP. Thus, the

largermeanabsoluterevisionsforGDIandnationalincomedo

nottranslateintolargemeanrevisions.

conclusions

Theprincipalresultsofthisreviewofrevisionsareconsistent

withthoseofpreviousBEAstudiesofrevisions.Theestimates

of current-dollar and real GDP and of GDI are reliable; the

mean absolute revisions for their respective quarterly esti-

matesaresomewhatmorethan 1percentagepoint.Positive

mean revisions for these measures primarily reflect the

improvements in the coverage of economic activity; these

were introduced in comprehensive revisions in order adapt

theNIPAstoachangingeconomy.However,therearemodest

declinesinthemeanabsoluterevisionsforcurrent-dollarand

realGDP,GDI,andmostoftheirmajorcomponentsfromthe

advancetothepreliminaryestimates.Correlationsofsucces-

sivevintagesofrevisionstocurrent-dollarGDParegenerally

verysmall;thus,revisionsdonothavemomentum.

References• Fair,R.C.andR.J.Shiller,1990,“ComparingInformationinForecastsfrom

EconometricModels,”TheAmericanEconomicReview,80:3,375-390

• Fixler,D.J.andB.T.Grimm,2002,“ReliabilityofGDPandRelatedNIPAEstimates,”

SurveyofCurrentBusiness,82,9-27

• Fixler,D.J.andB.T.Grimm,2003.“Revisions,Rationality,andTurningPointsin

GDP,”Paperpresentedatthesession“TrackingtheTurningPointsintheEconomy,”

AmericanEconomicAssociationmeetings,Washington,D.C.January3-5,2003.

AvailableonBEA’swebsite,atwww.bea.gov/bea/working_papers.htm,WP2003-01

• Fixler,D.J.,B.T.Grimm,andA.E.Lee,2003,“TheEffectsofRevisionstoSeasonal

FactorsonRevisionstoSeasonallyAdjustedEstimates,”SurveyofCurrentBusiness,

83,43-50

23

Economic


Robert YorkSenior Economist, International Monetary Fund

Paul AtkinsonDeputy Director, the Directorate for Science,

Technology and Industry, OECD1

Abstract

Nationalaccountsdataprovidethemostcomprehensiveover-

viewavailableofdevelopments innational economies.They

are of great interest to a wide range of users of economic

information.Theseusers,whichincludegovernmentsformu-

lating budgetary policies, central banks making monetary

policydecisions,businessesconsideringinvestmentdecisions,

andfinancialinstitutionsmakingjudgmentsconcerningport-

folio allocation have needs which may differ in various

respects.However, since their interestgenerally stems from

thelikelihoodthattheywillmakebetterdecisionsiftheyare

wellinformedabouteconomicdevelopments,theyallhavea

stronginterestintheaccuracyofnationalaccountsstatistics.

This paper examines the reliability of preliminary quarterly

nationalaccountsstatistics.Inparticular,itconsidersthelon-

ger-termbehavioroftheprovisionalestimatestoGDPgrowth

anditsmainexpenditurecomponentsthroughanexamination

oftherevisionstothoseestimates.Itcoversthesevenlargest

OECDcountriesand,assuch,updatesandextendsuponprevi-

ous OECD analysis on the topic. Overall, the results are

broadlysimilartotheearlierwork;thatpreliminaryestimates

foroutputgrowthhavenotbeenstatisticallybiased,although

theaveragesizeofrevisionshasbeenlargebutsmallerthan

thoseexhibitedbythedemandcomponentsofGDP.

251 OriginallypublishedbytheOECDinEnglishunderthetitle:‘TheReliabilityof

QuarterlyNationalAccountsin7MajorCountries’(WorkingPaper171,OECD/

GD/97/27)byPaulAtkinsonco-authoredwithRobertYork.OECDCopyright,1997.

ReprintedwithpermissionfromIOSPress,York,R.C.,andP.Atkinson,1997,‘The

reliabilityofquarterlynationalaccountsinsevenmajorcountries:user’s

perspective,’JournalofEconomicandSocialMeasurement,Volume23,nr.4,239-

262.Theauthorswouldliketoacknowledgenumerouscommentsandsuggestions

onearlierdraftsofthispaperfrommembersoftheEconomicsDepartmentand

StatisticsDirectorate,andthevaluabletechnicalassistanceprovidedbyIsabelle

Duong.

2 See,forexample,OECD(1979)whichoffersasurveyofpastliteratureacrosssev-

eralOECDcountries.MorerecentanalysisfortheUnitedKingdomcanbefoundin

CSO(1994).

3 Inventoryadjustmentsareomittedfromtheanalysisfortworeasons.Firstly,given

thefocusongrowthrates,theinherentinstabilityofchangesinstockswould

maketheirinterpretationdifficultandsecondly,itwasnotpossibleinanycaseto

constructaconsistentconstantpriceseriesfortheinventoryvariable.

4 Theaveragebiasisdefinedas1/nΣ(P-F);relativebiasasΣ(P-F)/ΣF;averagedisper-

sionas1/nΣ|P-F|;relativedispersionasΣ|P-F|/Σ|F|;andstandarddeviationasthe

squarerootof1/nΣ(d-d*)2,whered=P-Fandd*isthesamplemean.


requiresthattheexpectedaveragerevisionbezero,sothat

on balance initial projections are equivalent to outturns. (In

technicalterms,themeanrevisionshouldnotbesignificantly

differentfromzero.)Thenotionofforecastefficiencyassumes

thatstatisticalagenciesusealloftheinformationthatisavail-

abletothemwhenconstructingtheinitialestimates.Thus,in

order for revisions to embody forecast efficiency it follows

that they must be unrelated to information available at the

timethepreliminaryestimateismade.

The test for both unbiasedness and efficiency used in this

analysisisbasedontheworkofHoldenandPeel(1990)and

elaborated upon by Barrionuevo (1993). An ordinary least

squares(OLS)regressionoftherevisiononaconstantallows

asimplebutsufficienttestofunbiasedness.Thenullhypoth-

esisisthattheestimatedcoefficientfromsucharegression

isnotsignificantlydifferentfromzero.Ifitisnot,thesizeof

revisions could be systematically reduced by adjusting the

preliminaryestimatesbytheconstantcoefficient.Thecheck

ofefficiencyinvolvestestingthestatisticalsignificanceofthe

coefficients from an OLS regression of the revision on the

preliminary estimate (the so-called beta test) and from a

regression of the current period’s revision on the previous

period’srevision(theso-calledrhotest)5.Ifthebetaandrho

testsarejointlypassed(i.e.ifneitherbetanorrhoisstatisti-

cally different than zero), the advance estimates are pre-

sumedtobeefficient.Afailureofeitherorbothtestsimplies

inefficiency and that the variance of revisions could be

improvedupon.

Betaandrhocanprovidesomeindicationofthenatureofthe

inefficiencyofapreliminaryindicatorandapotentialmeans

of adjustment to improve them. Barrionuevo (1993) notes

that ifbeta iszerobut rho isnot, thevarianceof revisions

couldbereducedbyadjustingtheestimatebyrho,since in

thiscase,pasterrorsarebeingrepeated in thepresent. (In

otherwords,ifaprojectioniswidelyoffthemark,anon-zero

rhosuggeststhatsomeofthaterrorshouldbebroughtfor-

wardintonextperiod’sprojection,inthesamewayanerror-

correctionmodelaccountsforpastmistakes.)Ifontheother

hand,betaisdifferentfromzerobutrhoisnot,thevariance

of revisions could be improved upon by adjusting the

estimates by beta, since in this case the revisions contain

informationthatisnotbeingincorporatedinthepreliminary

estimate.

Thethirdpartoftheanalysisexploresthepossibilitythatthe

economiccycleand inflationmayeachhaveaseparateand

systematicinfluenceonthebehaviorofinitialindicatorsand

theirsubsequentrevisions.Thisismotivatedbyobservations

insomecountriesthatwhentheinitialestimateofgrowthis

high (low), more often than not, subsequent changes are in

the downward (upward) direction6. Research carried out by

theCentralStatisticalOffice(CSO)intheUnitedKingdomhas

similarlyuncoveredabiasinthepreliminaryindicatorslinked

to changes in the United Kingdom’s rate of inflation (CSO

1985). The experience in the United Kingdom is such that

whentherateofinflationisrisingquickly,theCSOfrequently

overstates the initial GDP figures and revises downward its

futureestimates.

Theexplanatorypoweroftheoutputgapand/ortherateof

inflation(GDPdeflator)indeterminingthesizeanddirection

ofupdatestoGDPgrowthisexaminedbyestimatingregres-

sionequations.Theoutputgap,definedhereasthedifference

between the preliminary estimate of quarterly GDP and the

trendlevel,wherethetrendisbasedonthefinalGDPseries7,

isused inthisstudytodefinethebusinesscycle,because it

explicitlydistinguishesbetweenquartersofaboveandbelow,

aswellasaverage,growth.Itisalsousefulasatestofthenull

hypothesissinceatanypoint intimepolicymakersandeco-

nomicforecastersoftenhaveafairlyclearnotionofwherean

economyisoperatingrelativetoitstrendorpotentialoutput.

Intheevent,atendencyforprovisionalestimatesofoutputto

be over-estimated (under-estimated) when the economy is

operating above (below) its trend level should be identified

by a negative (positive) correlation between the output-gap

variableandrevisions.

26


27

5 ThebetatestisconductedbyanOLSregressionoftherevisiononaconstantand

thepreliminaryestimate,whiletherhotestinvolvesaregressionofthecurrent

period’srevisiononaconstantandthepreviousperiod’srevisions.

6 Theremaybesomebiashere,sinceonlystudiescarriedoutbytheCentral

StatisticalOfficeoftheUnitedKingdomhaveverifiedsucharelationshipempiri-

cally.Studiesdoneforothercountrieshavebasedthesetypesofconclusionson

casualobservationsofparticularquarterswithhigh(orlow)growth.

7 TheoutputgapisexpressedasapercentageoftrendGDP,wheretrendGDPis

estimatedfromatimetrendthroughthefinalseriesoftheGDPfiguresforeachof

thesevencountries.

For many of the users of economic information, concerns

abouttimelinessmakethepreliminarynationalaccountsdata,

which in most OECD countries become available on a quar-

terlybasis,ofparticularinterest.Indeed,itisnotunusualfor

thesedatatohaveanimmediateimpactonfinancialmarkets

andtoinfluencemacroeconomicpolicydebate.Thefocusof

thispaper,therefore,isontheaccuracyofpreliminaryquar-

terlynationalaccountsstatistics.Inparticular,itconsidersthe

longer-term behavior of the provisional estimates to GDP

growth and its main expenditure components through an

examinationoftherevisionstothoseestimates.Itcoversthe

seven largest OECD countries and, as such, updates and

extendstheanalysisconductedinpreviousOECDworkonthe

topic2.Overall,theresultsarebroadlysimilartothefindings

ofpreviousOECDwork:

■ Preliminaryestimatesforoutputgrowthhavenotbeen

statisticallybiased,buttheaveragesizeofrevisionshas

beenlarge,insomecasesexceedingtheaveragegrowth

rate.

■ Thepictureforrevisionstopreliminaryinflationestimates

isbroadlysimilar.

■ Revisionstothedemandcomponentssimilarlyimplyno

statisticalbiastopreliminaryestimates,buttheyaregen-

erallylargerthanthosefortotalGDPgrowth—especially

forexportsandimportsofgoodsandservices.

■ Thebehaviorofrevisionsdoesnotsuggestthatprelimi-

naryestimatesaregettingbetterorworseovertime.

Theremainderofthispaperproceedsasfollows.Thenextsec-

tionsetsouttheapproachtakeninthispaperandthecriteria

forjudgingthequalityofthequarterlynationalaccounts,and

the following section summarizes the results of the assess-

ment.Thefinalsectionmakessomeconcludingobservations

fromauser’sperspectiveontheissueoftimelinessandaccu-

racyofnationalaccountsdataandtheprospectsforimprov-

ingthequalityofthisdata.

methodology: Judging accuracy of national accounts dataThisstudyfollowsthemethodologyusedbymanystatistical

agencies to assess the quality of national accounts data. It

focuses on revisions to the ‘preliminary’ (or the first pub-

lished) estimates of the quarter-on-quarter growth of real

GDP,itsmaincomponents(privateandgovernmentconsump-

tion, investment,andexportsand imports)andtheirassoci-

atedpricedeflators3.Growthrates,ratherthanvariablelevels

areexaminedinordertominimizetheimpactontheresultsof

conceptual changes in the data and changes in base years

usedtoconstructtheconstantpriceestimates;andbecause

growth rates are of more direct interest to the main user

groups of the statistics. Revisions are defined as the differ-

encebetweenthepreliminaryestimates(P)ofeachvariable

andthe‘final’estimates(F).Historicaldataforthepreliminary

estimatescomefromtheOECD’sQuarterlyNationalAccounts

publication, beginning with issue Number 4, 1979. Historical

dataforthefinalestimatesarebasedonthosestatisticspub-

lishedbytheOECDinQuarterlyNationalAccounts.

The analysis consists of three parts. The first involves the

calculationofseveralsummarymeasuresofthesizeanddis-

tributionofrevisionsandreplicatestheanalysisconductedin

previousOECDstudiesonthetopic.Thesummarymeasures

include:averageandrelativebias;averageandrelativedisper-

sion;thestandarddeviation;andthefrequencyofpositiveand

negativeerrors4.

ThesecondpartexpandsonpreviousOECDworkbyevaluat-

ingthestatisticalpropertiesofprovisionalestimatesandrevi-

sions in terms of unbiasedness and efficiency. A provisional

estimateisconsideredtobeaccurateifitisstatisticallyunbi-

asedandefficientinthesenseofMuth’s(1961)rationalexpec-

tationsframework.Randomadjustmentstotheinitialgrowth

rates of GDP and its components can and are expected to

occurforanygivenquarterandasaconsequencetherevision

overanygivenperiodmaybenon-zero.However,abstracting

from these random elements, the notion of unbiasedness


than it needs to be and that it might be reduced, perhaps

through better interpretation and more effective use of the

availableinformationset.Thefrequencywithwhichestimates

ofbetaandrhoarestatisticallydifferentfromzerosuggests

that there is unexploited information contained in the data

whichcouldbeusedtosmooththevariabilityoftheprelimi-

nary figures. Such information, obtained from estimated

equationsfromthebeta-andrho-testsofefficiency,isused

todemonstratethesepotential improvements inthecaseof

governmentconsumption.Wefindthatthestandarddeviation

ofanadjustedseriescanbeasmuchas50percentlowerthan

theactual(unadjusted)series8.

The stable behavior of revisions over time

Sincethemethodologyusedinthisstudycloselyfollowsthat

usedinpreviousOECDwork(OECD1979), itallowsforsome

general comparisons to be drawn with the earlier work.

However,thefindingsshouldnotbedirectlycompared,given

differencesinthewayrevisionsarespecificallydefined.Inthe

1979OECDstudy,forexample,revisionsarebasedonthedif-

ferencebetweenthe initialestimateof thevariable inques-

tionwiththefinalfigureobtainedone-yearlater.Here,onthe

other hand, final outturns are derived from the historical

seriespublishedinthe2ndquarterof1994.Asaconsequence,

thepresentstudymayexplorerevisionsofgreatermagnitude

andvariance.Itshouldbenoted,however,thattotheextent

thatthisisthecase,thelargerrevisionsarepresumablybet-

termeasuresoftheaccuracyofthepreliminaryestimates.

Incomparingthefindingsofthe1979studywiththosereport-

edhere,severalobservationscanbemadewhichpointtothe

consistent quality of the quarterly national accounts over a

longperiodof time.First,outputgrowth in themajorcoun-

tries covered in both studies is, on average, slightly under-

estimated in most cases, with the size and distribution of

revisions falling within a similar order of magnitude in both

studies. Second, the largest revisions (and widest variance)

continue to occur in the trade statistics. Like the present

study, the earlier analysis reported average dispersions to

revisions for both exports and imports which are 2-3 times

largerthanthosemadetopreliminaryprojectionsofnational

incomegrowth.Third,thisenquiryhasconfirmedoneofthe

findingsofearlierwork,namelythatrelativetofinaloutturns,

revisionstogovernmentconsumptiontendtobethelargest

ofthenationalexpenditurecomponents.Thiswouldappearto

suggestthatwidemarginsofuncertaintyaboutthewaygov-

ernmentscarryouttheirownexpenditureplanspersist.

Finally,thesampleusedinthepresentstudywassplitintotwo

periods (roughly 1980-86and 1987-93)andsummarystatis-

tics compared. (The sample split reflects the simplistic

assumptionthatchangesinqualitymightbevisibleasstatisti-

calmethodologies,techniques,andinformationsetsimprove

overtime).Acrossthegroupofcountriesandbetweenperi-

ods,theperformanceofpreliminaryindicatorsofGDPgrowth,

its components, and their associated price deflators was

mixed:someremainedlargerinthelaterperiodwhileothers

werereduced.Itcouldnot,therefore,beconcludedonewayor

theotherbythisinquiryifthequalityofthequarterlynation-

al accounts statistics either improved, deteriorated, or

remainedthesameovertheperiod.Asimilarconclusionwas

reachedinthetwoearlierstudies.

The influence of the economic cycle

Regression results using revisions to the quarterly growth

rateofGDPasthedependentvariableandtheoutputgapas

theexplanatoryvariableyieldnosignificantrelationshipsfor

sixoutofthesevencountriesunderstudy.Thereappearsto

be no systematic relationship between the magnitude and

directionofrevisionsandthepathoftheeconomyasitmoves

aboveorbelowthetrend-determinedlevelofoutput.

Some experimentation with a variation of this relationship,

however,doesrevealapositivebutstatisticallyweakinfluence

(adjustedR2of35percent)betweenrevisionsandchangesin

theoutputgapmeasure9 for theUnitedStates,Japan,and

theUnitedKingdom10.IntheUnitedKingdom,wherethesta-

tisticalrelationshipisthestrongestandexplains75percentof



summary of the main resultsAccuracy: summary statistics, bias, and efficiency

Overall, the results obtained in the present analysis are in

accordance with the findings of previous OECD studies

addressingthenatureandscopeofrevisionstothequarterly

national accounts of the seven major OECD countries. The

preliminary estimates of GDP growth for six of the seven

major countries have, on average, understated the final

growthfiguresovertheperiod1980-94.InthecaseofJapan,

theyhavebeenveryslightlyoverstated.Theseaverageerrors

(i.e.,theaveragebiases)aregenerallysmallandtheprelimi-

naryestimatesarestatisticallyunbiased(i.e.,notstatistically

differentfromzeroatthe95percentlevelofsignificance).

Whenjudgedagainstthemeasureofaveragedispersion(that

is, in terms of absolute revisions) the size of the ex-post

adjustmentsrisesappreciably,fromalowof0.29percentage

points(1.2percentatanannualrate)forFrancetoahighof

0.80percentagepoints(3.2percentatanannualrate)forthe

UnitedKingdom.Thesefiguresarequitesubstantial incom-

parisontoaveragegrowthratesand,inthecaseofGermany

and theUnitedKingdom,exceed them.Standarddeviations,

whichprovideanalternativeindicationofthesizeofabsolute

revisions,aretypicallylarger,rangingfromalowof0.37per-

centforFrance(1.4percentatanannualrate)toashighas

1.36percentfortheUnitedKingdom(5.4percentatanannual

rate).

With regard to the components of GDP average biases are

oftennegative,buttheyarenotstatisticallysignificantforany

component for any country. However, the preliminary esti-

matescontainwidemeasurementerrors,asreflectedinlarge

averagedispersionsandstandarddeviations,especiallywith

respecttoexportsandimportsofgoodsandservices.Across

the seven countries, the average dispersion for both these

componentsisaround1.6percent,or6.6percentatanannual

rate,withstandarddeviationsofaround2percent,morethan

8 percent at an annual rate, which is 2-3 times larger than

thoseforGDP.Ifthemagnitudesofadjustmentsarecompared

againstfinaloutcomes,i.e.intermsofrelativedispersion,the

largestchangesoccurtogovernmentconsumptionforsixof

thesevencountries,Francebeingtheexception.Themeasure

of relative dispersion for government consumption, which

indicatesthesizeoftherevisionasaproportionofthefinal

figure, is typicallyaroundunity.This impliesthatprovisional

quarterly estimates of the growth of government consump-

tionare,onaverage,100percenthigherorlowerthanthefinal

figures.

FortheGDP-basedmeasureofinflation,sixoutoftheseven

countriesonaveragealsorequireupwardrevisionstotheini-

tialquarterlyestimates,butonlyforItalyisthisbiasstatisti-

callysignificantordoesitamounttomorethan0.12percent,

around 0.5 percent at an annual rate. The exception is

Germany, where the average bias is zero. Indicators of the

absolutesizeofrevisions,i.e.theabsolutedispersionandthe

standarddeviation,aresimilartothoseforvolumes.Forthe

otherpricedeflators,nogeneralpatternemergesacrossthe

group of countries, although implicit prices for exports and

importstendtoneedthewidestmarginofadjustment.Except

forItaly’sgovernmentconsumptionandCanada’sinvestment

deflators,preliminaryestimatesofthecomponent-pricedefla-

torsarestatisticallyunbiased.

Overall,thereappearstobelittleadvantagetobegainedby

makingapriorijudgmentsconcerningthedirectionoffuture

revisionstotheadvancedestimatesofoutputgrowthandits

maincomponents.Withveryfewexceptions,revisionstothe

initial estimatesare statisticallyunbiasedand futureadjust-

mentsare largely random,somakingsuch judgmentscould

leadtounnecessaryerrorsinassessmentwithnonetbenefits.

However,sincetheabsolutesizeoftherevisionsthatcanrea-

sonablybeexpected is fairly large, therangeofuncertainty

thatsurroundstheearlynationalaccountsdataissignificant

andtheymustalwaysbeinterpretedcautiously.

Inmostcases,preliminarynationalaccountsestimatesarenot

efficient.This impliesthatthevarianceofrevisions is larger

29

8 Thenatureoftheadjustmentistousetheex-postinformationprovidedbythe

betaandrho-tests.Thegovernmentconsumptionvariableistransformedby

adjustingtheoriginalseriesbytheestimatedcoefficientsfromtheOLSregression

equationsexplainedabove.

9 Changesintheoutputgapreferspecificallytothefirstdifferenceoftheoutput-

gapvariable.

10 ThebiasuncoveredasaresultofthephaseofthecyclefortheUnitedKingdom

dataisconsistentwithpreviousworkcarriedoutbytheUnitedKingdom,Central

StatisticalOffice(CSO,1985).Inthiswork,however,theCSOdidnotexaminethe

relationshipbetweenrevisionsandtherateofchangeofquarterlygrowth.


informationabouttheeconomicsituationwhichcontributes

directlytopolicyformulation.Whereforecastsarerequired,

the latestnationalaccountsdataalmost invariablyserveas

the starting point and, indeed, the accuracy of forecasts is

highly dependent on their accuracy13. Furthermore they

influencebusinessand,especially,financialmarketbehavior

and,moregenerally,publicperceptionsabouttheeconomy,

whichinturnaffectpublicdebate.Theycan,therefore,affect

thefuturecourseofawiderangeofpolicies.Consequently,

timelyandaccuratenationalaccountsdatawouldbehighly

desirable.

The contribution that accurate national accounts data can

make to an understanding of the economic situation goes

beyondtheinformationcontainedinthetotalGDPfigures.In

particular, the expenditure-based accounting framework

whichprovidesadecompositionofGDPintothemaindemand

components lends itself well to examining developments in

different parts of the economy (i.e., households, business,

government, the foreignsector)and toanalyzing the forces

operatingontheeconomy.Gooddataondemandcomponents

from expenditure-based national accounts estimates are

therefore very helpful both for policy formulation and for

communicatingpublicpolicydecisions.Whiletheincomeand,

toalesserextent,production-basedapproachestomeasuring

GDP also produce useful decompositions, for many users

theseareofmorelimitedvaluethanthemaindemandcompo-

nentswhichemergefromtheexpenditureapproach.

Unfortunately,whilethelackofbiasinpreliminaryestimates

is reassuring, the large average dispersions and standard

deviationsoftherevisionsimplythatthesignal-to-noiseratio

inthesedataislowerthandesirable.Thepoorqualityofthe

demandcomponentestimatesisaseriousdefect.Inaddition,

therearesomegroundsforbelievingthatthesignal-to-noise

ratiointhepreliminarydatainanumberofcountriesispos-

siblyevenlowerthantheresultsreportedabovewouldsug-

gest.Thecomparativelylargesizeofrevisionswhichtypically

occur to national accounts data in the United Kingdom is

somewhatsurprisinginviewofthecomparativehighregard

withwhichmostusersregardBritishstatistics.Onepossible

explanationforthis isthattheCSOisdoingsomethingseri-

ouslywrongandthatithasmuchtolearnfromotherstatisti-

cal agencies. An alternative explanation, to which consider-

ableweightmustbeattached,isthatbyinvestingmoretime

and effort into revising the accounts the CSO eventually,

albeitoveraperiodofseveralyears,providesmoreaccurate

statistics than other statistical agencies. To the extent that

thisisthecase,theapparentlygreateraccuracyofprovisional

dataelsewhere isamiragewhich just reflects lessaccurate

finaldata—indeed,revisionscouldbereducedtozerobythe

simple device of never going beyond the preliminary esti-

mates.

An important feature affecting the usefulness of national

accountsdatatoeconomicpolicymakersandforecastersisits

timeliness.TheUnitedStates’statisticalagencyisthequick-

est in publishing provisional figures, just four weeks after

quarter’s-end, while Italian statisticians require fourteen.

Agencies in the other five countries take on average, 8-10

weekstoreleaseinitialestimates.Themostnotabledifference

amongcountrieswithregardstotimelinessisthetimetaken

by France, Italy, and the United Kingdom in publishing the

‘final’ quarterly estimates for a given year. For example, for

1993, France and the United Kingdom published final esti-

mates in 1995,while in Italy, theydidnotappearuntil 1996.

Meanwhile,theotherfourmajorcountriessucceedinfinaliz-

ingthequarterlyfiguresinonly6-9months.Thesefinalrevi-

sions are then, of course, often revised sporadically in later

yearsasfurthernewinformationbecomesavailable.

Forthepurposesofcontributingtotheunderstandingofthe

currenteconomicsituationandprospects,mostofthesepro-

visionalreleasedatesareneartheouterlimitofwhatisuse-

ful,particularlyintheItaliancase.Sincethenationalaccounts

variablesofinterestareflows,thegrowthrateforanyparticu-

lar quarter roughly measures the middle of the quarter in

comparisonwiththemiddleofthepreviousquarter.Thus,a


11 Thefirstdifferenceoftheinflationrate.

12 InthecaseofbothGermanyandCanada,theinfluenceofinflationonlyexplains

about10percentoftheoverallrevisions.


thehistoricalrevisions,asmallnegativerelationshipwiththe

levelofoutputrelativetotrendalsoemerges.Inthesethree

countries,increasesintherateofeconomicgrowthawayfrom

trend result in some tendency for preliminary estimates of

quarter-on-quarter GDP growth to be under-estimated, by

around 0.2, 0.4, and 0.9 percentage points, respectively. In

other words, when their economies are expanding (or con-

tracting)quickly,theregressionresultssuggestthatupdates

to advanced indicators are more likely to be in the upward

(downward)direction.

The influence of inflation

Sincemanyofthehistoricaldownwardrevisionstorealoutput

growthhaveoccurredinyearsinwhichinflationwashighand

rising—forexample,intheyearsafterthefirstoilpriceshock

—ithasbeensuggestedthatthesemovementsmaybeinflu-

encedbychangesinprices.Arelationshipofthisnaturehas

beenidentifiedinpreviousempiricalworkundertakenbythe

CentralStatisticalOffice(CSO)intheUnitedKingdom,which

uncoveredalikelihoodthatadvanceestimatesofgrowthare

overestimatedwheninflationishighandrisingandthatthey

are under-reported when inflation is decelerating [CSO

(1985)].

RegressionsofrevisionstoquarterlyrealGDPgrowthonthe

rateofinflation(GDPdeflator)andonchangesintherateof

inflation11 in the present analysis confirm a statistically sig-

nificant, but weak relationship in the data for the United

States,Germany,andCanada12.FortheGermanandCanadian

data,increasesinthepricelevelappeartobeassociatedwith

anupwardbiasintheadvancedestimatesofoutputgrowth,

byaround0.4and0.2percentagepointsrespectively.Incon-

trast, fortheUnitedStates,downwardadjustmentstoprovi-

sional figures, of the order of 0.4 percentage points, seem

necessarywhenevertherateofinflationisaccelerating.Inthe

caseoftheUnitedKingdom,previousresultsontheeffectsof

inflationwerenotcorroboratedhere.Thisapparentinconsis-

tencymaybeexplainedbythedifferentsampleperiodsused

—theCSO’sdatawasdrawnfromthehigh-inflationyearsof

the1970swhilethepresentinvestigationdrawsfromthecom-

paratively low-inflationyearsofthe1980s—andpossiblyby

thechanginginfluenceofoilintheUKeconomy.

concluding remarks from a user’s perspectiveFrom the perspective of users whose concerns are largely

abouteconomicpolicy, the foregoingresultsaredisappoint-

ing.Theanalysisofcurrenteconomicdevelopmentsandpros-

pectsisessentialformacroeconomicpolicyanalysisandfor-

mulation and also provides an important element of the

overallcontextinwhichstructuralpoliciesmustbeanalyzed

andformulated.Indeed,thereareatleasttwoareasinwhich

it iscrucialtopolicymakers ineconomicsandfinanceminis-

triesandcentralbanks.

■ Monetarypolicydecisionsmustbebasedonperceptions

ofhowtheeconomyisevolving,whatforcesareoperating

onit,andhowitwillevolveonthebasisofalternative

monetarypolicydecisions.

■ Largepartsofgovernmentspendingandrevenuesare

tiedtotheevolutionofimportantmacroeconomic

variables.Itisdifficulttosee,forexample,howamodern

budgetcouldsensiblybeproducedinisolationfromawell-

definedviewofhowtheeconomyisevolvingand,indeed,

aformalsetofmacroeconomicprojectionsisalmosta

necessity.

More generally, the analysis of current economic develop-

mentsandprospectsprovidesapremiseformuchofthestory

tellingwhichgovernmentsmustdoastheycontinuouslycom-

municate,explain,defend,andselltheirpolicies,bothmacro-

economicandstructural,tothegeneralpublic.

The comprehensiveness of national accounts data, which

makes themnotmere indicatorsbutvirtually thedefinitive

statement of how the economic situation has evolved, is

almostuniqueamongthevarioussourcesofeconomicinfor-

mation.Wheretheybecomeavailablesufficientlyearlytobe

ofcurrent, rather thanhistorical, interest theyprovidenew

31

13 Ameetingamongbusinessandtradeunionexpertson‘ImprovementofEconomic

Forecasts’concludedthatdatainaccuracieswereasignificantcauseofforecasting

problems.Therapporteurnoted:‘Itwasfeltthatanaccurateup-to-datepictureof

whatwashappeningintheeconomywouldhavehelpedsubstantiallyinmaking

forecastsinanumberofcountries.’[Mansley(1995)].Whilearecentanalysisby

theOECDSecretariatonforecastingaccuracyemployedaslightlydifferentmeth-

odologyandcoveredadifferentperiodthandoestheMansleystudy,itsresults

wouldsuggestthatforecastingaccuracycomparesreasonablywellwiththatof

thenationalaccountsdatathatprovidethestartingpointfortheseforecasts.See

OECD(1993),especially,thesummarystatisticsreportedinTables16-19.


secondquartergrowthrateroughlycomparesthepositionof

theeconomy inmid-Maywith itsposition inmid-February,a

periodwhosemiddlewastheendofMarch.Iftheinitialesti-

mate becomes available in September, i.e., 9-10 weeks after

theendofthesecondquarter,itisalreadyratherdated.For

manypurposes,notablymonetarypolicydecisionmaking, it

wouldhavetobediscountedinfavorofmorerecentinforma-

tion, however sketchy, even if its accuracy were completely

reliable.Thuswhileholdingoffonreleasingdatauntilalater

stage in the information collection process could reduce or

eveneliminatetheneedforfuturerevisions,andwouldhave

thebenefitofreducingtheextenttowhichmisleadinginfor-

mationissometimesplacedinthepublicdomain,thedeterio-

rationintimelinesscouldnegatetheirusefulnessintermsof

providing information about the current economic situation

altogether.

Clearly, goodnationalaccountsdata, especiallywhere they

are expenditure based, are highly desirable, and improve-

mentswouldbewelcome.However, theyarealwayssupple-

mented by other sources of information — financial market

developments,lesscomprehensivedatacoveringareassuch

asunemploymentandtrade,surveydata,anecdotalinforma-

tion, etc. — and users of any type of data must always be

aware of their limitations and discount them accordingly.

Furthermore, ifnationalaccountsdatadidnotexist central

bankswouldstillconductmonetarypolicy,financeministries

wouldstillpreparebudgets,andgovernmentswouldstillhave

totellastoryabouttheeconomicsituationanddefendtheir

policiesagainstthebackgroundofcurrenteconomicdevelop-

ments and prospects. The fact they are costly to produce

meansthatimprovingtheirqualitywillinvolvetrade-offs,and

thesetrade-offsmustbeconsideredcarefully.

Theaboveconsiderationssuggestthatposingthequestionof

trade-offssimplyintermsoftimelinessversusaccuracyisnot

helpful.Thevalueofmoretimelynationalaccountsdatawill

belimitediftheyareattheexpenseofaccuracy,whileusers

alreadyhavethealternativeofwaitingfor improveddatato

become available if timeliness is not a major concern. The

practicalissuesarewhetherthereisscopeforimprovements

ineithertimelinessoraccuracywithoutsacrificingtheother,

orinbothatthesametime,andwhethertheextentofsuch

improvementswouldjustifythecosts.Inthecurrentbudget-

aryclimatefewresourceincreasesarelikelytobeforthcom-

ing for the purpose of improving data collection. If more

resourcesare tobedevoted to improvingnationalaccounts

data,theywilllargelyhavetobedrawnfromtheproductionof

othertypesofstatistics.Thekeytrade-offs,therefore,involve

theimprovementsthatcouldalternativelybemadetoother

typesofstatisticsorthesavingwhichcouldbemadebydrop-

pingstatisticsthatarecurrentlyproducedbutwhoseuseful-

ness ismarginal.This inturnpointstotheneedtoevaluate

thescopeforimprovingthequalityofnationalaccountsdata

inthecontextofabroaderprioritizationandcostevaluation

ofallstatisticalproductionactivities.

References• Barrionuevo,JoseM.,1993,“HowAccurateareWorldEconomicOutlook

Projections,”StaffStudiesfortheWorldEconomicOutlook,Washington

• CentralStatisticalOffice,1985,“RevisionstoQuarterlyEstimatesofGrossDomestic

Product,”EconomicTrends,No.381,London

• CentralStatisticalOffice,1994,“TestingforBiasinInitialEstimatesofthe

ComponentsofGDP,”EconomicTrends,No.489,London

• Holden,K.andD.A.Peel,1990,“OnTestingforUnbiasednessandEfficiencyof

Forecasts,”ManchesterSchoolofEconomicandSocialStudies,Vol.58

• Mansley,N.,1995,“ImprovementofEconomicForecasts,ReportonaJointMeeting

ofManagementandTradeUnionExpertsheldundertheOECDLabourManagement

Programme,”[OECD/GD(95)39],Paris

• Muth,J.F.,1961,“RationalExpectationsandtheTheoryofPriceMovements,”

Econometrica,29,315-35

• OECD,1979,QuarterlyNationalAccounts:AReportonSourcesandMethodsin

OECDCountries,Paris

• OECD,1993,“HowAccurateareEconomicOutlookProjections,”pp.49-54,inOECD

EconomicOutlook53,June


Economic

The effect of telecom density data on growth, efficiencies, and distributions in global economieslall Ramrattan

Lecturer, university of California, Berkeley

Frank DimeglioConsultant, Macrosoft Company

michael szenbergDistinguished Professor of Economics,

Pace university

Abstract

Thispaperpresentsdataanalysisfortheglobaltelecommuni-

cation industry from the modern points of view of growth,

efficiency,anddistribution.Itdemonstratesadirecteffectof

telecomvariablesonGDPpercapita,whichhasbeeninitiated

intheliterature,butwasappliedonlytotheteledensitydata.

Efficiency is a concern for the use of scarce resources as a

means for growth, whether for developing or developed

nations.Weused thenewStochasticFrontier technology to

assess how countries are measuring up to production effi-

ciency standard. As for distribution, we appraise how the

countriesinthesamplearefaringintermsofthedistribution

of scarce telecom capital. Our significant statistical results

points inthedirectionthattelecomdatadoesaffectgrowth

directly, thatefficiencyontheproductionsidehasroomfor

improvementacrossbothdevelopinganddevelopednations,

and that while the gap in inequality is closing for the tele-

phone density, it is widening for the Internet and cellular

densities.

33


Somevirtuesofthismodernviewoftelecomcapitalmustbe

extolledinorderforreaderstoappreciatetheimportanceof

theGEDdataanalysisweendeavor tobring forth.Oneper-

spectiveisthatitincreasescommunicationbybringinginter-

nationalbuyersandsellerstogetherandthusenhancesgains

fromtradeamongnations.Forthedevelopedcountriesthat

areinthecorridorofequilibriumincomeandgrowth,thenew

technologyallowsthemtoreducetransactioncosts,thereby

enhancingconsumersurplusforbothproducersandconsum-

ers[MaddenandSavage(2000)].

Although there are barriers-to-entry that the cost for this

technologycreates,thetechnologyisrapidlybeingdiffused

intheglobaleconomyperhapsfromthefearpropensitythat

no country in the global economy wants to be left behind.

As with any technology, the initial R&D cost is prohibitive.

We learn that in the 1990s, assuming that ‘…the cost of

installingatelephonelineisequalto2000-2500dollars,then

1.5billiontelephoneswouldrequire3000to3750billiondol-

lars,’ which is prohibitive for many countries in our sample

[Kudriavtzev and Varakin (1990)]. But as the technology

becomesmoreaffordable,andasbetterandcreativefinanc-

ing through foreign direct and indirect investments are

secured the diffusion will occur at a more enhanced pace.

Therearemanyarticlesthatareconcernedwiththestudyof

countriesconvergingtotheiroptimalleveloftelecomdensity

infrastructure. Findlay (1996) notes that we have a race

betweentwotrainsandthatthedevelopingcountries’(LDC)

trainshavebeencatchingup.

Anotherperspectiveisthattradeamongnationsisbecoming

lesscostly,andthereforeprofitsareconverging,andtradeis

moving towards its long-run equilibrium level. In that case,

countries, both buyers and suppliers, have to seek creative

waysofincreasingconsumerandproducersurplusesinorder

to be more efficient. This analysis is subsumed under the

notion of transaction costs. For instance, ‘…when personal

computerswere firstdeveloped, theywere relativelyexpen-

sivetoproduce.Overtime,companieslearnedtoreducethese

production costs. They have developed computer links that

reducedtheircostoftransactingwithconsumers’[Brinkleyet

al.(2004)].

Thetreatmentweofferintherestofthispaperistoinclude

telecomdensitydatafromtheGEDpointofview.Thelogical

step we take is to show how the data is distributed across

countries.Growthentersfromthepointofviewofproduction

functionspecification.Finally,wewillintroducetheconcepts

ofgrowthandefficiencyfromthemodernStochasticFrontier

production point of view. This study does not exhaust the

amountofdataanalysisthatcanbedoneonthetelecomdata

series. For instance, we could have analyzed the stationary

propertiesinco-integrationanderrorcorrectionmodelofthe

telecom data. However, looking at the GED characteristics

givesitsufficientlinkwithmodernliterature.

Distribution aspect of the dataWebeginbylookingattherecentdispersionofmoderntele-

com variables. Table 1 displays the quintile distribution, fol-

lowedbytheirdefinitionsanddiscussions.

Table1entailsamixedstory.TelephoneandInternetdensities

havebeenproceedingatamoreequitablerate,whilewireless

andcabledensitieshavebeenproceedingatanunequalrate.

TheGiniratiosubstantiatesthistrend.Itcanrangefrom0to

1,where1indicatesperfectinequality,meaningthatonecoun-

tryinoursamplehasallthedensity,and0indicatesperfect

equality,meaningthateachcountryinoursamplehasequal

sharesofdensity.Telephonedensitydefinedasthenumberof

maintelephonelinesper100populationhasbeenthemajor

technology variable driving growth in the global economy

[Maddenetal. (2000)]. It is interestingthatourdatashows

thattheGiniforthisserieshasbeengoingdownintheworld,

from0.468in2000to0.400in2001,to0.348in2002,the

mostcurrentthreeyearsforwhichdataisavailable.Theimpli-

cation of this is stunning as a textbook notes that ‘…skill-

biasedtechnologicalchangeexplainsmostoftheincreasein

wage inequality in theUnitedStates’ [Borjas (2005)]. In the

samevein,Internetdensitydefinedasthenumberofinternet

users per 100 population also shows decreasing Gini



35

The telecommunication industry includes the movement of

voice, video, and data through local exchange, wireless ser-

vice, satellite broadcast, fiber optics, copper wire, undersea

andcoaxialcable,theInternet,microwave,networkslongdis-

tanceservice,andvideoconferencing[Plunkett(2002)].This

suggests a transfer of information over distances. A recent

survey notes that ‘In the realm of telecommunications, one

valuelinksmanycountriesaroundtheworld:therecognition

thatcompetitionbringswithitprospectsforeconomicgrowth

andimprovedservicesathome,andforgreaterparticipation

in the global economy’ [PricewaterhouseCoopers Industry

Study(1998)preface].Thepurposeofthispaperistoanalyze

andstudyrecentdataseriesforthetelecommunicationindus-

tryfromthestandpointofgrowth,efficiency,anddistribution

(GED),whichareimportantfactorsfortheeconomicstateof

countriesinthemodernglobaleconomy.

Themodernliteratureleanstowardthehypothesisthatden-

sitymeasuresofthevariousdataseriesareimportantinputs

forGED.Thenumberofdensitymeasuresisasvariedasthe

numberofmeasuresusedtoevaluatetheconditionofacoun-

try.Themostfamousexpressioninregardstoourinvestiga-

tionischaracterizedbytheJ-curvethatshows‘thewealthof

acountryw in termsof its telephonedensityorviceversa’

[Jipp(1963)].Allrecognize,includingJipp,thatthenumberof

telephonescannotbeusedasasolecriterionformeasuring

the‘wealthofacountry.’Inthemodernliterature,itistheper

capitaGDPwhichisthemostwidelyusedmeasure.Thelitera-

ture,however,doesnotaddressthequestionofGED,atleast

notinacomprehensivewayfromthedatapointofview.This

paperfillsthatgap.

ThetypeofinvestigationofGEDwemakeisgreatlyfacilitat-

edbythereworkingoftraditionalgrowthmodels.Withinthe

Keynesiantradition,theknife-edgegrowthmodel—Y=ce(s/v)

t,whereYisGDP,sisthesavingsratio,visthenaïveaccel-

erator,andtisthetime—hasemphasizedsavingsandinvest-

mentsasthesourceofgrowth.Todaythesourceofcapitalfor

manycountriesforinvestmentinthetelecominfrastructure

isstillapuzzlingquestion.Whiledevelopingnationsexplore

newinvestmentstoraisethetelephonepercapitaratio,other

countriescopewithupgradingexistingequipment.Itisfound

thatGDPincreasesbyaboutU.S.$2,000foreveryincreasein

telephone line per 100 population [Plunkett 2000)]. Then

there is the neoclassical steady-state model that compares

s/vwithn,thepopulation/laborforcegrowthrate,whichnot

onlyallowsthetechnologyeffecttobeestimatedasaresid-

ual,butalsoactsasalimitingmeasureofthetelecominfra-

structureper100population[Solow(1957)].Anextensionby

Mankiw,RomerandWeil(1992),allowstheinclusionoftele-

com variables alongside physical and human capital in the

independentvariableset.Thevirtueofthatextensionwasto

movetheresearchprogramthatassessestelecomvariables

on GDP away from the residual approach of technology

towardamoredirectapproach.Thisstephasbeenusedby

thehumancapital theoristssuchasBen-Porath (1967)who

wrotemodelsoftheformQ=β0(stKt)β1Dβ2,whereQisoutput,

sistheamountofhumancapitalusedfortheproductionof

humancapital,andDisotherpurchasedinputs.Thesemod-

elssuggestthatwecanapproachGEDfrombothproduction

and cost points of view, where production will have wealth

dependentoninputvariables,andcostwillhaveinvestment

in telecom infrastructuredependentonthealternativecost

ofusingcapitalfortelecominvestment,andotherdirectpur-

chases of materials for telecom investment. A Stochastic

Frontiermodel,atimerelatedmodelsubjecttorandomvaria-

tion,caneasilybeadaptedtothedataofthetelecomindus-

trytoanalyzeboththeproductionandcostaspects,inorder

toshowgrowthandefficiency.Thisinvestigationfitsmoreon

theproductionsidethanonthecostsidebecauseimproper

skewedness of the residual of the cost data still needs a

properspecificationtogeneratemeaningfulresults.Inorder

toestimatetheinefficientcoefficientwehavetofitanequa-

tion to the telecom data. In general, the estimate of ineffi-

ciency is made on the residual term generated from the

translog(Eq1)fittedtothedata.TheStochasticFrontiermod-

elsallow two typesof fit to the residualdata,a costanda

productionfunction.Ifastatisticaldensityplotoftheresidu-

al, U, is not skewed to the right, then a production model

cannotbeestimated.


2002.

Table 2 shows that all the equations with wireless densities

arehighlysignificant,andhavethecorrectpositivesignsthat

weanticipated.Theelasticitiesfortelephonedensityfellcon-

sistently from 0.37 in 2000, to 0.35 in 2001, and to 0.21 in

2002, as the sample size increased from 129, 139, and 151,

respectively.Inotherwords,in2000aonepercentchangein

telephone density increased GDP per capita by .37 percent,

whilein2002itincreasedGDPpercapitabyonly.21percent.

Anumberofreasonscanbesuggestedforthedecliningelas-

ticities. It might be the effect of including small countries

within the sample where we can have smaller elasticities

because the markets and institutions in small countries are

notasmatureasinthedevelopedones.Also,ascompetition

equalizes profits across the globe, thereby enhancing the

convergenceoftechnologicaladoptionanddiffusionoverthe

globe over time, profits and returns are becoming smaller,

showingupaslowerelasticities.Theliteratureappearstosug-

gestthatconvergenceishappeningintheglobaleconomybut

thatwouldbeamatterforfurtherstudy.

36 - The Journal of financial transformation 1 Someadditionaldefinitionsmayhelpcomprehension.Telephonewiresarefor

homesandbusinessesintheformofcopperwireandcables.Withportsortrunks,

networksarereached.TheinternetreachesserversaroundtheworldviaInternet

protocol.Wirelessiswireless.Cablesareusuallycoaxialthatareconsideredsmart

intheareasoftransmissionofvoice,data,andvideo,andconsidereddumbina

onewaytransmission,asforinstancewithacableTV.

number of equation, year, and observations Constant Telephone Internet Wireless Cable Adj-R-Sq D.W

EquationA(2002),151Obs. 5.99 0.21 0.35 0.29 -- 0.84 1.62

(56.37)c (2.85)c (4.26)c (4.85)c

EquationB,(2001),139Obs. 5.74 0.35 0.14 0.46 -- 0.80 1.46

(31.61)c (3.75)c (1.67)a (6.09)c

EquationC,(2000),129Obs. 6.11 0.37 0.19 0.38 -- 0.79 1.46

(28.56)c (3.59)c (2.10)b (4.35)c

EquationD,(2002),58Obs. 6.10 0.17 0.71 -- 0.06 0.79 1.03

(16.54)c (0.97) (4.73)c (0.74)

EquationE,(2001),77Obs. 6.26 0.17 0.78 -- 0.02 0.85 1.08

(24.23)c (1.27) (7.98)c -- (0.40)

EquationF,(2000),86Obs. 6.63 0.18 0.75 -- -0.03 0.85 0.77

(25.58)c (1.40) (8.13)c -- (-0.65)

Note:a-90%significancelevel.b-95%significancelevel.c-99%significancelevel

Table2:ResultsforOLSregressions,Year2000,2001,2002,(dependentvariableGDPpercapita)

Telephone Density internet Density wireless Density cable Density

Quintile 2000 2001 2002 2000 2001 2002 2000 2001 2002 2000 2001 2002

First .01 .02 .02 .01 .01 .01 .02 .002 .002 .02 .02 .07

Second .07 .08 .10 .02 .02 .04 .06 .03 .03 .08 .07 .14

Third .11 .15 .20 .04 .05 .08 .10 .09 .13 .17 .16 .16

Fourth .36 .38 .35 .14 .19 .19 .57 .34 .32 .23 .19 .55

Fifth .45 .37 .33 .79 .73 .68 .25 .54 .52 .05 .56 .08

Gini .468 .400 .348 .672 .644 .596 .388 .553 .529 .444 .480 .172

Observations 191 190 176 203 202 188 151 174 179 66 83 93

Table1:Distributionofhigh-techvariablesintheworld-2000-2003

Source:EstimatedbytheauthorsfromITUdatabase


measures:0.672in2000,0.644in2001,and0.596in2002.

However, three other density variables are countering the

declininginequalitytrend.

1. Wirelessdensitydefinedasthenumberofcellularsub-

scribersper100population.

2.Cellulardensitydefinedasthenumberofcellularsub-

scribersper100population.

3. Cabledensitydefinedasthenumberofcablesubscribers

per100population.

These counteracting variables would also help to explain

belowwhyfactorssuchasNetExport,definedasexportsless

imports, would yield an incorrect sign in explaining ineffi-

ciency1. In our case we estimate that many more countries

are importing than exporting, leading to the net exports

becominganegativenumber.Thiscanbeexpectedsinceour

samplecontainsmanydevelopingcountries.

Growth modelsIt iscustomarytoplottelephone, Internet,wireless,andcel-

lularvariablesagainstGDPpercapitatoshowthattheyare

positively correlated, and therefore acting as drivers for

growth[Jipp(1963),Maddenetal.(2002)].Inthelanguageof

the difference-in-difference methodology, countries that

adoptthenewtelecomtechnologycanbeconsideredasbeing

injectedwithanewtreatment,andthosethatcannotaffordit,

canbeconsideredasbeing inacontrolsituation.Thenaïve

conclusionwereachisthattelecomtechnologyisthecauseof

growth. Of course, this is not a sufficient test to prove that

telecom variables are the cause for growth. For instance, a

country might have grown for other reasons, such as trade

liberalizationorfrombelongingtoafreetradezonesuchas

NAFTAandtheEMU.

Anotherapproachtoshowgrowthinthetelecomfactorsisto

include themwithother traditionalvariables inaproduction

function relationship. Such a practice is widespread, even

thoughthemodelssufferfrommulticollinearity.Anystudent

ofdataanalysiswillnotewithDhrymes(1974)that ‘…it isan

empirical fact that in many sectors of the U. S. economy at

leastlnLtandlnKtarehighlycollinear…Thisisnottosaythat

such estimation has not been carried out in the literature.

IndeedthepioneeringworkofCobbandDouglasemployspre-

cisely thisapproach.’Thereareadditionalproblemswiththe

proper definition of capital and labor in such an analysis. In

studyingproductivitychange,Uri(2002)advocatedtheuseof

FullTimeEmployment(FTE)forlabor,butactuallyused‘total

numberofemployees.’SomelimitedFTEdataisavailablefor

ourmodels,buttheydidnotyieldsignificantresults,soweuse

population as a proxy. As for capital, the tradition has been

establishedtousespecifictelecominvestmentdata,whichwe

shalladoptaswell.StartingwithaCobb-Douglasrelationship,

Table2showsamodelrelatingGDPpercapitatothreevari-

ables—telephone,Internet,andcabledensity—inalogarith-

micform.Twosetsofthreeequationsareshownbecauseone

setusesthewirelessvariableandomitsthecablevariable,and

theothersetdoes thereverse.Toevaluate this relationship,

equations for three years are considered: 2000, 2001, and

37


two players, giving 50 cents to each, is a Pareto optimal

allocative efficiency because you cannot give more to one

withouthurtingtheother.Wewill,however,beconcernedwith

productiveefficiency.Intuitively,itdealswithsituationssuch

as the tangency of the isocost with an isoquant curve in a

production relation. Such tangency is an efficient point

becauseanexpansionlinefromtheoriginthroughthepoint

of tangencymarksoff thesamedistance fromtheorigin to

theisoquantastotheisocost,andthereforetheratiooftheir

distance away from the origin is one [Farrell (1957)]. This

intuitive approach has evolved into the modern concept of

‘StochasticFrontierModels’(SFM)inthemodernliterature.

OnemethodofestimatingSFMefficiencyistofocusonapro-

ductionspecificationsuchasthetraditionalCobb-Douglasor

thetranslogformsuchasused inCaves’book.Thismethod

regressesoutputonanation’ssetofinputs,ameasureofinef-

ficiency,andarandomerror.AsGreene(2000)putsit,‘Ify=

f(x)definesaproductionfunction…thenforanygivenx,the

observedvalueofymustbelessthanorequaltof(x).’Inother

words,weneedacountry’sactualproductionfunctionandan

efficientproductionfrontierforanestimateofinefficiency.If

acountry’sactualfunctioncoincidesonitsfrontier,wehave

productiveefficiency;ifnot,productiveinefficiencyresults.

Table3indicatesourestimateforinefficiencyforthetelecom

dataintheglobaleconomyfortheyears1997through2002.

Theestimateisfromatranslogspecificationfunctionofthe

form:

Log(GDP/Pop)=a0+a1log(K/pop)+a2log(Pop)+

a31/2(log(K/pop)2)+a41/2(log(pop)2)+

a5log(K/pop)*log(pop)+a6Z+V-U (Eq.1)

where:

GDP–grossdomesticproduct

K–telecomcapital

Pop–population

Z–othervariables

V–normalregressionerrorterm

U–inefficiencyerrorterm

BeforeweappliedthetranslogformtotheStochasticFrontier

estimation, we tested it on the data used in Table 2. The

resultslistedinTable3useanamalgamofthetelecomden-

sitydataforcapital,namelytelephone,Internet,andcellular

density.Thesearethevariablesthatyieldedsignificantresults

intheexploratoryanalysisofTable2.

Table 3 shows a dominance of significant coefficients. The

resultsfortheyear2000haveallpositivecoefficients.Those

for 2001 reveal insignificant coefficients for the population,

whichlingeredontotheyear2002.Thissuggeststhatthe9/11

shocktotheglobaleconomyhasnotbeenfullydampenedin


Year 2000 Year 2001 Year 2002

Variables Estimates T-values Estimates T-values Estimates T-values

Constant 11.66 5.44c 11.36 4.82c 13.12 4.67c

Log(cap) -0.54 -1.93b -0.72 1.74a -0.72 -1.67a

Log(cap)2 0.31 10.36c 0.35 6.51b 0.29 4.7c

Log(pop) -0.65 -2.53c -0.50 1.69a -0.66 -2.05b

Log(pop)2 0.03 2.03b .02 1.17 0.03 1.32

Log(cap.pop) 0.04 2.19b 0.04 1.66 0.05 1.98*

R-square 0.89 0.85 0.82

D.W. 1.93 2.25 2.24

Observations: 155 139 129


Table3:Translogfunctionsfordataintable2,paneldata2000-2002


Theelasticityestimatesof Internetandwirelessdensities in

EquationsA,B,andCexhibitamixedtrend.Theelasticitiesof

Internetdensitydeclinedfromabout0.19in2000,to0.14in

2001,andthenincreasedtoahighof0.35in2002.Inother

words, in 2000 a one percent increase in Internet density

raisedGDPpercapitaby .19percent,while in2002 itraised

GDPpercapitatoahigherlevelof.35percent.Thevariations

ofInternetusershadthelargesteffectonGDPpercapitain

2002, after a temporary setback in 2001. On the other hand,

variationsofwirelessdensityhadanopposite trendeffecton

GDP per capita, namely 0.38, 0.46, 0.29, for the respective

years.

The resultsofEquationsE,F,andGwherecellular replaced

wireless are less forceful. The entire cellular and telephone

density coefficients were insignificant. This is a case where

the sample size might be inadequate, as it is much smaller

than we observed for Equations A, B, and C. However, the

growtheffectofallthevariablesexceptcabledensity,asindi-

catedbythepositiveelasticitycoefficients,doesunderscore

growthinthisCobb-Douglasmodelthatgeneratedtheresults.

Tosummarize,thisempiricalsectionhasvalidatedthedirect

effectoftechnologyonGDP.Theresultsaremoresignificant

for three variables namely telephone, Internet, and wireless

densities.Partofthesuccessinthesignificantresultsisdue

to our ability to get large sample sizes for the Internet and

wirelessvariables,whilethe literaturetendstofocusmostly

ontelephonedata.Wehave,therefore,shownthatanexpand-

edsetoftelecomvariablesaredriversofgrowth.Wehavealso

demonstratedthatthecoefficientsappeartodriftdownward

overtime(2000to2002)fortelephonedensity,whilethepat-

ternforInternetandWirelessdensitiesaresomewhatinverse.

That the coefficient for telephone density fell could be

sourced to the varying sample size, shocks in the economy

suchas9/11,oreventhattheworldisconverginginthediffu-

sionoftechnology.

stochastic frontier efficiencyEconomicdatahavebeenusedtodemonstratetwotypesof

efficiencies, allocative and technical or productive [Caves

(1992)]. The simple experiment of dividing a dollar between

39

Half normal model Half-normal model Exponential model

Column 1 Column 2 Column 3 Column 4 Column 5 Column 6

Variables Estimates T-values Estimates T-values Estimates T-values

Constant 14.22 4.67c 17.78 6.81c 15.62 5.62c

Log(cap) -0.29 -0.86 -0.65 2.61c -0.51 -1.77a

Log(cap)2 0.21 20.56c 0.03 1.96b 0.29 1.54

Log(pop) -0.74 -2.40c -1.47 5.47c -1.30 -4.63c

Log(pop)2 0.03 2.05c .03 1.94b 0.02 1.26

Log(cap.pop) 0.03 1.71a 0.03 2.26b 0.03 1.61a

Log(PC) 0.86 19.65c 0.88 18.31c

Lambda 1.28 2.65c 1.87 3.65c

Theta 6.62 0.71

Sigma 0.82 9.03c 0.69 11.11c 0.46 7.08c

Sigma(U) 0.51 0.61 0.02

Sigma(V) 0.65 0.32 0.21

LogL -394.18 -274.96 -276.71

Observations 407 399 399


Table4:Stochasticproductionfrontier,e=v-u,paneldata1997-2002


(1-2/pi) x Sigma-square(U) = .16224, and Sigma-square(V) =

.25743. In other words, as previously noted, the variance of

thespecificationinEq1canbeaccountedforbytwocompo-

nents,namelytherandomerrorcomponentofanyregression

analysis,V,andthevarianceofthetruncatednormaldistribu-

tionusedbythecomputertoestimatetheinefficiencycoeffi-

cients intheappendix,U. InthecaseofColumns1and2, in

Table 4, about 60 percent of the overall variance (e=V-U in

productionspecifications), isexplainedbythevarianceofU,

whileforColumns3and4,withmoresignificantresults,the

percentofvarianceexplainedbyUissmaller(about22per-

cent).

Caves(1992)positsthat‘Aninternationalstudyofefficiency

naturallyseekstolearnwhichcountryisthemost(least)effi-

cient.’Intryingtoanswerthisquestionforthemanufacturing

sectorsofseveralcountries—Japan,Korea,Australia,Canada,

U.K,U.S.,andOECD—theCavesstudyfoundthatthemean

valueofefficiencyvarygreatlyacrosscountries,thattheesti-

mates forsome industries failedtocomputebecauseof the

wrongskewofdata,andthatthereisthegeneralquestionof

standardizationofdataandtechniquesacrosscountries.The

inefficiencydatathatweprovideintheappendixtothispaper

should therefore be interpreted with caution. The appendix

indicates that countries with an inefficiency index greater

than 70 for the 1997-2002 period include: Australia, Brazil,

Bulgaria, Estonia, Republic of Korea, Lithuania, Paraguay,

Peru,Philippines,Taiwan,Vietnam,Zambia,Zimbabwe,Chili,

Colombia, Czech, Malaysia, Malta, New Zealand, Nigeria,

Russia,Singapore,Slovakia,Ukraine,andSouthAfricawhich

representsabouta thirdof thecountries in the sample.We

notethatmostofthesehighindicesoccurredcloserto2002,

whichmayrecorda9/11effect.Amoregeneralsummaryof

theinefficiencycoefficientsisintheformofitsquintiledistri-

bution,whichshows47,148,81,67,and56observationsfrom

the first to the fifth quintile. In this context the distribution

referstothespreadofthe inefficiencydata intheappendix

overthefivequintiles.Inefficiencyismostlyinthesecondand

middle quintile, implying that there is much room for coun-

tries to still lower their inefficiencies. A popular use of the

inefficiencycoefficientsthatiscommonintheliteratureisto

explainthemwithothervariables.Onesignificanthypothesis

[Caves(1992)]isthatinternationalcompetitiontendstoraise

efficiency,andthuslowerinefficiency.Anotheristhatthedif-

ferencesincapital-laborratiosarefoundtonegativelyaffect

efficiency. We use exports less imports to measure interna-

tionalcompetition,andtheratioofgrossfixedcapitalforma-

tion in U.S. dollars to population for capital-labor ratio. The

specificationforthisinefficiencyestimateisasfollows:

Inefficiency=a+b.log(NETX)+c.log(K/L)+e (Eq.2)

where:InefficiencyismeasuredfromthebestrunofColumns

3and4ofTable4.Netexports(Netx)arethelogdifferenceof

exportsminusimports.K/Lisgrossfixedcapitalformationto

population.

TheresultsinTable5areforeachyearsince1997.Theyears

1998-2001 indicate significant coefficients for all variables.

Netexportshaveapositiveeffectoninefficiencyintheuseof

telecom infrastructure. In other words, if we regress ineffi-

ciency that is estimated from GDP per capita on telecom

infrastructure variables on net exports, the estimate will be

positive.Itisnotwhatweexpected.Examiningthedataindi-

catesthatmostcountriesexportedmorethantheyimported,

whichgivesadominanceofnegativesignsforthenetvariable

thatmayhaveskewedthedata.Humancapitaltheoristshave

argued that as the export sector expands, it attracts more

skilledworkers,andthereforecreatesmoreinequality.Indeed,

Borjas (2005)] found that skill-based technological change

increasedwagedisparity in theU.S.Nooneseemstoargue

withthepropositionthatworkerswhousepersonalcomput-

ersearnmorethanworkerswhodonot.Whileitisarguedthat

productivity should increase with increases in capital, the

resulting inequalitymaycause inefficienciestodominate,as

unskilledworkersremainidleforthecountriesinourdataset.

Ontheotherhand,thecapital-laborratiovariableshowsthe

propersign,indicatingthatcapitalintensitygenerallyincreas-

esefficiency.ItisnotunusualtofindlowR-squaresforthese

typesofstudies.Caves(1992)reportsanR-squareof0.16.

Policy implication



theyears2001and2002.Wewouldhavelikedtoproceedwith

theestimateofinefficiencyofthecountryonanannualbasis,

but as is well-known in the literature, it is not possible to

alwaysgetestimatesofefficiencywhenthesecondandthird

momentoftheresultingresidualarenotoftheproperskew,

asisthecasewiththeseannualdata.Properskewreferstoa

plotofthestatisticalfrequencyagainsttheresidualU.Ifthe

plotisskewedtotheleft,thenthedatawillprovideestimates

fortheproductionspecificationonly.Therefore,wewilllookat

the inefficiency estimates from paneling of the annual data

from1997to2002,whichwill lowerthenumberofobserva-

tionsbecauseoftheadditionalvariablessuchasthenumber

ofpersonalcomputers,andgrossfixedcapitalformationdata

whichwewishtouse.

Table 4 presents the results for our first estimates of effi-

ciency.Weobservefirstthatthenegativityofthecapitaland

populationcoefficients inthetranslogspecification isnota

violationoftheaprioriexpectedsignsofthosecoefficients

in thenormalwayCobb-Douglasspecifies.Thesignsof the

coefficientsareinfactafunctionofthepartialderivativeof

thespecificationwithrespecttocapitalorpopulation,which

aretobeevaluatedatthemeanvalueofthelogforcapital

andpopulation[Greene(2000)].Theresults,however,show

aninsignificantt-valueofthecapitalvariable,definedasthe

sumoftelephone, Internet,andcellulardensityforthefirst

equation.Tobringupthesignificanceofthecapitalvariable,

wehaveincludedanadditionalvariable:thenumberofper-

sonalcomputersinlogform(LPC)forZinthespecification.

The resultsareshown incolumns3and4,wherecapital is

now highly significant. The elasticity of capital, however, is

verysmall.Theequationuseda1+a3Log(K/Pop)+a5*log(pop),

isthepartialofEQ1withrespecttocapital-populationratio.In

ordertogetapositiveelasticity,wehavetousetheestimate

in its expanded decimal form, namely, [0.6453757519 +

(0.3061919319E-01 x 3.620191) + (0.3330021577E-01 x

16.80555)],whichyieldsalowelasticityof0.0251forcapital.

WeobservefromEquation1thatthemeasureofinefficiency

turns on the two components of the error term (e). The

V-component measures the normal random disturbances

associatedwithregressionmodels.TheU-component,whenU

ispositiveorzero,measuresaonesideddistributionoftech-

nicalefficiencybeneaththeproductionfrontierweareusing

here[Caves(1992)].Inthehalf-normalmodelcaseofcolumns

1and2ofTable4,thevariancecomponentsarecalculatedas

41

Years Constant Capital/Labor Net export R2 DW Obs.

1997 0.67 -0.05 0.01 .10 1.72 51

(3.82)c (-2.20)c (0.63)

1998 1.04 -0.09 0.03 .24 1.71 49

(5.78)c (-3.67)c (2.17)c

1999 1.25 -0.10 0.03 .25 1.95 60

(6.70)c (-4.36)c (2.42)c

2000 1.59 -0.14 0.03 .35 2.07 50

(7.50)c (-5.05)c (1.69)a

2001 1.48 -0.12 0.04 .24 1.97 52

(6.37)c (-3.89)c (2.30)c

2002 1.95 -0.17 0.03 .33 1.85 30

(5.23)c (-3.55)c (1.04)


Table5:Explanationofinefficiencyindex,1997-2002


finallyfiguredoutthattheirnationsmusthaveadvancedtele-

communications systems in order to compete in the world

marketplace,andthatthegovernment-ownedmonopoliesof

yoreare inefficient’ [Plunkett’sAlmanac2003-2004).Korea,

for instance, restructured its telecommunication industry in

1990,1994,1995,and1997inordertomakeitmorecompeti-

tive[Tchaetal.(2000)].Whilestillundergovernmentcontrol,

evenChinahasseentheneedtodecentralizesomewhat,from

theMinistryofPostandTelecommunicationtoconsortiasuch

asUnitedTelecommunicationCorporationsLtd.,andJiTong

Communications Co., Ltd. Similarly, India has moved away

fromcontrolbytheDepartmentofTelecommunicationstothe

Videsh Sanchar Nigam Ltd. for international services, and

MahanagarTelephoneNigamLtd.fordomesticservices.After

thedemiseoftheSovietUnionin1991,severaldecentralized

programswerebroached.In1995,anewlawallowedentrepre-

neurialactivitiestobeinitiatedbyforeigncompaniesinaddi-

tiontorestrictingmonopolies.Weshouldnotforgettomen-

tiontheAT&TCorporationwhosemonopolywasencouraged

duringtheL.B.Johnson’spresidency,hashadseveraldivesti-

tures.Thetrendpointstowardgreaterprivatizationandliber-

alizationofthetelecommunicationmarkets.

conclusionsTheGEDapproachgivesanewperspectiveforlookingatdif-

fusionof telecom infrastructure in thenewglobaleconomy.

While growth is indeed driven by telecom density data, it

needstobemonitoredcloselybygovernmentsandindustry

toascertainmoreefficientuses.Inshort,savingsandinvest-

mentshouldbeviewedfromthetelecomangle,notfromthe

traditionalKandLfactorsthatwereonceconsideredthemain

enginesofgrowth.

Unequaldistributionseemstohavealageffect,improvingfor

themorematureinfrastructure,suchastelephone,butstillin

its initial state for newer advances, such as cellular and

Internet.WhiletelephoneandInternetdensitiesarebecoming

less concentrated, the other densities are becoming more

concentrated.

Efficiencymodelsarestill intheirbeginningstates.Wewere

abletogetsomeestimatesforasampleofcountriesfromthe

productionpointofviewwhichshowsthatthetelecomdatain

Table1:telephone,Internet,wireless,andcabledensities,are

distributedmoreinthesecondandmiddlequintiles,andnot

intheupperquintiles.Yetthereisroomforimprovementfor

developedcountriesinupdatingtheirtelecominfrastructure,

andfor lessdevelopedonesin improvingtheirtelecomden-

sity per 100 population. Further studies will improve our

understanding of the cost inefficiencies of this industry as

well.



Policyimplicationsarebothqualitativeandquantitative.High

inefficiencyvariancesmaybedependentonthehighcostof

telecom infrastructure. In that case, governments and large

firms have a large role. ‘Dozens of governments worldwide

43

nation 1997 1998 1999 2000 2001 2002

Argentina 0.15 0.16 0.19 0.23 0.29 0.18

Australia 0.36 0.48 0.52 0.62 0.73 0.73

Austria 0.18 0.20 0.23 0.28 0.35 0.36

Belgium 0.19 0.20 0.20 0.24 0.25 0.24

Brazil 0.14 0.17 0.35 0.45 0.72 0.94

Bulgaria 0.71 0.64 0.69 1.04 0.98 0.91

Canada 0.31 0.38 0.43 0.45 0.51 0.54

Chile 0.22 0.34 0.49 0.59 0.78 0.85

China 0.29 0.44 0.62 0.79 0.88 1.12

Colombia 0.35 0.43 0.55 0.62 0.75 0.88

Cuba 0.16 0.19 0.27 0.32 0.52

Czech 0.50 0.56 0.63 0.77 0.81 0.79

Denmark 0.25 0.26 0.32 0.41 0.44 0.43

Ecuador 0.38 0.43 0.73 0.80

Egypt 0.19 0.23 0.27 0.26 0.37 0.47

Estonia 1.05 1.05 1.16 1.26 1.28 1.23

Finland 0.31 0.34 0.35 0.43 0.46 0.44

France 0.18 0.21 0.25 0.34 0.38 0.37

Germany 0.21 0.24 0.28 0.39 0.47 0.49

Greece 0.11 0.12 0.14 0.18 0.19 0.18

Hong Kong 0.19 0.24 0.30 0.35 0.39 0.44

Hungary 0.41 0.44 0.50 0.61 0.59 0.52

India 0.16 0.21 0.25 0.35 0.47 0.57

Indonesia 0.22 0.68 0.48 0.50 0.59 0.52

Ireland 0.27 0.27 0.30 0.35 0.35 0.32

Israel 0.26 0.29 0.34 0.35 0.37 0.42

Italy 0.13 0.15 0.18 0.24 0.27 0.28

Japan 0.14 0.18 0.19 0.20 0.27 0.31

Jordan 0.27 0.39 0.45 0.72 0.72 0.82

Korea Rep 0.39 0.77 0.83 1.06 1.27 1.29

Lithuania 0.49 0.69 0.75 0.76 0.77 0.90

Luxembourg 0.24 0.23 0.21 0.25 0.29 0.31

Malaysia 0.30 0.62 0.73 0.78 1.01 1.08

Malta 0.59 0.66 0.72 0.81 0.85 0.86

Mexico 0.21 0.22 0.24 0.26 0.31 0.38

nation 1997 1998 1999 2000 2001 2002

Netherlands 0.28 0.32 0.34 0.43 0.46 0.45

New Zealand 0.39 0.56 0.61 0.74 0.80 0.76

Nigeria 0.21 0.22 0.99 0.77 0.74 0.70

Norway 0.22 0.27 0.28 0.30 0.31 0.27

Pakistan 0.30 0.35 0.37 0.38 0.43

Paraguay 0.23 0.31 0.36 0.87 1.22

Peru 0.31 0.44 0.60 0.67 0.78 0.70

Philippines 0.41 0.62 0.60 0.73 0.87 1.00

Poland 0.30 0.35 0.49 0.54 0.60

Portugal 0.19 0.20 0.21 0.27 0.30 0.31

Qatar 0.16 0.21 0.19 0.15 0.16 0.17

Romania 0.40 0.39 0.62 0.70 0.72 1.16

Russia 0.29 0.63 0.99 1.11 1.06 1.11

Saudi Arabia 0.14 0.18 0.19 0.17 0.25 0.44

Singapore 0.32 0.46 0.54 0.54 0.64 0.77

Slovakia 0.61 0.72 0.93 1.11 1.12 1.14

South Africa 0.36 0.55 0.64 0.74 0.87 0.99

Spain 0.17 0.19 0.20 0.27 0.31 0.32

Sri Lanka 0.20 0.22 0.26 0.32 0.46 0.63

Sudan 0.51 0.39 0.44 0.44

Sweden 0.28 0.25 0.38 0.47 0.59 0.59

Switzerland 0.23 0.25 0.28 0.46 0.48 0.45

Taiwan 0.27 0.32 0.63 0.65 0.81 0.88

Thailand 0.23 0.38 0.36 0.47 0.60 0.69

Turkey 0.20 0.25 0.39 0.41 0.67 0.58

ukraine 0.45 0.66 0.96 1.04 0.91 0.88

united Kingdom 0.25 0.26 0.31 0.36 0.40 0.40

united States 0.34 0.38 0.41 0.45 0.50 0.52

uruguay 0.29 0.43 0.52 0.57 0.66

Venezuela 0.26 0.27 0.28 0.26 0.29

Viet Nam 0.48 0.63 0.73 0.79 0.85 0.90

Yemen 0.15 0.22 0.21 0.18 0.17 0.66

Zambia 1.02 1.11 1.06

Zimbabwe 0.66 1.02 1.11 1.06

Note:theinefficiencyindexisestimatedfromafitofEq.1tothetelecomdata.

Fortheactualcomputations,seeGreene2000,394.

Financial

corporate action processing: complexity and risk

what lies beneath

Hedge fund indices

Data management in financial services 2004 and beyond

integrated data architecture — The end game

Reference data primer

Data in financial institutions

Data mining in finance: From extremes to realism

1 Theopinionsexpressedinthisarticlearethoseoftheauthorsalone.Thesupport

fromourcolleaguesatDTCCandOxeraisgratefullyacknowledged.

2 ThisfigurecomesfromDTCCdata.Itdoesnotincludetheoverthreemillion

scheduledfixed-rateinterestpaymentsandscheduledmaturitiesthatoccurevery

year.

3 Kindlynotethatthearrowsinthefigureshowthecontractual/businessrelation-

shipsbetweentheparticipants.Thecorporateactioninformationandinstruction

flowsarenotshown.TheillustrationislargelybasedontheU.K.model,butthe

structureinothermarketsissimilarintermsofthelevelofcomplexityandtypes

ofintermediariesinvolved.

corporate action processing: complexity and risk1

James Femia, Managing Director, Asset Services, DTCCGunnar Niels, Economist, Oxera Consulting Ltd.

46

scrubbing to get the information rightThelargenumberofcorporateactionsandlongchainofinter-

mediariesmean thateveryday financial institutionsaround

the globe are flooded with millions of faxes, phone calls,

e-mails,andletterscarryingnewsofvariouscorporateaction

events.Theinformationcomesfromadiverserangeofsourc-

es, including custodians, broker/dealers, depositories, and

exchanges as well as news journals, wire services, and data

vendors. The information may be ambiguous, contradictory,

outdated,andsometimesjustwrong.

Part of the problem is inherent to the nature of corporate

actions: terms and conditions can be complex and highly

event-specific,andmayevenbemodifiedseveraltimes(i.e.in

atakeoverbid).Inaddition,thereisnocurrentstandardfor

corporateactioncommuniquésandmostrequiresomeman-

ual processing. For cross-border events, terminology and

practicesdifferacrossmarkets.Forexample,whatisdescribed

in the U.K. as a ‘one-for-one’ share distribution (whereby

holdersofasecurity receiveonenewshare foreachshare

held)isknownasa‘two-for-one’stockdistributionintheU.S.

Responsibilityforfailuresintheinformationflowusuallycan-

notbepassedontothedirectsourceoftheinformation.Thus,

ultimately,eachparty in thechain is responsible forgetting

theinformationright.Thisiswhytheindustryspendsconsid-

erable resourcesonvarious,oftenduplicative,externaldata

sourcesandinternaldatascrubbingefforts.Theseresources

clearlyrepresentinefficienciesandcostsinthesystem.

Failures in the processing of complex corporate actionsFormorecomplex,voluntaryevents,suchasrightsissuesor

takeovers,theinvestorisgivenanumberofoptionstochoose

fromwithinasettimeframe.Iftheinvestorisnotgiventimely,

accuratenotificationorthedecisionisnotrelayedbyexpira-

tion,thefinancialinstitutionprocessingtheeventmayhaveto

payasubstantialsumtocoverlosses.Around10%–15%ofall

corporateactionsareofavoluntarynature, translating into

100,000to150,000complexeventseachyear.

Thescopeforfailuresintheprocessingofcomplexcorporate

actionsishigh,forvariousreasons:

■ Thesheernumberofdifferentintermediariesandinves-

torsmeansthereisalargenumberofinstructionsfor

eachevent.

■ Mostinstructionsaresentviafax,telex,orunformatted

email,andprocessedmanually,resultinginsomepotential

formisinterpretationormishandling.

■ Themoreintermediariesinthechain,thetighterthe

deadlinefortheultimatedecision-maker,sinceeachinter-

mediarysetsitsowndeadlinetoallowsufficienttimeto

handleprocessing/communications.

■ Fundmanagerorinvestordecisionsaresometimes

changedbeforethedeadline,andthecustodianwill

receiveasecondinstructionforthesameclient.

Inpractice,theliabilityforprocessingfailuresisusuallyborne

bythemarketparticipantwherethefailurearises.Thispar-

ticipantwilltypicallyhavetoincurthecostofcompensating

theclientforlossesincurred,orre-establishingthepositionin

which the client would have been, had the instruction been

processedcorrectly.Forexample, if,forascripdividend,the

investor(oritsportfoliomanager)optsforcash,butduetoan

errorbythefundmanagerorthecustodian,theinvestorends

upwithextrastock,thecostbythepartyresponsibleforthe

error would basically be the loss incurred when selling the

stockforcashaftertheevent,potentiallyatalowerprice.4

ArecentstudybyOxeraestimatesthatthedirectriskstoany

individual firm involved in the corporate action processing

chain can be very significant.5 Failure in handling a single,

complex event has the potential to result in losses running

intotensofmillionsofeuros.Theseestimatesareinlinewith

riskassessmentsthatsomefirmshavemadeinternally.The

risk is highest for individual custodian firms because they

safeguardlargeamountsofassetsonbehalfofmanyinves-

tors,butbrokers,fundmanagers,andotherfinancialinterme-

diariesalsofacerisks.

474 Inthisexample,thereisonlyacosttotheinvestorifthestockpriceislowerafter

theevent.Ifthestockpriceturnsouttobehigher,theclientisbetteroffbecause

ofthemistake,andnocompensatingactionmayneedtobetaken.

5 Oxera,2004,‘Corporateactionprocessing:Whataretherisks?’,reportsponsored

byDTCC,May.Thereportisavailableonwww.oxera.co.ukandwww.dtcc.com.

Figure1:Participantsinthecorporateactionchain

custodian’s nominee

Nameonregister

csDWheresharesare

ultimatelyheld

custodianAppointedbyinvestor

issuer

Registrar/agentAppointedbyissuerto

maintainregister

Broker/dealerExecutionoftrades

wherenecessary

Fund managerAppointedbyinvestor

OwnershipAccount

Service-levelagreement

Service-levelagreement(oroperationalrelationshipifbothappointedbyinvestor)

Mandate/service-levelagreement


Registersasshareholders(vianominee):notifieschangesinownershiptoregistrar


Registersasshareholder

Account

Inthesecuritiesindustry,theprocessingofcorporateaction

eventsprobablyrankslastintermsofefficiency,automation,

and standardization. Close to one million corporate actions

takeplaceeveryyearworldwide.2Theyhappeneachtimea

changeismadetothecapitalstructureorfinancialpositionof

an issuer that affects any of the securities it has issued.

Dividendpayments,rights issues,tenderoffers,conversions,

takeovers, mergers, and early redemptions are just a few

examples.

Asingleeventmayinvolvehundredsofdifferentmarketpar-

ticipants (includingcustodians, fundmanagers,broker/deal-

ers,anddepositories),ultimatelycascadingdowntotensof

thousandsofinvestors.Eachoftheseparticipantsfaceshigh

risk because corporate action processing is complicated,

deadline-driven,notstandardized,andtoa largeextentstill

manual.

In thisarticlewedescribehowtherisks incorporateaction

processingariseandprovidesomeindicationsofthemagni-

tudeoftheserisks.Weexplainthatcorporateactionsnotonly

involveprocessingrisksinthebackoffice,butalsosignificant

tradingrisksinthefrontoffice.Weconcludethearticlewitha

discussionofpossibleimprovementstothesystem.

A long chain of intermediariesIntheory,corporateactionsoughttobesimple information

ormoneyflowsbetweenanissueranditsinvestors.Inreality,

however, each event involves a range of intermediaries.

Figure1presentsanillustrationofthevariousparticipantsin

thecorporateactionchain.Informationandinstructionflows,

which are not shown3, add to the complexity. Furthermore,

the illustration represents a domestic chain. Cross-border

transactionstypically involvealargernumberofcustodians

(or‘sub-custodians’)andotherintermediaries.

investor

6 ThesefiguresarefromtheElkins/McSherryGlobalUniverseRanking(Q42003).

7 G30(2003),‘GlobalClearingandSettlement:APlanofAction,’January.

8 GiovanniniGroup(2003),‘SecondReportonEUClearingandSettlement

Arrangements,’April.

Severalmajorindustryinitiativesareconcentratingoncorpo-

rate actions. There is a push under way to achieve what is

called ‘at source’ standardization, targeted to address the

integrity of corporate action announcements at the point of

origin or creation. The ultimate goal of at source is twofold.

One is to streamline communications between the issuer or

companyinitiatingthecorporateactionandtheinvestor.The

other is to allow for the electronic distribution of corporate

actioninformation,sothatitispossibletoeliminatetheerror-

prone messaging by fax, phone, and email that plagues the

industrytoday.

BoththeInternationalSecuritiesServicesAssociation(ISSA)

and theSecurities IndustryAssociation (SIA) in theU.S.are

promotinginitiativestoprovideclear,consistent,anduniform

corporate action information in prospectuses and proxies.

Otherorganizationsarealsoinvolvedinthis,whilesecurities

marketsgroupsallaroundtheworldareworkingtodevelop

bestpracticeguidesfortheuseofthe ISOcorporateaction

messagestandards.

Recognizingthattechnologycanbeacatalystforchange,a

series of automation and straight-through processing (STP)

initiativesoncorporateactionsarealsobeingpursued.These

projects are aimed at improving the quality of corporate

action information, streamlining processing, eliminating

redundancies and duplication of effort, and advancing stan-

dardsandbestpracticesindustrywide.

Whatisclearfromalltheseinitiatives,studies,andreportsis

thatgreatchallengesfacetheindustrygloballyindealingwith

corporateactions.Notonlyarevolumesgrowing,butthefact

is that more and more corporate action events occur cross

borders. Thus, no single approach will result in a complete

solution.Whilenationalandregionalinitiativesareimportant

to improve the way corporate actions are handled, a global

perspectiveiscriticaltoguidedevelopmentoverall.Clearly,a

confluenceofefforts,ideas,andtechnicalexpertisebymarket

participants and industry groups, working within national

markets and globally, is the only way to bring stronger risk

management, greater efficiencies, and STP for corporate

actions.


Thesamestudysuggeststhatactuallossesduetoprocessing

failures are somewhat lower, precisely because firms in the

industryspendverylargesumsonfailureprevention.Available

data on the European fund management industry indicates

thatfirmsinEuropeincurtotalactualcosts intheregionof

65millionto140millionperyear.

Trading risks to the front officeAcorporateactionannouncementoftenrepresentsnewinfor-

mationaboutanissuerandthereforemayimpactthemarket

valueofsecurities.Corporateactioneventsthuscreate(tem-

porary)arbitrageopportunitiesforbrokerageandfundman-

agement firms tradingeitheronbehalfof investorsor their

own proprietary account. This is illustrated by the fact that

somefirmshavespecializedtradersattemptingtogainfrom

arbitrageopportunitiesarisingfromcorporateactions.

Inpractice,tradingdeskswithinthesefirmstendnottorelyon

the corporate action chain depicted in Figure 1 for informa-

tion. This chain is simply too slow. Rather, information on

eventstendstocomethroughthenewspapers,newswires,as

wellaswordofmouthandrumors.Onlyatalaterstage(often

afewdaysafter)doesthemoreformal,detailednoticeofthe

actioncomethrough,whichisreliedonbythebackofficeto

processtheevent.

Thelackofup-to-dateinformationormisinformationcanlead

tosub-optimaltradingdecisions.Twotypesofcostarelikely

toflowfromthis.Firstly,thetradesthemselvesincurtransac-

tioncosts,whichcanbesignificant.Averagetransactioncosts

in themajor stockmarketscurrently range fromaround20

basis points in Japan and Switzerland to 60 basis points in

Taiwan,SouthKoreaandMalaysia.6Thisincludestradingfees,

brokeragecommissions,andmarketimpactcosts.Forround-

triptrades,i.e.,whenthemistakentradeneedstobeunwound,

thiscostisincurredtwice.Secondly,misinformedtradersrun

theriskofadversemarketmovement.Ifthestockpricedoes

notinstantaneouslyincorporatetheinformationcontainedin

a corporate action announcement, the failure by individual

traderstounderstandthecorporateactionplacesthematan

informational disadvantage to the rest of the market. This

disadvantagemeansthatthestockpricewillhavesystemati-

callymovedagainstthemwhentheyattempttounwindtheir

mistakentrade.

TheaforementionedOxerareportprovidesanestimateofthe

order of magnitude of these trading risks. The estimate is

basedonanumberofconservativeassumptions,forexample,

that only a subset of complex corporate actions have the

potential to move stock prices significantly (including take-

overs,mergers,andrightsissues),andthatonlyaverysmall

proportionoftraderswillbemisinformedatanypointintime.

Nevertheless, the total trading riskonaglobal scale ranges

between1.6billionand8.0billioneveryyear.

Tradinglossesduetocorporateactioninformationfailureare

reflectedinlowerreturnstofundmanagersandinvestors,and

reducednettradingincometobrokers.Asaresult,although

the direct costs of failure in the dissemination of corporate

action informationmaynotbedirectlyobservable, theyare

nevertheless real costs to their business, profitability in the

caseofbrokersandsuccessinthemarketplaceforfundman-

agers.

Towards greater standardization and efficiencyIn pursuit of greater efficiencies, lower costs, and stronger

risk management, industry groups and policymakers are

focused on post-trade activities, including corporate action

processing.IndeedoneoftherecommendationsinaGroupof

30 report (January 2003) is to automate and standardize

asset-servicing processes, including corporate actions.7

Underpinning the G30’s recommendation is the observation

that, ‘Corporate actions, across the market, are the major

sourceof financial lossesattributable tooperational failure.’

Likewise, the second Giovannini report (April 2003), for the

European Commission, stated that there are significant

nationaldifferencesintherulesandpracticesgoverningcor-

porateactionswithintheE.U.8Thesedifferencesmayactas

abarriertoefficientcross-bordersecuritiestransactions,and

possiblytheintegrationofEuropeanequitymarkets.

49

what lies beneathLars HamichManaging Director, STOXX Ltd.

ofourkeyissuesistransparencyforlicenseesandusers.This

approachrequiresevenmoretimeandaccuracyaswestrictly

followourrulesandpublishaforecastlisttwotofourweeks,

plusthefinalnumberstwodays,priortotheeffectivedate.

Gettingthis informationisnotalwayseasy.Theindexteams

encounteredacoupleofinstanceswheretheyhavecalledthe

companiesinvolvedandtheirinvestorrelationsdepartments

didnotevenknowwhattheyweretalkingabout.Hostiletake-

oversinparticularseemtobringoutsomepeople’sstonewall-

inginstincts.Sometimes,thesepeoplehavetobetoldwhatis

publicrecordandthattheyneedtoprovidetheinformation.

Youwillhaveinstanceswhereyouhavetotalkto15different

people.Theyarebouncingyouallovertheplace,andyoujust

havetobeveryclearaboutwhatyouwantandkeepondriv-

ingithome.Andthenyouwindupwiththesecretarywhosays

somethinglike‘Iamsorry,Idonotknowwhatyouaretalking

about,’andthentriestohanguponyou.Youhavegottobe

veryaggressiveandassertive.Somecompanieswill tellout-

right lies. In one case, the index-support team researched

whatseemedtobeadiscrepancywithawesternrailcompany

thatwasbeingacquired.Uponcheckingwithacolleagueat

anotherindexprovider,wediscoveredthattherewasindeeda

discrepancyandthattherailcompanyhad liedtobothDow

Jones Indexes and the other index provider. On the other

hand,someofthecompanieswillbendoverbackward.Inone

case in which two utilities were merging, the department

receivedregularcallsfromtheacquiringcompany’sCEOand

hissecretary.Basically,therewasnolegworkonourpartsince

theygaveustheinformationbecausetheyknewweneededit.

And many times, poor communication can be the source of

difficulties. Inemergingmarkets, informationcanbehardto

find,andlanguagebarrierscanthrowupadditionalhurdles.In

other cases, a corporate action, such as a triple merger in

Japan,isjustextremelycomplicated.Atothertimes,thetim-

ingofadeal’scompletioncanbeelusive.Amergermightbe

delayed for months while approval is sought from various

quarters.Oncethatapproval isreceived, themergercango

throughinaslittleas20minutes.

In the case of the formation of Mitsubishi Tokyo Financial

GroupInc.,threecompanies—BankofTokyo-MitsubishiLtd.,

MitsubishiTrust&BankingCorp.,andNipponTrustBankLtd.

— merged in April 2001. It was a complicated deal both

because three companies were involved and because of

Japaneserulesregardingmergers.InJapan,thereisnosur-

vivingcompanyafteramerger.Themergingcompaniesare

de-listedforthetransactionandthenthenewcompanyisre-

listedaboutaweeklater.Thehandlingofthistransactionwas

particularlyimportantbecauseBankofTokyo-Mitsubishiwas

acomponentintheDowJonesGlobalTitans50,akeyindex

thatunderliesavarietyofinvestmentproducts,aswellasthe

benchmarkDowJonesWorldIndex.Moreover,theDowJones

GlobalTitans50 isan indexwitha fixednumberofcompo-

nents.Normally,ifonewerede-listed,asubstitutewouldtake

its place. But in this instance, the old company essentially

wouldbeavailableagain inadifferentformonlyafewdays

later.Withlicenseesusingtheindexforinvestmentpurposes,

itwasveryimportantthattheindex-supportteamsmadethe

changeinsuchawaythatitcouldbeeasilyreplicatedinthe

realworld.

Itisamatteroftallyingthevotesofourlicensees.Ofcourse,

wedonotletthemdecidewhatwedo,butwecertainlydolet

themknowwhatisgoingonandtakeintoaccountthepracti-

calproblemstheyface.Wehaveourmethodology,butalsowe

need to consider their issues, such as possible transaction

costs. In the end, we decided to hold the Bank of Tokyo-

Mitsubishi ‘dead’foraweek,meaningtheDowJonesGlobal

Titans50wascalculatedforashorttimewithoneofitscom-

ponentsheldatthesamepricethatitlasttraded.Thisimple-

mentationsavedtheindexfromturnover‘noise’anditslicens-

eesfromsellingastockoneweekandbuyingbackessentially

thesamesecuritythenext.

Inadditiontohard-to-findinformationandcomplicatedcorpo-

rateactions,baddatafromoutsidesources isanoccasional

factoflifethattheindexteamsintheU.S.responsibleforDow


51

For most investors, an index is a number. The number tells

them something about the current state of a market for

stocks,bonds,creditderivatives,orsomeotherinstrument,be

itforeignordomestic.Forthosethathaveinvestedinanindex

fund,thechangeinthatnumberfromdaytodaycloselyrep-

resentsthemovementsoftheirportfolio.

Nottoomanypeoplethinkmuchaboutwhatittakestopro-

duce that number, and the few that do usually regard the

processasabitmurkynomatterhowclearlystatedtheindex

methodologymaybe.Imaginativefolksmayenvisionatroop

of grumpy gnomes scribbling with quill pens, feeding their

handiwork to awaiting index fairies that deposit the results

underthepillowsoffinancialpublisherseverywhere.Intruth,

of course, the process involves complex computer systems

and endless cycles of research and adjustment. Like an ice-

berg,thevisiblepartofanindexmisleadseveninformedview-

ersabouttheimmensity,andtheimportance,ofworkthatlies

beneaththesurface.

Iwillskipthepartsaboutconceivinganindex,developingits

methodology,establishingadataandcomponenthistory,and

thenlaunchingitontodatascreensaroundtheworld.Thatis

the‘glamor’ofindices,suchasitis.Afterthat,theindexpro-

viderfacesendlessdailymonitoring,research,andauditingto

keepallitsindicesbeforeinvestors’eyes.Ifanindexprovider

doesnotdothisjobwell,itsindexnumberswillsoonerorlater

be wrong, which probably will cost investors some money

beyondwhatthemarketitselfmayinflict,andinturn,inves-

tors will stop using indices from that provider. This is not a

businessmodeltheindexproviderwantstofollow.Toillustrate

theprocess,wefocusonDowJonesIndicesandSTOXXLtd.,

respectively the index teams responsible for index support

and production. Every index provider, though, faces similar

issuesandchallenges,whichdemandenormousinvestments

ofresourcestohandle.

index supportThe support groups are responsible for researching and

recordingallchangestothecomponentsofastockuniverse

fromwhichhundreds,ifnotthousands,ofindiceshavebeen

derived.Thegroupsalsonotifylicenseesofthesechangesin

greatdetail.

Therearetwomaintypesofchanges,ongoingandperiodic.

Ongoingchangesaretypicallycorporateactions.Throughout

the quarter, researchers scan news articles for mention of

indexcomponents.Ahandfulofpeoplemonitorapproximate-

ly5,500companiesaroundtheworld.This‘corporateactions

team’ devotes a significant portion of each day to ferreting

outeventsaffectingcomponentcompanies.Therearenodata

vendorsofferingalltheneededinformation,sothesegroups

shoulder the responsibility. They are looking for mergers,

acquisitions, spin-offs, bankruptcies, delistings, stock divi-

dends, stock splits, and cash dividends. Part of their day is

spentdoingresearch;anotherpartsendingreports.

Stocksplitsareamongthebestknownofcorporateactions.

Theytendtobefairlystraightforwardandarevirtuallyguar-

anteed to take place on their pre-announced date. Mergers

andacquisitionsareanotherstoryentirely.Fromthe timea

merger or takeover is announced, researchers collect every

documentreleasedonthetopicandregularlycallthecompa-

niesinvolvedtocheckonthestatusofthedealandtogetas

manydetailsastheycandigout.Regulatoryapprovalmustbe

receivedbeforethedealcangothrough,andsomeindustries,

suchasutilities,havetheirownregulatoryagenciesthatmust

approvethemergers.AsinthecaseofHewlett-Packardand

Compaq,therearesometimesshareholderlawsuitsthatdelay

thecompletionofamerger.Differingrulesinforeigncountries

regardingmergerscanalsocomplicatethemonitoring.

Wekeep trackofeverything.Wehavenotes. From the time

thatamergerisannounceduntilitiscompleted,wefileevery

pieceofpaperthatcomesoutrelatingtothedealorrecordit

in our database. This documentation is particularly helpful

whenexplainingtolicenseeswhycertainchangeshavebeen

madeintheindexandwhyDowJonesIndexesorSTOXXLtd.

mighthavechosentomakethechangesinadifferentway,or

atadifferenttime,thananotherindexprovidermayhave.One

Jones Indexes and in Switzerland responsible for the Dow

JonesSTOXX indexesacceptwith relativeequanimity,while

at the same time trying diligently to minimize its effects.

Theteamsgatherinformationfromavarietyofsourcesand

makeitapolicytoverifyeachpieceofdatawithatleasttwo

sources.

Nonetheless, during the past several years, the information

flowhasimprovedsubstantiallyinbothquantityandquality.

Severalyearsago,theKualaLumpurexchange,forexample,

wasstillusingachalkboardtorecordthepricesofitsstocks.

Today,ithasafirst-ratewebsitethatprovidesmoreinforma-

tionthansimilarsitesmaintainedbysomestockexchangesin

developedmarkets.

Thekeytohandlinganyunresolvedquestions,inallcases,is

tomaintaincontactwiththe licensees,whether it isbytele-

phone, e-mail, or drums and smoke signals. Index providers

have their own ways of informing licensees and others. For

example, S&P has an Internet-based product called Index

Alert, which provides details of corporate actions and their

expectedeffectonitsindexes.MSCIoffersanarrayofindex

informationservices,asdoesFTSE.AtDowJonesIndexesand

STOXX Ltd., customers are alerted to ongoing changes via

daily reports that are sent out by the index-support teams,

whicharealsoincharge,notsurprisingly,ofcustomerservice.

Thereports listupcomingchanges to indexcomponents for

thenexttwoweeks.Wedoeverythinginourpowertogivetwo

businessdaysnotification.Sowithintwobusinessdaysofits

effectivedate,ithastobesetat100%probability—itishap-

peningorwehavetomoveit.Andoncewehave100%prob-

ability,weknowthenewsharesandthenewfloatinformation.

Periodicchangesalsoaremade to indices.Most indicesare

reviewed quarterly, semiannually, or annually, meaning new

components are selected. Shares outstanding are updated

monthlyandtherankofacompanyisamendedaccordingly,if

necessary.DowJonesIndexesandSTOXXLtd.,adjustacom-

pany’ssharesimmediatelyifitamountsto10%ormoreofthe

marketcapitalization;smalleradjustmentsareaccountedfor

during thenext review.Thereviewsare implementedat the

endofeachcalendarquarter,meaningthethirdFridayofthe

lastmonthinaquarter,butthereviewprocesskicksoffbegin-

ningofthesecondmonthofthequarter.

A review basically begins by taking a snapshot of the index

componentsastheyexistonaspecifieddateandcomparing

it to the previous quarter’s snapshot to spot what has

changed.Deletions,namechanges,andinitialpublicofferings

mustbeidentified.Onecoulddescribeitasasceneoforga-

nizedchaos,sincereviewsessentiallydoubletheeffortsofthe

indexteams.

Forthethirdmonthofthequarter,acoupleofweeksbefore

thatthirdFridayattheendofthequarterwhenthechanges

become implemented, the workload is incredible. At that

point,allthedecisionsregardingthecomponentshavebeen

madeanditisamatterofcheckingandrecheckingthedata,

enteringitintothedatabaseandcommunicatingthechanges

tothelicensees.

ProductionThe changes that index-support people have researched,

tracked,anddocumentedthenmustbeincorporatedintothe

indicesbeingcalculatedinrealtimeorendofday.Thejobof

implementing changes falls to the ‘production group.’

Production also implements routine changes such as stock

symbols,identificationnumbers,suchasCUSIPs,andswitches

inprimaryexchangelistings.Productionisascloseasitgets

totheindexfairybecauseitisthemaindistributionhubforall

indicescreatedandmarketedbyDowJonesIndexes.TheDow

JonesSTOXXindexesarecalculatedinZurichanddistributed

through the Deutsche Börse in Frankfurt. The Zurich index

teamisinchargeofmaintaining,calculating,anddisseminat-

ingindexdataonaday-to-daybasis.BecausetheDowJones

STOXXindexescoverEurope,theindexteamisavailablefrom

8:00a.m.CETto8:00p.m.CET.

Production in the U.S. is a 24-hour operation from Sunday

evening throughFridayevening.Asmarketsopenandclose


each day, beginning in Asia and then in Europe and the

Americas, there is a constant cycle of auditing data and of

generatingreportsforclientsandlicensees.

ThetwomaindailyauditsintheU.S.arerunat2:15p.m.ET

andat5:15p.mET.Whenanauditturnsupadiscrepancy,itis

uptotheproductionstafftotraceitbacktothesource.There

isnotonesingleansweronhowtofixadiscrepancy.Itisjust

troubleshooting.Ausualplacetostartwouldbethechanges

thatweremadeinthesystemsattheendofthepreviousday.

Severalotherauditsthroughouteachdayzeroinonparticular

dataelements,suchasnumericalsecurityidentifiers,security

names,andsoforth.AseparateauditscrutinizesDowJones

STOXXindexdatasenttotheU.S.fromEurope.Stillanother

audit is runagainstReutersdisplaysof indexdata toverify

thatthedatavendorsreceivedthesameinformationthatwas

sent out. Before heading home, production personnel verify

thatallthecorporateactionstotakeeffectonthenexttrad-

ingdaywereappliedcorrectly.

Quite frequently, exchanges are sending out corrections to

previouslytransmittedprices,orreportingatradewellafter

themarkethasofficiallyclosed.Theindexteamsthenhaveto

pause thesystemtokeep thesummaries fromgoingout. If

theydonot,theindexdatawouldbeincorrect.Butcustomers

cangetimpatient.Peoplearelookingfortheirfilesatapar-

ticulartimeeveryday.Ifthereisaproblem,theteamscontact

thecustomerstoletthemknowthatthefileswillbedelayed.

Oncetheproductionteamsjobsaredone,though,thenum-

bersgooutintotheworldandbecomeintegralpartsofinvest-

mentdecision-making.

conclusionThere is nothing particularly difficult about any single step

toward the creation of an index of investable securities.

However, the process of calculating and maintaining them,

either second by second or day by day, is enormous, espe-

ciallyonaglobalbasis.Peoplewelloutofwhateverlimelight

mayexist intheworldofindicesdevotecopiousamountsof

intense attention to endless details to ensure the accuracy

and,byextension,usefulnessoftheseproductstotheirusers.

Their efforts, especially among those who use index-based

investmentproductstothetuneoftrillionsofdollars,should

gounrecognizednolonger.

53

Hedge fund indicesJames R. Hedges, IVPresident & Chief Investment Officer, LJH Global Investments

Ashedgefundinvestorscontinuetheirquestforabenchmark

bywhichtomeasureinvestmentperformance,thesubjectof

hedgefundindicescontinuestoattractattention.Newindices

continuetocropupascompaniescontinuethequestforthe

Holy Grail, a precise index that would accelerate industry

growth much the same as the S&P 500 has furthered the

equityandmutualfundindustries.

Whilehedgefundindicesarenotyetonthesamelevelasthe

S&P500,duetothelackofconsistentandcompleteindustry

data, they are good tools for investors. They do measure

recenthedgefundperformancewithinasmalldegreeoferror

andcancontributetoinvestors’abilitytodetermineexpecta-

tionsoftheirownhedgefundinvestingexperience(Figure1).

Beginningin2004,theWallStreetJournalbeganpublishing

several hedge fund strategy indices in an effort to capture

performanceinthespace.Additionally,manyfirmsareestab-

lishingapresenceeitherthroughtheirownproprietarysetof

indices or through a much debated, passive investment

approach.Theseindices,whethercharacterizedas‘investable’

or ‘simplebenchmarks’, trackeithera specific fundstyleor

theoverallhedgefundmarket.Despitethemanyinconsisten-

ciesandbiasesassociatedwiththem,hedgefundindiceshave

the ability to reasonably characterize the directionality of

hedge fund performance. Relative benchmarks for hedge

fundsdomakesenseandshouldbeutilizedasadirectional

gauge.Asthehedgefundmarketdevelopsandtransparency

increases, it is likely that a practical benchmark will rise to

becometheindustrystandard.

Atthispointintime,hedgefundinvestorsneedtounderstand

the utility of the existing hedge fund indices and the data-

basesusedtocollectfunddata.Itiscriticaltobeawareofthe

shortcomings associated with these indices, including data

discrepanciesandbiases,constructionmethodologies,classi-

fications,andtheabsolutereturnversusrelativeperformance

debate. It is interesting tocompareandcontrasteach index

providerwithrespecttotheirconstructionmethodologiesand

performancedata,toexplorethenotionofinvestableindices,

andtodiscusstheprosandconsofanactiveversuspassive

approach.

Finally, we consider the future of hedge fund indices in the

contextofrecenttrendsinthehedgefundindustry.Specifically,

weexaminetherolethattransparencyandincreasedregula-

tion will play on these indices and hedge funds in general.

Clearly,hedgefundinvestorscanbenefitfromtheusefulness

ofarelativebenchmark.Althoughthereisnouniversalhedge

fund index that can adequately represent the hedge fund

worldandwhileexistingcompositesdifferwidelyincomposi-

tionandperformance,hedgefundindicesarestillreasonably

goodindicatorsofperformance(Figure2).

Hedge fund data and databasesIn an attempt to monitor hedge fund performance, several

hedgefunddatavendorscollectmonthlyperformancefigures

for thousandsofhedge funds.Additionally, some firms that

produce hedge fund indices maintain their own databases

fromwhichtoconstructtheirindices.Thetypicalhedgefund

database collects performance figures for each fund on a

monthlybasis.Therearetwoprimarymethodsfordatacollec-

tion: analyst or manager entry. According to a study con-

ducted by Strategic Financial Solutions (a comparison of

majorhedgefunddatasources), twocommercialdatabases,


index 2003 YTD RETuRn

EACM100Index 12.40%

VanU.S.HedgeFundIndex 19.00%

CSFB/TremontHedgeFundIndex 15.44%

HennesseeH.F.Index 19.69%

HFRIFundWeightedCompositeIndex 19.56%

TheBernheimIndex® 15.30%

MSCIHedgeFundCompositeIndex 14.71%

InvestHedgeCompositeIndex 9.28%

S&PHedgeFundIndex 11.10%

Disclaimer:Theinformationandstatementsoffactsinthisarticlearebasedupon

sourcesLJHGlobalInvestments,LLCbelievestobereliable,butdoesnotguaran-

teetheiraccuracy.Opinionsandestimatesincludedinthisarticleconstitutethe

judgmentofLJHGlobalInvestments,LLCasofthedateofpublicationandare

subjecttochangewithoutnotice.

Figure1:Hedgefundindicesperformancein2003

55

AltvestandHedgefund.net,currentlyrelyonmanagerentry,

andtherestuseanalystentry.

Thetypeofdataprovidedbythesevariousdatabasevendors

should also be taken into consideration. Databases contain

bothqualitativeandquantitativeinformation.Qualitativedata

for each fund includes fields such as assets under manage-

ment,feerequirements,performancereturns,legalstructure,

minimum investment, and investment style. The Strategic

FinancialSolutionsstudyalsoshowedthatdataqualityamong

thevariousvendorsdiffers.Discrepancieswerediscoveredin

mostlyqualitativedatafields,includingminimuminvestments

aswellasentry/exit/lockupinformation.

Worthnoting,subscribingtoadatabaseisamethodbywhich

hedgefundmanagerscandemonstratetheirperformanceto

the industry and potentially obtain new investors. On the

otherhand,hedgefundsarenotobligatedtoreport toany

database. When funds falter, they may elect not to report.

Likewise,whenfundsclosetonewinvestment,theymaystop

reporting. Clearly, hedge fund data (or the lack thereof) is

among the key issues facing the reliability of hedge fund

indices.

Existing hedge fund indicesThe first indicesused to trackhedge fundsappeared in the

1980s,butthemajorityofthemwerestartedwithinthelast

decade. Currently about a dozen firms produce a variety of

hedge fund indices that trackeitheraspecific fundstyleor

theoverallhedgefundmarket.Asopposedtothetraditional

equitymarketwheremanylooktotheS&P500,noparticular

Hedge fund inception number of classification selection/sampling (weighted/simple)

indices ‘strategy’ indices methodology criteria examples mean vs median

HFR 1994 37 Manager ·NominimumtimeorAUM Simplemean

·Separatesamplesforoffshoreandonshore+combinedone

Altvest 2000 14 Altvest ·NominimumtimeorAUM Simplemean

·Includebothonshore&offshorefunds

Hennessee 1987 24 Themanagerand ·MinimumAUMofU.S.$10mm Simplemean

committeeapproved ·1year

CSFB/Tremont Nov-99 14 Tremont ·1yearorU.S.$500milAUM Assetweightedmean

·MinU.S.$10milAUM

·Includebothonshore&offshorefunds

S&P Indices 2002 10 S&P ·MinimumAUMandtrackrecord,volatilityscreens Simplemean

MSCI Indices Jul-02 4:furthersegmented Themanagerand ·Includesallfundsinuniverse Assetweighted&

into190indices committeeapproved ·Eliminatesduplicates(onshoreonly) simplemean

Dow Jones 2003 5 Themanagerand ·MinimumAUM&trackrecord,duediligence,qualitativescreens. NAVcalculation

committeeapproved ·Eachindexisrunasamanagedaccount— (asanaggregate

essentiallytheyareinvestableindicesaswell portfolio)

Van Hedge 1994 25 VanHedge ·NominimumtrackrecordorAUM Simplemean

Disclaimer:TheinformationandstatementsoffactsinthisarticlearebaseduponsourcesLJHGlobalInvestments,LLCbelievestobereliable,butdoesnotguaranteetheiraccura-

cy.OpinionsandestimatesincludedinthisarticleconstitutethejudgmentofLJHGlobalInvestments,LLCasofthedateofpublicationandaresubjecttochangewithoutnotice.

Figure2:Hedgefundrepresentativeindices

firm’ssetofhedgefundindiceshasbeenestablishedasthe

industry’sstandardforfundperformance.However,theindi-

cesareefficientenoughthattheycanserveasavaluabletool

forthehedgefundinvestor.Attheveryleast,currentindices

provideinvestorswithareasonablerepresentationofperfor-

mance for the hedge fund market and individual investing

strategies.

The typical set of indices published by each firm is divided

accordingtofund investmentstyle.Hedgefundsareusually

divided into several broad categories of strategy and then

classified according to more specific subtypes. Most index

providershaveestablishedanindexforeachclassificationof

hedge fund they have identified. Through the use of these

indices, investors can track with reasonable confidence the

directionality of performance for funds adhering to certain

stylesofinvesting.

investable indicesAnotherrecenttrendinthedevelopmentofhedgefundindi-

ces is the inception of investable indices. These indices are

essentiallytrackingportfoliosfollowingapassiveinvestment

approach.Theyseektoemulatetheaggregateperformance

ofindividualhedgefundstrategiesthroughcarefulconstruc-

tion methodologies and analyses. The products are geared

more toward the institutional investor and provide a cost-

effectivewaytogainaccesstohedgefunds.Currently,onlya

handfulofindexprovidersofferinvestablehedgefundindices.

Some of the more recent players in the arena include

StandardandPoors,MorganStanleyCapitalInternationalInc.

(MSCI),andFTSEbasedinLondon.

Therearemanyproponentsof investable indices,yetcritics

arguethatthesameinefficienciesassociatedwithdatabase-

producedindicesundoubtedlyfaceinvestableindicesaswell.

Aswewilldetailbelow, thereareseveral shortcomings that

investorsshouldbeawareofbeforechoosinganappropriate

hedgefundindex.

Key considerations of hedge fund indicesWhilethevariousindicesrepresenttheactualperformanceof

hedge funds to a good degree, several drawbacks exist for

theseindiceswhentheyareconsideredastruebenchmarksof

industry-wide performance, mainly associated with the fact

thatthenumbersusedtocalculatetheseindicescomefrom

the various imperfect databases, as discussed previously.

Thus,ahedgefundinvestorshouldkeepcertainthingsinmind

aboutindicesbeforeheorsheacceptstheindices’returnsas

whollyaccurate.

inconsistencies and biasesAlthough databases contain a bounty of information on

hedgefunds,therearemanydiscrepanciesbetweenthevari-

ous databases. As noted previously, information on assets,

fees,andreturnsvariesamongthedatabases.Themostsig-

nificantreasonfor thedifferencesamongdatabases is that

hedge fund managers voluntarily submit their own perfor-

mance figures to the databases. Some fund managers may

report toonlyacertaindatabase,whileothersmaychoose


investable indices promote the following benefits:

■ Faithfulrepresentationoftargetuniverse

■ Presentanaccurate,unbiasedpictureoftheuniverseof

fundsittracks

■ Definewhatitseekstotrack

■ Transparency

■ Constructedinasystematicandconsistentway

■ Public,pre-specifiedcalculationmethodology

■ Publishedconstituents

■ Accountability

■ Auditedoroverseenbyindependententity

critical questions to ask:

■ Aretheysolidpassiveinvestmentvehicles?

■ Dotheymakesenseversusactivelymanaged,tailoredfundof

hedgefundportfolios?

■ Dothefundsselectedprovidetherepresentativeselectionof

thehedgefundmarket?

■ Whatistheassetallocationstructuredtoaccomplish?Isit

equalweighted?

■ Cananinvestorbeensuredofequalrepresentationandnot

justchasinghotmoney?

Figure3:Investableindexing:Abetteravenueforinvesting?

571 Fung,W.andD.A.Hsieh,2000,‘PerformanceCharacteristicsofHedgeFundsand

CommodityFunds:NaturalversusSpuriousBiases,’workingpaper,FuquaSchoolof

Business,DukeUniversity

2 Gupta,A.,andB.Liang,2003,‘RiskAnalysis&CapitalAdequacyofHedgeFunds,’

workingpaper,CaseWesternUniversity

nottosubmittoanydatabases.Aconflictofinterestarisesin

thatafundmanagermayormaynotsubmitdataonhisor

her fund based on the quality of its performance. Not only

may the data be unreliable, but the performance figures in

databases also tend to be untimely. Hedge fund managers

reporttheirperformanceonamonthlyreturnbasis,yetthe

submission of data can lag behind by several months. This

makes for a stark difference from the continuous pricing

informationavailableforcommonstocksandeventhedaily

updating of mutual fund values. In addition, the databases

differinthenumberofdissolvedfundsthatstillappearinthe

database, which leads to a distorted view (called survivor

bias)ofthetrueperformanceofthehedgefundmarket.The

fact remains that a single centralized database containing

accurateinformationonallactiveandinactivefundsdoesnot

currentlyexist.

Due to the lackofa complete recordofhedge fundperfor-

mancedatathatgoesintoindices,therearenumerousbiases

inherenttothemethodusedtocalculateindicesfromexisting

databases. Foremost among biases associated with hedge

fundperformanceissurvivorbias,whichissonamedforthe

tendencyofdatabasestoonlypresentreturnsforfundsthat

arestillactive,asopposedtofundsthatdidnotsurvive.Asa

result,adatabaseusuallydoesnotendaperiodwiththesame

fundswithwhichitbegan.Hedgefundsaregenerallydeleted

fromdatabasesforreasonssuchasbeingmerged,liquidated,

or for halting the reporting of performance data. Although

some funds that stop reporting performance data do so

becausetheyareenjoyingexcessprofitsanddonotwantto

attractnewinvestors,itisgenerallyacceptedthatmostfunds

stopreportingtodatabasesbecauseofpoorreturnsorexcess

volatility.Thus,databasestendtobedisproportionatelycom-

prisedoffundsthathavemanagedalongtrackrecorddueto

strongreturns.Theindicesthatarethencalculatedfromthe

databasestendtohaveanupwardbiastotheirresultsdueto

theexclusionof thefundsthatdidnotsurvive.Thepositive

effectofsurvivorbiasonhedgefundreturnsisroughlyesti-

matedtobe2to3percentinrecentstudies,suchastheone

conductedatDukeUniversity1in2000.

Selectionbiasoccursindatabasesandindicesduetothefact

that not all possible funds in the industry are included in a

databaseorindex.Inessence,thishappenswhenadatabase

selects particular funds to be included in the database, or

when a fund manager decides not to submit performance

returnstocertain,orany,databases.Althoughalargenumber

of hedge funds are not represented in databases, it is esti-

matedthatselectionbiasdoesnotsignificantlyaffecthedge

fundperformancereturns.Thereasonisthatfundmanagers

arethoughtnottoreleaseperformancenumberstodatabases

becauseoftwooffsettingreasons.Somefundmanagersmay

notreporttodatabasesbecauseoftheirsuperiorreturnsand

desiretoremainoutofthepubliceye.Thus,thefundmanag-

ers that do not report because of poor returns offset the

strong performance of the other funds that do not submit

data.

Another bias in index returns is instant history bias, which

occurswhenanewfundisaddedtoadatabase.Anewhedge

fundusuallyoperatesforaperiodoftimesoitcanestablisha

performancerecordbefore itbegins tosolicitnew investors

andmarketitselftodatabases.Onceitisincludedinadata-

baseitcanthenuploaditsperformanceintothedatabasefor

theperiodoftimebeforeitwasacceptedintothedatabase.

Not only do the resulting performance figures represent an

investmentthatmaynothavebeenavailable tohedgefund

investors over that period, fund managers are also likely to

include these performance numbers in the database only

when they showed strong performance. A study at Case

Western Reserve University estimated2 that instant history

biashasapositiveeffectofcloseto1percentonreturnscal-

culatedfromdatabases.

strategy classificationOnecharacteristicoftheindicesthatvarieswidelyfromindex

to index is the classification of hedge fund styles. Although

broad similarities exist among the indices’ categorization of

funds,thespecificstylesreferredtointhedifferentdatabases

can vary greatly between different sets of indices. For

instance,onefirm’ssetofindicesisdividedamongtenidenti-

fiedstrategies,andanotherfirm’ssetof indices isbasedon

morethan30identifiedstrategies.Anotherproblemconfront-

ingtheuseofcategorizedstylesistheinabilityofoutsidersto

verify thataparticular fundmanager is strictlyadhering to

theinvestmentstyleforwhichhisorherfundiscategorized.

Itistheveryessenceofahedgefundmanagertobeflexible

inhisorherinvestmentchoices,anditmaybeimprudentto

believethatallfundsinanindexclassifiedasacertainstyle

investpurelyalongthelinesofthatstyle.Forexample,some

indicesclassifyafundaccordingtothestyleinwhichthelarg-

estpercentageofitsassetsisinvestedwhileotherindicesuse

advanced statistical techniques, such as cluster analysis, to

classify funds regardless of their stated strategy. Given the

differencesamongtheexistingindices’classificationofstyles,

itissafetosaythattherearenouniversalcategoriesbywhich

tocategorizehedgefunds.

construction methodologyAnotheraspectbywhichthehedgefundindicesdifferisthe

methodologyusedtoconstructan index.For themostpart,

indicesuseequalweightingoftheincludedfundstocalculate

thevalueof the index.However, some indicesuseanasset-

weightedmethodtocalculatetheirindex.Asthereareseveral

acceptedmethodstocalculateanindex,itisnotunusualfor

indicestousedifferentmethods.Forinstance,theDowJones

Industrial Average uses a price-weighted method while the

S&P 500 uses an asset-weighted method. The differences

betweenthemethodsaresomethingofwhichinvestorshould

remainaware.Inaddition,thenumberoffundsusedinhedge

fundindicesvariesgreatly.Setsofindicesmaydrawuponas

littleas100fundstocalculateperformancewhileothersmay

use well over 1,000 funds from a database to compute an

index. Typical numbers of funds used to compute a specific

styleindexrangefromabout20toover50hedgefunds.Asa

result, thediscrepanciesamongtheconstructionofexisting

hedgefundindicesprecludetheacceptanceofaunanimous

benchmarkforhedgefundperformance.

The future of hedge fund indicesThereareseveraltrendsatworkthatarecausingthehedge

fund industry togrowandevolveataquickpace.Primarily,

therecentincreasedpopularityofhedgefundshastriggered

asignificantcapitalinflowintohedgefundsandpromptedthe

creationofmanynewfundsandproducts.Astheequitymar-

ketshaveexhibitedincreasedvolatilityinrecentyears,many

newinvestorshavesearchedouthedgefundsasameansof

reducingtheriskexposureoftheirportfolios.Amongthenew

investorsflockingtohedgefundsarelargeinstitutionalinves-

tors,suchaspensionfundsandendowments.

Institutions have begun to place considerable weight in the

hedge fund industry by either ownership of hedge funds or

apportioningtheirclients’assetsintohedgefunds.Whilethe

largeinflowofinstitutionalmoneymaybeabonustohedge

fund managers, it promises to alter the face of hedge fund

investing at the same time. Institutions, particularly those

withafiduciaryresponsibility,suchaspensionfunds,require

greatertransparencythanwhathastraditionallybeenexpect-

edofhedgefundsbeforetheyinvesthugeamountsofcapital.

In addition to this pressure for greater transparency from

potential investors, hedge funds are also feeling pressure

fromregulatoryauthoritiesandtheInternettoincreasetheir

transparency.AgrowingnumberofInternetsitesnowreport

currentinformationandperformancefiguresforhedgefunds.

Bybeingabletoinstantlydistributeinformationtotheinvest-

ing public, the Internet is certainly working to increase the

transparencyofhedgefunds.Sincealackofinformationisat

theheartofthechallengefacinghedgefundindices,increased

transparencywillundoubtedlyservetoimprovethereliability

ofindicesandpushthemtowardcompleteaccuracy.

While index-based investing is a new development in the

hedgefundindustry,avarietyofnewproductshavestartedto

develop, such as principal-protected notes, exchange-traded

certificates, and swaps. Investors can now have index-based

investmentsstructured to fit theirneeds.While indexbased

derivativesarestill intheirearlystages,thesenewproducts

mayprovetobethenewparadigminhedgefundinvesting.


Data management in financial ser-vices 2004 and beyondAndy DilkesChief Technology Officer, unity Systems

manageable description of what may be a complex data

repository. In a typical managed meta-data environment

(MME), themeta-datadefinitionscome together tocreatea

virtualviewofbusinessinformationpresentedtotherequest-

or. A consumer initiates a request to the EII server, the EII

serverinterpretsthemeta-datarequirementsassociatedwith

therequest,acquiresthedatafromthesourcesystem(s),and

presentstheresultstotheuser.Thisamountstoavirtualdata

warehouseinthattherequestedinformationpresentedtothe

userdoesnotreallyexistinapersistedform;itisformulated

atdemandtime.ThismodelworksverywellforADHOCque-

ries,where response time isof little concernorwheredata

volumesarenotreallyaconsideration.Wheresetsofdataare

requiredtopresentormanageadashboardviewoftheenter-

priseorwhereoperationalsystemsrequireimmediateaccess

toreferenceoroperationaldatatheon-demandmodelwillbe

unabletoachievetherequiredresponsetime.ThetypicalEII

modelalsolackstheinherentcapabilitiesofaneventdriven

integration architecture such as notification and alert pro-

cessing.

Enterprise information unification (Eiu)Integrationcanbedescribedastheefforttoprovideanenvi-

ronment inwhichentitiesofdifferentstructure,characteris-

tics,andattributescanco-exist.Byitsverynature,integration

impliesthattheenvironmentorthecontainerhastoadaptto

absorb new entities into the collective representation.

Unificationimpliesanenvironmentbuiltonstandardswhere

insteadof theenvironmentadaptingtoabsorbnewentities,

entities are transformed through standards and then intro-

ducedintothecollectiverepresentation.

Theadoptionofstandardsisthekeytoaflexibleandadaptive

informationexchangeplatform.Thedrivingforcesintechnol-

ogy have embraced standards to the point where IBM,

Microsoft,Oracle,andSunallagreeonwebservicesstandards

throughXML,SOAPandWSDL.Thesecollaborationsmarkan

enormousstepforwardinengineeringtheadaptiveinforma-

tionenterprise.

Tomorrow’sinformationenterprisewillabstractdataconsum-

ers completely from underlying technology, operations, and

storage schemes. Information will be delivered in standard

formats,throughstandardservices irrespectiveofsourceor

content.

Inorder tosatisfy thevaryingdemandsof informationcon-

sumerswithinthefinancialservicesenterprise,anintegration

solution must inherently display certain characteristics that

supportahighlyadaptableandmanageableimplementation.

Thefollowingcorecharacteristicsarecriticaltoasuccessful

andadaptableintegrationimplementation.

■ inter-operability–Thesolutionshouldbecapableof

acquiringanddistributingdatairrespectiveofthetechnol-

ogyorformat.DevelopmentofrigidcodebasedAPI’s

increasesdevelopment/maintenancecosts,risk,and

deploymenttimes.Openstandardsandtechnologiessuch

asXML,WebServices,JavaMessageService(JMS),and


legacyapplication

operationalsystem

legacysystem Database

webapplication

mediator

61

Intoday’sworld,InformationTechnologyexecutivesandsolu-

tionprovidersareunderever increasingpressure todeliver

timely enter- p r i s e wide infor-

mation with decreasing

b u d - gets, aggressive

d e l i v e r y time- f r a m e s ,

andincreasing return on investment

e x p e c - tat ions.

Streamlined

b u s i n e s s p ro - c e s s e s ,

i n c rea se d c o m p e t i -

tion, economic c l i m a t e s ,

and regulatory requirements have caused

user demands and expectations for information delivery to

soar well beyond the ability of developers and providers to

respondwithtraditionaldevelopmentmethodologies.

Years of isolated application development has resulted in

numerousoperationalapplicationseachmanagingindividual

informationsilos. Integration initiativesare typicallybornof

the realization that an enormous amount of data already

exists throughout the enterprise in the form of operational

datastores,applicationspecificrepositories,data-feeds,and

transactional legacy systems. Information executives are

relentlesslypursuingintegrationsolutionstotransformthese

datasilosintointegratedinformation.

The evolution of data managementInthelate1980’sandthrough1990’s,thepushwastowardthe

all inclusive application solution for the financial services

enterprise.Themainstreamadoptionoftherelationalmodel

andsupportingtechnologyresultedinenterprisedata-models

andapplicationsolutionsdesignedtorepresentthebusiness

in an all inclusive implementation. These solutions are typi-

cally engineered to offer rich business functionality but fall

shortinsimplicity,flexibility,andadaptability.Deployingnew

linesofbusinessalmostalwaysrequirescodeenhancements

orphysicalmodelchanges.Theseinitiativesaretremendously

expensive to deploy and maintain, carry an enormous risk,

and generally fail to realize an acceptable return on invest-

ment.Technologyanduserrequirementstendtoout-runlarge

scaledevelopmentprojectsandrarelyaretheuserssatisfied

withtheresultingsolutions.

In the late 1990’s technologyexecutivesbegandisqualifying

largescalereplacementprojectsinfavorofintegrationinitia-

tives designed to leverage existing systems in an effort to

reduce cost and risk, minimize disruption to operational

departments,andrealizeatimelyreturnoninvestment.

This line of thinking resulted in a number of technologies

emergingunderthelabelofEnterpriseApplicationIntegration

(EAI) solutionsandExtract,TransformandLoad (ETL) solu-

tions.

The basis of these technologies is to provide a centralized

processingarchitecturethatbehavedasamediatorbetween

applications. In layman’s terms,a language, technology,and

formatinterpreter.Thesesolutionsworkedquitewellinterms

of functionality but proved very expensive to develop and

maintain. EAI solutions have satisfied the risk management

concerns associated with application rewrites or re-deploy-

mentsbuthavenotprovidedthecostsavings,flexibility,and

rapid return on investment in terms of cost reduction or

streamlinedimplementation.

The early 2000’s brought about the concept of enterprise

informationintegration(EII).Technicaladvancements,suchas

opendatabaseconnectivity,madetheideaofsystemsintegra-

tion throughthedatabases insteadof the transactionalsys-

tems practical. To enhance this processing and to further

abstract theusersanddataconsumers fromtheunderlying

technology, EII models typically employ a meta-data layer

thatdescribesthedataandtheirsourcesthatparticipate in

the consolidated representation. Meta-data is commonly

describedas‘datadescribingdata’andisunderstoodtobea

legacysystem

operationalsystem

operationalsystemrequest

webrequest

consumer request

Eiiserver

Databasesystem

avoidinfrastructuredevelopmentandfocusonbusiness

modeling.Applicationserversolutionsarenowcommer-

ciallyviableandshouldbeexploited.

Adherencetostandardsinconjunctionwithaflexible,adapt-

able,andoperablearchitecturewillresult inadatamanage-

mentsolutioncapableofabsorbingnewsourcedataandser-

vicing new consumers with minimal disruption and risk and

maximum return on investment. Information technologists

have come together on open standards and technology

designed to support information exchange independent of

contentandunderlyingtechnology.Thesuccessfuldataman-

agement solution will have no dependence on a particular

dataformat,content,orprotocol.Deploymentinsideanappli-

cation server framework will avoid costly, high risk mainte-

nanceoftheinfrastructureandallowtheprojecttofocuson

supporting business modeling. An application server frame-

work will also provide inherent failover and scaling support

throughclusteringcapabilities.

Viabletechnologynowexiststodevelopanopen,adaptable,

highperformancedatamanagementsolution thatcangrow

withdemandbothintermsofperformanceandscalabilityand

rapiddeployment.


JavaOpenDatabaseConnectivity(JDBC)providesupport

forinter-operabilityvoidofrigidcodebasedinterfaces.

■ Flexibility–Providersandconsumersshouldbeabstract-

ed,asfaraspossible,fromtheunderlyingtechnology.

Configurationintermsofsource,destination,andformat

descriptionsandbusinessmodelrepresentationsshould

beachievablethoughameta-datadefinitionlayerthat

allowsuserstodefinewhattodo,nothowtodoit.

Ameta-datadrivensolutionwillsupportthestandardized

representationofanybusinessinformationwithoutcode

orphysicaldata-modelchanges.

■ Performance and operability–Proprietaryprocessing

infrastructuresdesignedtosupportspecificbusiness

applicationsveryoftenenduprequiringmoredevelop-

mentandmaintenanceeffortthantheunderlyingbusi-

nesssolutionstheywerebuilttoservice.

Attheheartofmostenterprisesolutionsisabusinessdata

model built to represent information across the enterprise.

Thedatamodel,intheend,isthephysicalrepositoryinwhich

the logical business entities are stored, managed, related,

and represented. Almost always, the physical data models

(table definitions, genealogy etc.) are a direct implementa-

tionofthelogicalbusinessmodel.Youoftenhearvendorsof

pre-fabricated data models boast ‘Over 1000 tables, over

100,000columns,over5000relationships…’.Whiletheyare

obviouslyfocusingproudlyonthebreadthoftheirsolution,in

realityitmeansarigidpredefinedmodelintowhichyoumust

put your business. Successful data management initiatives

must be business driven. Flexible modeling starts with the

premisethatthephysicalimplementationiscontentunaware.

Rigidphysicaldatastructuresintroduceanundesirableman-

agement requirement to the point that implementers will

targetbusinessinformationintoaninappropriatespotwithin

thedatamodeljusttoavoidaphysicalmodelchange.

While it is clear that a physical repository is necessary to

satisfy performance, historical, and set processing require-

ments, it is also clear that a rigid fixed data model creates

adaptabilityandflexibilityissuesresultinginincreasedmain-

tenancecosts,deploymenttimes,andrisk.Atthecenterofa

unifiedinformationimplementation,arepositoryisrequired

toabsorbenterpriseinformationirrespectiveofcontentand

withoutphysicalmodification.Thismodelamounts toavir-

tualbusinessmodelstoredinsideaphysicalcontainerrender-

ingtheflexibilityofavirtualon-demandmodelwiththeper-

formance and historical capabilities of a central physical

repository.

considerations for successDatamanagementprojects,typicallyinitiatedasinformation

integration efforts, will be on the radar for virtually every

largeinformationenterpriseoverthenext3years.Thefoun-

dationonwhichthesesolutionsarebasedwilldeterminethe

critical factors inmeasuringsuccess.Consider the following

characteristicsduringarchitecturaldesign.

■ standards–Standardswilldefinethesuccessofanydata

managementimplementation.Informationexchangeis

becomingentirelybasedonstandards,suchasW3CXML

Schemas.Businessspecificstandards,suchasMDDL,

ISO15022,andXBRL,arestandardsbasedonstandards.

WebservicesviaXMLandWSDLisquicklybecomingthe

defactostandardsforwebbasedinformationexchange.

■ services based–Serviceorientedarchitecturesprovide

theisolateddevelopmentanddeploymentopportunities

whichresultinminimaldisruptionwhenaddingnewbusi-

nesslinesintothemix.Servicebasedarchitecturealso

providesreusabilityofdevelopedfunctionality.

■ openness–Avoidtechnologydependence.Today’sopen

technologiessurroundingdatabase,messaging,andvirtual

machinesallowforcompleteindependenceinoperating,

database,andmessagingtechnologies.

■ Adaptability–Avoidtherequirementtochangedata

models,code,andAPI’swhennewdatasourcesandcon-

sumersrollintothesolution.Keepuserinteractionatthe

meta-datalevel.

■ operability–Performance,reliabilityandscalabilityare

criticalfactorsindeterminingthesuccessofadataman-

agementinitiative.Datamanagementinitiativesshould

63

1 Dizdarevic,P.,andS.Shojai,2002,‘WebServices:Theenablerofthenewbusiness

serviceoperatingmodel,’JournalofFinancialTransformation,6,70-72

integrated data architecture — The end gamePredrag Dizdarevic, Partner, CapcoShahin Shojai, Director of Strategic Research, Capco

togetaholisticviewofthedataacrossthewholeorganiza-

tionisthemanydisparatesystemsthatexistwithineachfirm.

DizdarevicandShojai(20021)highlightedthemanyproblems

associatedwiththedevelopmentsofnewcapabilitiesonold

andincongruenttechnologies,whichhasresultedinasitua-

tion where most organizations have found themselves with

operational technologies that are simply an amalgam of

many systems cobbled together. ‘As volumes of trade, and

revenuesasaconsequence,increased,sodidthenumberof

distinct technologies that were introduced, making the

alreadycomplicatedshapeofthefinancialservicesindustry’s

operational IT even more spaghetti-like.’ ‘…The situation

becamesobadthatinmostinstitutionsitwasnotpossibleto

identifywhichbusinessunitsweretrulyaddingvaluetothe

bottom-line.Simple,yetmission-critical,activity-basedcost-

inginformationthatisprevalentamongmostotherindustrial

sectorsisstillnotavailabletomostfinancialexecutives.’

Webservices,anenablingtechnologyfordisparatesystemsto

interoperate,couldhelpalleviatesomeoftheseproblems.But,

ithasyettofulfillitspotential.Furthermore,evenifwebser-

viceshadachieveditsobjective,organizationsstillneededto

gettheirinternaldataarchitecturesinorderbeforetheycan

takeadvantageofthiscapabilityfromaninter-companyper-

spective.Ifwebservicesareappliedonafunctionallevelthen

theycanbeusedasawaytohidethecomplexityofexisting

environmentandtostandardizeinterfacetowelldefinedpro-

cesses.

Datawarehousingisanotherpossiblesolutionformakingdata

moreeasilyavailableacrosssilos,but this isusefulonly for

datathat isnotneeded immediately,sincethegapbetween

theoccurrenceofaneventanditscaptureandanalysisistoo

long.Withinadatawarehousingenvironment,data,whichis

typicallyspreadacrossmanydatabasesandinmanydifferent

formats, is extracted using ETL (Extract-Transfer-Load) to

anotherformatandstoredwithinasinglelocation.Whilethis

is a very useful way of gathering data in one location for

analysis purpose, there is latency problem. This batching, if

youwill,happensonceaday,week,ormonth,whichmeans

thatreal-timedataisnotavailable.Consequently,itisoflim-

itedvalueforoperationalandtacticaldecisions.

Daily batches to real-timeIn recent years, we have been witnessing silent moves into

event/real-timeprocessingofdatabecauseofthechangesin

how business is done. Real-time processing is now moving

from the front-office, where it was present for quite some

time, intobothmid-and-backofficeprocessing.Frontoffice

clientfacingfinancialexecutiveshavealwaysneededasclose

to real-time information as possible, and to a large extent,

thanks to third party market data providers, this need was

met.Internaldata(forexample,legalentitydata,clientdata,

accountdata,etc),ontheotherhand,hastypicallybeenavail-

ablewithlatency.However,theneedforcomplianceconfirma-

tion prior to trade posting or electronic trading in general

maketheneedforreal-timedataavailabilityessential.Real-

time information would also enable the risk management,


65

In recent years there has been a lot of debate about what

financialinstitutionsneedtodoinordertoimprovehowthey

compile,store,anddisseminatedatainordertoincreasethe

organization’sanalyticalandexecutioncapabilities.

Most commentators can provide a very impressive view of

whataperfectfinancialservicesfirmeco-systemcanlooklike.

Anenvironmentwheretoday’srigidandbatch-basedinforma-

tionsilosarereplacedwithafullyintegratedandautomated

structureinwhichsystemsareflexibleenoughthattheycan

bechangedwiththebusinessneeds.Thebestcasescenariois

onewheretransaction,market,andreference(bothproduct

andclient)dataareavailableinreal-timeacrossallfunctional

lines, on a single platform, so that financial institutions can

shareinformationacrossbusinesssilosandensurethatthey

haveamoreholisticviewof theirbusiness.Fromarevenue

perspective, this information can be used to improve our

understandingofaclient’srelationshipwithourentiregroup,

acrossallbusinesses,andhelpensurethatwherepossiblewe

canincreaseourshareofthatclient’swalletorbusiness.And,

of course, from a cost perspective, it can drastically reduce

costs of operation — through elimination of replication and

reductions in the costs of maintaining and communicating

amongsomanysystems—optimizeuseoffinancialresources,

andhelpensurethatweareincompliancewiththemyriadof

newregulations.

These new regulations are forcing financial institutions to

haveaccesstodatamuchmorereadilythanwasnecessaryin

thepast.Forexample,certainregulationsrequirethatmany

of the compliance checks previously undertaken during the

overnight batch processing be done in real time prior to

postingoftransactions.And,thewavesofregulationsdonot

seemtobestopping.Moreindepthknowledgeofcustomers,

to more reliable accounting data, to prevention of money

laundering,andnottomentionnewcapitaladequacyregimes

haveallmade it imperativefor financial institutionstohave

betterdatacollection,verification,dissemination,andanaly-

siscapabilities.Theseregulatoryimperativesmaketheneed

foramoreholisticviewoftheinstitution’sreference,market,

andtransactionaldataessential.Consequently,thereiseven

more of an impetus to improve the process through which

dataismadeavailableacrossthewholeorganization.

While there are no disputes that such an environment is

highly desirable, the path to achieving it has in many cases

beenoverlooked.Expertsareveryhappytodesignthefuture

butarelesscomfortableexplaininghowtogetthere,mainly

becauseitishardtoprovideadviceonanendstatewhenyou

arenotstartingfromacleanslateandneedtodealwithexist-

inglegacysystems.Thatiswhatwearehopingtodointhis

piece.

In order to achieve our objective, we will focus on the two

problemsthatwefeelareinherentwithinmostfinancialinsti-

tutions inrelationtodata,disparatesystemsandbatchpro-

cessing.Webeginouranalysiswithadiscussionoftheprob-

lemsassociatedwitheachandthendescribehowtheymight

besolvedinordertoreachtheendgamescenario.

Disparate systemsOneofthereasonsthatitissohardforfinancialinstitutions

Future

Operational

data

stores

Today

Silos (product/regions/front-mid-back) All products

Multi-multi systems One platform

Inflexible/restrict business growth Flexible

Batch/end of day/long latency Real-time

Figure1:Gettingtotheendstate

compliance, and many other departments to do their jobs

muchmoreeffectively.Ifbankshadaccesstoreal-timetrans-

actionandreferencedata,especially ifthatinformationwas

madeavailableacrosstheorganization,theywouldbeableto

managethebusinesssubstantiallymoreefficientlythanisthe

casetoday.

However,thiskindofcross-siloreal-timeprocessingofdatais

simplynotpossiblewithintoday’sfinancialinstitutions.

Moving from daily batch to real-time processing takes time,

and most financial institutions are investing large sums of

moneytoachievethisobjective.Theproblemisthatmanyare

working in their own silos, without a top-down view of how

thesedisparatesystemsaretocommunicatewithoneanoth-

er,sharingfunctionsanddataacrosstheenterprise.

solutionWebelievethatthemovetoanintegratedreal-timeprocess-

ingfinancialinstitutionwilltaketime.Itisveryhardtomake

thechangesnecessarytoachievethisobjectiveinoneclean

sweep.Itwouldcertainlybeamuchmoremethodicalprocess.

Forone thing, youcannot throwaway the systems thatare

currentlysupportingtheseinstitutions.

Thesolutionthatwesuggestisthefirstleveltoachievingthe

end-state of a single operating platform, what is known as

Operational Data Stores (ODS) (Figure 1). ODS operates by

enabling interoperability between functions that are both

insideandoutsideoftheinstitution.Thisisachievedthrough

replicationofthedatathatisbeingusedandupdatedwithin

themanysystemsspreadacrosstheorganizationorthoseit

hasoutsourcedto.

The way the ODS operates is that it captures events from

manydisparatesystemsspreadacrossabankorthein-sourc-

ingorganizationsbyusingasetofadoptersthatinterfaceinto

datastorageordirectlywiththeapplications.Aseventsoccur

theyarepropagatedinreal-timeintoanODS,whichallowsfor

datamapping,validation,consolidation,andprocessing.Once

thedataisstoredintheODSyouhaveareal-timeintegrated

view of the many different transactions that have occurred

acrosstheinstitution.

Onceallthedatamaintainedwithintheoriginalsystemsare

transferred to the ODS, direct entry of new reference and

transactiondata to theseoldsystemscanbeprohibited.All

newdataisnowstoredwithintheODS,whichmeansthatthey

will be kept within formats that allow for real-time analysis

anddissemination.Oncewereachthisstageofdevelopment

itwouldbetheODSthatsuppliesupdatestotheoriginalappli-

cations.Thisprocesscanbedoneonabatchbasisasthereis

nourgencyinensuringthatdatakeptwithintheoriginaldata-

basesisupdatedinreal-time.

Theflexibilityofthissystem,whichshouldbecomponentized

basedonbusinessfunctions,enablesdeploymentofdifferent

components in-house and to third-party providers. However,

the fact that you are able to propagate the data from its

original source in real-time means that you will still have a

holisticviewofyourdataacrossthewholeorganizationinan

event-drivenformat.

Webelievethatthisisthemostviablewayofbuildingabridge

to a fully automated real-time event-driven environment,

wherealldataisstoredinasinglesharedplatform,andenable

integratedoperations.

67

Financial

Referencedata primer1

marilyn HignettPartner, Capco

Abstract

Thisreferencedataprimerisacompositeviewofthecurrent

effortstoprovidereferencedatastandardswithinthefinan-

cialservicesindustry.Itoutlinesthesecurityindustry’srolein

thisendeavorandprovidesaprogressreport.

691 Asubstantialportionofthedatausedforthisarticlehasbeensourcedfromthe

SecuritiesIndustryAssociationsStandards&ProtocolCommittee.Thisgroupisa

sub-committeetotheSTPindustryproject.

2 InSearchofUniqueInstrumentIdentifier,byReferenceDataUserGroup(RDUG)&

ReferenceDataCoalition(REDAC),June2003


traded.Howeverasinternationaltradinghasgrown,sohave

thenumberofdifferentidentifiers.Today,‘therearecurrently

65nationalnumberingagenciesissuingmorethan20differ-

ent local codes.’ The closest standard currently available is

thatofISIN,howeveroneissuewithusingtheISINisthatit

couldbeused innumerous locationswherethereareminor

differencesinthecharacteristicsofasecurity.

Thelackofreferencedatastandardsleavesfirmstodecideon

theirownwhattostoreandhowtoformatthedataforpro-

cessing.Somestandardshavebeeninformallyadoptedbythe

industry (ISO15022) due to the requirements of commonly

used facilities like Swift. Unfortunately multiple bodies exist

which produce industry data and suggested formats. Firms

decidewhichstandardsorevenwhichversionofastandardto

adopt. A sampling of standards organizations currently in

operationisprovidedintheAppendix,whichisbynomeans

exhaustive.Inadditiontothoselisted,therearemanyassoci-

ated organizations that influence the work of these groups.

The relationship between these organizations is complex.

Finally,mostarepeerorganizationswherestandardscanonly

bedevelopedthroughmutualagreement.

The securities industry Association’s roleInthemistofallofthisactivity,asafocusoftheSecurities

IndustryAssociation’s(SIA)STPproject,asub-committeewas

formedtoidentifytheprimaryreferencedataproblemsfacing

theindustrywithregardtostraightthroughprocessing.The

sub-committeewastaskedwith identifyingthecurrent land-

scape,assessingtheroleofreferencedatastandardsinfacili-

tating STP, identifying the scope of the relevant standards

basedinitiativesneededorunderway,anddefiningandunder-

standingtheglobalstandardsprocessandSIA’srole.

Thesubcommittee’sfindingsandrecommendations included

thefollowing:

■ TheISOstandardsdevelopmentprocessisbothviableand

functional.

■ Therearefourstandardsactivitiescurrentlyunderwayin

threecoreareasthat,ifsuccessful,willprovideacommon

dataandprotocolinfrastructureforsecuritiesprocessing

automation.Theobjectivesarerealandachievable.

■ TheSIAshouldpubliclysupportthesestandardsactivities

asbothanorganizationandasaparticipantintheglobal

ReferenceDataCoalition(REDAC),provideoversightto

theISOstandardsdevelopmentprocesstoensurethatthe

initiativesremainontasktodelivervalue,andencourage

itsmemberstodirectlyparticipateinthestandardsdevel-

opmentprocess.

Withthegoalofobtainingacommonmarketdatainfrastruc-

tureforsecuritiesprocessingautomation,fourobjectivesfor

referencedatastandardsweredeveloped:

■ Identifyallfinancialinstrumentswithprecision.

■ Identifyallbusinessentitiesforprocessingefficiency,

regulatorycompliance,andriskmitigation.

■ Identifyalldataelementsassociatedwithafinancial

instrumentlifecyclewithabsoluteprecision(standard

terms,definitions,andrelationships).

■ Defineacommondistributionprotocolforefficientand

accurateprocessing.

Potential solutionsSpecifically,intermsofproductandclientreferencedata,the

groupidentifiedthreeissuesthatneedtobepursued,which

werethelackofauniquesecurity,business,anduniquefund

identifiers. These three identifiers are discussed further


Figure1:Reasonswhydatamaybedifferentindifferentinternalclientdatabases

Errors

spelling errors

Abbreviations

word re-ordering

noise words

Date format errors

Additional data

Data in wrong field

invalid data

Extract data

missing data

Examples

• MorganStanleyvsMorganStanly

• AAAMvsABNAMROAssetManagement

• A/C2090JPMorganAmerAirvs

JPMorganAmerAirA/C2090

• MrJohnSmithvsDrJohnSmithBsc.Jnr.

• 1/12/01vs12/1/2001

• CapcovsCapcoA/C786780

• Postcode/ZIPcodeinCityfield

• Addressdoesnotmatchpostcode/ZIPcode

• Countrymissing


71

The problem…Referencedatahasbeenalong-standingprobleminthefinan-

cialservicesindustry.Poorreferencedatacanimpactafirm’s

abilitytomeetregulatoryrequirements,tomonitorcounter-

partyandcreditrisk,andachieveSTP.

The results of the September 2002 TowerGroup Survey: ‘Is

the Securities Industry Making Progress on Reference Data

Management?’ are well known and include the following

soundbites:

■ 40%ofdataelementsfoundwithinapostexecutiontrade

recordareculledfromstaticreferencedatabases.

■ Almosthalfofallexceptionsaretheresultofincomplete,

non-standardized,orinaccuratereferencedata.

■ Thegreatestsourceoftheproblemsinautomatedtrades

processingarethecodesusedtoidentifylegalentities.

■ Morethan1in5brokersconsiderdatamanagementatop

priorityfortheirfirms,while59%ofcustodiansassignit

atleastahighpriority.

Thereareanumberofreasonswhythereferencedataprob-

lempersists,includingthelackofformalstandardsforfirms

tofollowandthemeansbywhichthesedatastoreshavebeen

developed.Manyfirmshavedevelopedtheirinternalsystems

inproductorlineofbusinesssilos,witheachcontainingdata-

bases for product and client related data. As a result, data-

basesoflikeinformationcanbespreadacrossthefirm.While

aprimarydatabasemayexistinthefirm,thereisnoguaran-

teethateverydatafieldorvaluerecordedinthesilo’eddata-

basesalsoexistsinthesameformatintheprimarydatabase.

With regard to client related data, the same values may be

recordeddifferentlyineachdatabase.Manualdataentry,data

manipulation,limitedvalidation,andeditcriteriaallcontribute

toinconsistentdata(Figure1).

Reasons for data inconsistenciesThe pace at which new regulatory requirements have been

introduced only exacerbates the problem. The Patriot Act

requiresfirmstoknowtheirclientsacrossassetclasses,geog-

raphy,andlegalentities.Gleaningandnormalizingthisdatato

monitormoney-launderingactivitiesbecomesamonumental

task.Inaddition,thenewAgencyLendingDisclosureinitiative,

whichwill requireagents todisclose theirunderlyingprinci-

pals to broker/dealers for stock loan transactions will intro-

duceanothersourceofdatatobeaggregatedforcreditrisk

analysisandcapitalrequirementscomputation.

Similar problems exist in the product area. Security data,

while more standardized, still encounters problems with

regard tosecurity identifiers.While therearemanysecurity

identifiers, there isnounique identifierthat facilitatesauto-

matedprocessing(Figure2).

Theuseofmultiple identifiers for thesametransactionfur-

thercomplicatestheproblemandrequiresfirmstomaintain

cumbersome cross-reference tables. In addition there is no

oneidentifierthatsupportstradinginmultiplelocations.This

isacriticalproblemforfirmsprocessingsecurities listedon

multipleexchanges.

A discussion paper published by the Reference Data User

Group (RDUG), and the Reference Data Coalition (REDAC)

states:‘Thecoreproblemisthattherearetoomanysecurity

identification numbers, but none that uniquely identify all

attributesrequiredforprecision.’2Historicallysecurityidenti-

fiers were assigned by the location in which the securities

Figure2:Useofdifferentidentifiersatdifferentstagesofthetrade

Source:ReferenceDataUserGroup(RDUG)&ReferenceDataCoalition(REDAC),

DiscussionPaperinSearchofUniqueInstrumentIdentifier,June2003

CUSIP CUSIP CUSIPISIN+

PSET

ISIN/

CUSIP

SEDOL SEDOL SEDOL SEDOL

Ticker/

ReutersRIC

Issuelevel

OPOL

Placeoftrade

(Exchange)

Trading Allocation Confirmation Settlement Reconcile

Lev

el o

f u

niq

uen

ess


entitieswillflowsmoothlyduetotheuseofproprietarydata

formats.

Inaddition,giventhepaceofregulatorychangesintheindus-

tryitappearsasthoughfirmswillalwaysbeonestepbehind

inanticipatingandobtainingthedataneededtosatisfyregu-

lators and the new industry initiatives. Tower Group esti-

matesthatafirmcouldspendU.S.$10milliontoU.S.$50mil-

lionto implementafirm-widedatastandardand infrastruc-

ture forpublishingdatacontentdependingonthescopeof

theproject.3Thatrepresentsamajorinvestmentbyfirmsin

an area that provides no particular business advantage.

Common indicativesecurityandpricing information isused

acrossthe industry. Itwouldseemlogicalthatfirmslookto

vendorutilitiestoprovidethisinformation.

Products currently available offer solutions covering data

models, data feed handlers, data warehouse products, data

outsourcingservices,anddatacontent.

DatacontenthasrecentlybeenofferedbyDTCCintheformof

itsGlobalCorporateActionsservice.Anumberofcommercial

providersdoofferanumberofservicesinthisspace—suchas

FTI,Cicada,Eagle,Fame,Capco,WiproandTAP—howeverno

oneprovideroffersalloftheservicesacrossalloftheasset

classes that most firms require. As a result, even if firms

desiretoadoptavendorsolution,someproprietarydevelop-

ment and maintenance of internal applications will still be

requiredatthistime.

Whilesomeelementsoffirmclientdataarecommonacross

the industry, it is less likely that firms will look to external

vendorstohouseandprovidethisdata.RecentlyOmgeohas

proposedholdingthedeliveryinstructionsfortheindustryto

facilitatesettlementprocessing.Thispropositionhasmetwith

a considerable amount of resistance. It is more likely that

someelementsofcounterparty,aswellasregulatory(Patriot

Act),datawouldbeheldcentrally.DTCChashadsomediscus-


Figure3:REDAC/RDUGEntityIdentifierFramework

Source:StandardsOverview,Asynopsisofthestructureandcurrentchallengesofstandardreference

dataandmessagesimpactinginstitutionalSTP,SecuritiesIndustryAssociation,August14,2003.

Client and counterparty data illustrative

XYZ Holdings (uK)

FundA

FundB

FundC

Germanequities

U.K.equities

FundA

FundB

FundC

Germanequities

U.K.equities

FundB

U.K.gilts

J.S.governments

H.K.equities

Certofdeposit

XYZ Trading XYZ n.V.

XYZManagement&Research

FundD

XYZInternationalU.K.

FundF

XYZInternationalLuxembourg

FundE

Parent

subsidiaries

Business/legalentity

Fund levelinformation

settlement instructions• by market• by instrument

Business entity information (BEi)

Parent–subsidiaryrelationships

Riskrelatedissues

REDACfocus

Fund identifier

legal entity information (lEi)

STPrelatedissues

RDUGfocus


below.

In order to provide a unique product identifier, RDUG and

REDACsuggestusinganOfficialPlaceofListing(OPOL)iden-

tifiertoindicatethemarket,country,andplaceoftradeindica-

tor to identify theplaceof listing.Asapotential framework

thefollowinghierarchyhasbeenputforth:

■ unique issue identification (isin) –importantforsecuri-

tyroll-up,riskmanagement,overallpositionkeeping,and

tradingdecisions.

■ unique instrument identification at the official place of

listing level (oPol) –importantforportfoliovaluation,

fungibilityissuesformulti-listedinstruments,corporate

actionissuesacrossOPOLs,riskacrossOPOLs/currencies,

positionkeeping,settlement,VMUinteraction(cross-

bridgesettlement,SSI),andarbitrageacrosscurrencies,

markets,andOPOLs.

■ Trading identification at the place of trade level –

importantforintra-daypricingdecisions,arbitrage,and

marketcompliance.

Severalsolutionshavebeenputforthincludingthefollowing:

■ lsE sEDol–ExtendSEDOLbyallocatingSEDOLatthe

officialplaceoflistinglevelwithlinksprovidedbetween

SEDOLandISIN.TheLondonStockExchangewould

establishthesecodesinrealtimeandassociatetheMICto

identifywheretheinstrumenttrades.

■ AnnA service Bureau (AsB)–NewOPOL+ISIN+MIC–

AddofficialplaceoflistingandplaceoftradeviatheMIC

(MarketIdentifierCode,ISO10383)totheISINnumber.

Currentlynoonesolutionhasbeenagreedtobytheindustry.

With regard to client information, two problems have been

identified,thelackofamastercompanyidentifier(BEI)linking

legalentities(parenttosubsidiaryrelationships)andthelack

ofacommon industry-widefund level identifier (LEI).These

identifiers are used for identification of counterparties on

transactions,counterpartyand issuerriskmanagement,col-

lateralmanagement, legalagreements,regulatoryreporting,

andresearch.

Fundidentifiersareneededbyinvestmentmanagerstofacili-

tatetheallocationprocess.Lackofthese identifiersdirectly

impactstheabilityoffirmstoaggregatedataforriskcalcula-

tionsandknowyourclientregulations.Theabsenceofstan-

dardizeddataimpactsafirm’sabilitytoprocessbusinesswith

externalpartiesregardingbusinessentities,funds,andstand-

inginstructions.

IndustrystandardidentifierslikeBIC,MIC,CountryCode,and

DUNSnumberdonotsolve theproblemduetosomecodes

being proprietary and some firms utilizing more than one

code. As a result, many firms rely on internal coding tech-

niques.REDACandRDUGarereviewingthisproblemandhave

laidoutaframeworkasapotentialsolution(Figure3).

Severalstandardsgroupsareworkingonpotentialcombina-

tionsormodificationsofexistingcodestoidentifyBEIandLEI.

While these proposals have generated interest, no standard

hasbeenagreedtobytheindustry.

Althoughnotdiscussedindetailinthisarticle,thesamelevel

of effort is currently being placed on identifying a common

data model with common data element names and formats

forreferencedataaswellasastandardcommunicationspro-

tocoltotransmitinformationtoexternalentities.

what is a firm to do?Firms could tackle these issues internally by enforcing the

useofone‘goldencopy’database,oneproprietarydatabase

distributed to other sources, or developing an abstraction

layerovermultipledisparatedatabases.Theexpenseinvolved

inadditiontothepotentialdisruptiontoassociatedsystems

has served as a major deterrent to undertaking this work.

Much of the cost incurred in these efforts revolves around

datamigrationwheredifferentdatabaseshavedistinctdata

structures,duplicatedata,staleandincorrectdata,andfree

format text.Evenwhen firms takeon thiswork there isno

guarantee that the data communicated to external

733 Lind,T.,2003,‘VendorSolutionsforManagingReferenceData:KeepingtheFaith

inaSlowMarket,’byTimLind,TowerGroupResearchNotes,December


utilitiesseemtobeslowincoming.



sionsinthisareabutnoindustryeffortsareunderway.

conclusionReference data is a long-standing problem in the financial

services industry. Everyone is concerned with it and many

groups are working on it. Although there is some progress,

long-term and efficient solutions with respect to common

75

name Description (charter/mission) Geographic focus

ANNA(AssociationofNationalNumberingAgencies) RegistrationandmaintenanceauthorityforISINandCFI.ANNAcreatedtheANNA Global

ServiceBureau(ASB)inJuly2001toactasacentralhubtoreceiveandconsolidate

ISINdatafromANNAmembers.

ANSI(AmericanNationalStandardsInstitute) Promoteandfacilitatevoluntaryconsensusstandards(Financialandnon-financial). U.S.

ComprisedofAccreditedStandardsCommittees(ASC).ASCX9isofficialU.S.representative

committeetoISOforFinancialIndustryStandards.X9DissubgroupofASCX9devotedto

securitystandards.X9Dcomprisedofdomesticworkinggroups(e.g.,WG10,CFI)thatprovide

inputtoISOworkinggroups.

D&B(Dun&Bradstreet) WhileDun&Bradstreetisaproviderofcreditreports,italsoissuesaproprietarylegalentity Global

identifier,calledaDUNSnumber.

ISDA(InternationalSwapsandDerivativesAssociation) TheInternationalSwapsandDerivativesAssociationistheglobaltradeassociation Global

representingderivatives.ISDAwascharteredin1985.

ISITC-IOA(InternationalSecuritiesAssociationfor Thegroup’smissionistofosteralliancesandadvocatestandardsthatpromoteSTPof Global

InstitutionalTradeCommunication- securitiestransactionsbyelectroniccommunication(trades,reconciliations,corporateactions)

InternationalOperationsAssociation) betweenfundmanagersandcustodianbanks.Includesbrokers,custodians,investment

managersandvendors.Workinggroupsprovideassistanceinanalysis.

ISMA(TheInternationalSecuritiesMarketAssociation) TheInternationalSecuritiesMarketAssociation(ISMA)istheself-regulatoryorganization Global

andtradeassociationfortheinternationalsecuritiesmarket.ISMAoverseestheefficient

functioningoftheinternationalsecuritiesmarketthroughthecodecoveringtrading,

settlementandgoodmarketpractice.

ISO(InternationalOrganizationforStandardization) TheInternationalOrganizationforStandardization(ISO)isaworldwidefederationofnational Global

standardsbodiesfrommorethan146countries,onefromeachcountry.ISOisa

non-governmentalorganizationestablishedin1947.ThemissionofISOistopromotethe

developmentofstandardizationandrelatedactivitiesintheworldwithaviewtofacilitating

theinternationalexchangeofgoodsandservices,andtodevelopingcooperationinthe

spheresofintellectual,scientific,technologicalandeconomicactivity.

ISSA(InternationalSecuritiesServicesAssociation) Topromoteprogressandtransparencyinthesecuritiesservicesindustry,openup Global

communicationchannelsbetweenanddeveloppersonalcontactsamongsecuritiesservices

providers,increasetheprofessionalknowledgeofsecuritiesindustryparticipantsandthe

investmentcommunityandworktogetherwithotherfinancialsectorindustryorganizations.

LSE(LondonStockExchange) LSEistheregistrationandmaintenanceauthorityforSEDOL. UKandglobal

RDuG(ReferenceDataUsersGroup TheReferenceDataUserGroup(RDUG)isaforumforleadingmembersoftheglobal UKfocus

securitiesindustrytodiscussandidentifypracticaldatamanagementsolutionsthatwillhelp

achievehigherratesofstraightthroughprocessing(STP).

REDAC(ReferenceDataCoalition) TheReferenceDataCoalition(REDAC)isaninternationalcoordinatingbodyofbroker/dealers, Global

investmentmanagers,custodianbanks,depositoriesandothersdesignedtodefinethedata

elementsandstandardsnecessarytopreciselydescribetheassetsandaccountentries

requiredtomakeglobaltradeprocessingmoreefficient.REDACisfacilitatedbyFISD.

S&P(Standard&Poor’s) WhileStandard&Poor’sisamarketdatavendor,itisalsotheregistrationauthorityfor Global

CUSIP(assignedcontractbytheAmericanBankersAssociation).

SWIFT(SocietyforWorldwideInter-bank SWIFTisanindustry-ownedcooperativeestablishedin1975.WhileSWIFTownsandoperates Global

FinancialTelecommunication) itsownnetwork,itisalsotheregistrationandmaintenanceauthorityforBIC,MICandthe

ISO15022DataDictionary.TheSWIFTcooperativeismandatedbyitsownerstoplayaleading

andneutralroleinthesupportofstandardsconvergenceforthefinancialindustry.

Telekurs Financial TelekursisamarketdatavendorandalsoisresponsiblefortheissuingofSwissvalor Global

numbers(localsecurityidentifier).TelekursisalsoafoundingmemberoftheAssociation

ofNationalNumberingAgencies(ANNA).

W3C(WorldWideWebConsortium) W3CwascreatedinOctober1994toleadtheWorldWideWebtoitsfullpotentialby Global

developingcommonprotocolsthatpromoteitsevolutionandensureitsinteroperability.

W3Chasaround450memberorganizations.

Financial

Data in financialinstitutions

Richard mclaughlinSolicitor, Technology,

Media and Telecommunications Department, Nabarro Nathanson solicitors

Abstract

Financialinstitutionsareclearlyheavilydependentontrans-

mission and storage of data and are likely to become even

moresointhefuture.Thisarticleconsiderssomeoftheissues

facedbyfinancialinstitutionswithregardtoitstreatment.For

reasonsofpracticalityIhavelimitedmyanalysistoproviding

onlyanoverviewofwhatisavastandcomplexsubjectmatter.

77


mindalthoughtheystillapplytothem.Applyingsuchgeneral,

nonindustry-specificlegislationacrosstheboardclearlyhas

the potential to cause difficulties as is the case with many

types of legislation which attempt to be all-encompassing.

Thisisdiscussedfurtherbelow.

U.K. and E.U. legislation on data protection, electronic com-

merce,electroniccommunications,andevenhumanrightsall

have a potential or actual impact on financial institutions.

Specifically relating to financial institutions, the Basel II

accordhasseriousdata technologycompliance implications

whichneedtobediscussed.

Basel iiBackground

In1988theCommitteeonBankingSupervisionoftheSwiss-

based Bank for International Settlements established the

regulatory framework for attempting to ensure that banks

were sufficiently capitalized. This regulatory framework

becameknownastheBaselCapitalAccordandisnowapplied

widely by financial institutions throughout the world. This

BaselCapitalAccordwaslatermirroredinE.C. legislationas

the Investment Services Directive and has now been imple-

mented into U.K. law under the ultimate supervision of the

FinancialServicesAuthority.

Aswellassettingminimumamountsofcapitalreserveswhich

financial banks must hold, the first Basel accord outlined

mechanisms for defining credit risk. However, these risk

assessment models had become dated in view of advance-

mentsinfinancialservicesandtechnologyanditwasconsid-

eredthatnew,moreflexible,provisionswererequired.

Basel II was, therefore, agreed as an update to the original

BaselCapitalAccord.Itappliestoallfinancialinstitutionsand

haspotentiallyhugeramificationswithrespecttoinformation

technology(IT)systemsinsuchinstitutions.Accordingtothe

Chairman of the U.K.’s Financial Services Authority, Callum

McCarthy,speakingataconferenceinlate2003,itisexpected

that Basel II will be implemented in the U.K. by the 31st

December2006asscheduled,thoughhesoundedawarning:

‘The2006datewillrequireveryhardwork,detailed

consultationandcloseandproductiveworkingbetween

theFSAandbanks.’

summary of main provisions of Basel ii

Basel II is based on three mutually reinforcing pillars. This

wordingimpliesthateachofthepillarsistobeaccordedequal

importance.Theyare:

■ minimum capital requirements–Thesearetobemea-

suredusinganenhancedframeworkformeasuringrisk,

includingmarket,credit,andoperational.

■ supervisory review process–Sufficientinternalprocess-

esmustbeputinplacetoallowmonitoringandself-

assessmentofcapitaladequacy.Supervisorsmustinter-

veneintheeventthatafinancialinstitution’srisk

becomesgreaterthanitscapitalreserves.

■ Effective use of market discipline–Thisisintendedto

encouragesoundbankingpracticebyeffectivedisclosure

byinstitutions.Thiswillallowpotentialinvestorstoassess

whetheraninstitutionhassufficientreservesandrisk

managementsystemsinplace.

However,aswellasitsITimplications,BaselIIgoesrighttothe

heartofcorporategovernancestrategy.Itisessentialthatthe

processesforachievingBaselIIcomplianceareadoptedatthe

highest level within financial institutions to ensure buy-in

throughout the organization and subsequent successful

implementationandmanagementoftheprocesses.

Aswithanylarge-scaleITproject,thecurrentstatusofdata,

systems,andtheirabilitytoprocessdatamustfirstbeaccu-

ratelydeterminedbefore future requirements canbeascer-

tained. This can clearly be a time-consuming and expensive

process; however the efficiency of the implementation pro-

gramwillbegreatlyenhancedifaclearstartingpointisestab-

lished.Inrelationtodata,anydataauditshouldcompriseat

least:



79

Thesubjectofdatasecurityandretentioninfinancialinstitu-

tions is one that has been increasing in importance for

severalyears.Withtheadventoftheuseoftechnologyinthe

financialsector,regulationsandguidanceregardingprocess-

ing,security,andretentionofdataweredrawnup.Astheuse

of technology has become more widespread the guidance

and regulations have necessarily attempted to keep pace

andinsomecaseshavesoughttopre-emptfuturedevelop-

ments.

Todaythevastmajorityofoperationscarriedoutbyfinancial

institutionstakeplaceelectronically.Thishasledtoamyriad

oflegislationonthesubjectinanindustrywhichisalreadythe

mosthighlyregulatedintheworld.Financialinstitutionsmust

keep abreast of all of the requirements and this can be an

extremely time-consuming and expensive exercise — hence

theincreasedimportancesuchinstitutionsareplacingonthe

issue.

security of dataSecurity of data is clearly a particularly sensitive issue for

financial institutions. The institutions have always been

attractive targets for crime due to the high volume of high

value transactions they process. Threats to data can poten-

tially emanate from anywhere — for example from hackers,

dishonestemployees,generalfraud,orsimpleusererror.

Hackers attack computer networks by techniques such as

password cracking, exploiting known security weaknesses,

and network spoofing. Network spoofing is where a hacker

presentsasystemtoanetwork.IfsystemA‘trusts’systemB

andsystemC‘spoofs’systemBthenaccesscanbegainedto

systemA.Spoofing reliesononeof thesystemsbeingshut

downformaintenancesothatmessagesarenotreceivedbyit

andareinsteadpickedupbythebogussystem.

Despitethesecuritymeasuresalreadytakenbyinstitutionsa

recent survey (Deloitte Touche Tohmatsu 2003) found that

only35%offinancialinstitutionswere‘somewhatconfident’

that their organization was safe from external threats.

AnothersurveybyanalystfirmTowerGroupfoundthatpublic

confidenceinrelationtoonlinebankingactivitieshaschanged

little in the past year despite the continued investment in

securitymadebyfinancialinstitutions.

It isgenerallyaccepted that thegreatest threat to security

faced by financial institutions comes from ‘insider attack’.

The insider is any person who has legitimate access to the

systemandisabletouse,destroy,ordisseminatedatatothe

institution’sdetriment.Thisinsiderattackisthemostdifficult

tocounter,astheperpetratorsdonotneedtobreachsecu-

ritymeasuresandoftenhaveagreatdealoftimeinwhichto

carryouttheirillicitactivities.Theuncertainprovidenceand

natureofthesethreatsonlyservestoheighteninstitutional

fear.

Beingabletodemonstratethatdataissecureisalsoincreas-

ingly seen as a positive message to give out in marketing

terms.Institutionsthatcanpersuadecustomerstouseonline

facilities can make substantial cost savings. However, there

remainsmuchresistancefromconsumersduetoaperceived

lackofsecurity.Whileinstitutionsprovideinformationrelating

toSecureSocketLayer(‘SSL’)encryptioninordertoreassure

their customers, it is not as effective as expected mainly

becausetheconsumersdonotunderstandenoughaboutthis

technologytobereassuredbyit.Theoftenconvolutedtechni-

calnatureofthedescriptionsofthistechnologydoesnothelp

alleviatethecommunicationimpasse.

Retention of dataTherequirementsimposedonfinancialinstitutionsrelatingto

dataretentionarealsoincreasingexponentially.Thisislead-

ingtoincreasingfrustrationfrominstitutionsastheystruggle

tocopewiththeonslaughtoflegislation.Moreworryinglyfor

institutionsisthattheyareoftennotevenawareofallofthe

requirementswithwhichtheyareexpectedtocomply.

TheU.K.andE.U.haveintroducednumerouspiecesoflegisla-

tion concerning data privacy and retention. Many of these

were not enacted specifically with financial institutions in


betakenagainstunauthorizedorunlawfulprocessingof

personaldataandagainstaccidentallossordestruction

of,ordamageto,personaldata.

8.Personaldatashallnotbetransferredtoacountryor

territoryoutsidetheEuropeanEconomicArea,unlessthat

countryorterritoryensuresanadequatelevelofprotec-

tionoftherightsandfreedomsofdatasubjectsinrelation

totheprocessingofpersonaldata.

Clearlyeachoneoftheseprinciplescandediscussedatsome

lengthandthereisnodoubtthatthekeyfactoristhemanner

inwhichtheprinciplesareinterpreted.

However,onapractical level thereare identifiableelements

whichorganizationscanputinplacetoassisttheminstriving

fordataprotectioncompliance.Thesecanbesummarizedas

follows:

■ Encryption of data –thisisvitallyimportantinfinancial

institutionswhereextremelyhighvaluetransactionsoccur

daily.

■ clear and strong security policy–themorecomprehen-

sivethesecuritypolicyproceduresarethelesschanceof

breachesofsecurity.Thisdovetailswithriskmanagement

proceduresrequiredbyBaselII.Highqualitytrainingwill

ideallyensurethatsuchsecurityproceduresbecome

secondnatureforstaff.

■ Targeting security resources effectively–perhapsan

obviouspoint,howeverparticularlyhigh-riskareasshould

beidentifiedand,ifdeemednecessary,additionalresourc-

esorprocedurescanbeaimedatthem.

■ Data integrity–procedureswillideallyexisttoallow

randomtestingofdatatoidentifyanytampering.

It is clear that data protection and security are not exact

sciences.Theyareameasureofdegreeandcommercialreali-

tieswillalwaysdictatethatabalancemustbestruckbetween

security and business efficiency. In any event, it is widely

acceptedthatnosystemiscompletelysecureandthismust

alsobeborneinmindwithrespecttoriskmanagement.

Thefollowingtableshowssomeofthelegislation,regulations,

andguidancewithwhichcompaniesmustcontendinattempt-

ingtocomplywithdatarequirements:



■ Determiningexactlywheredataisheld.

■ Assessingwhatformatdataisheldin.

■ Establishingwhatsystem(s)thedataisheldon.

■ Assessingthequalityofthedata.

■ Assessingwhetherthequalityofthedataisconsistent

acrosstheorganization.

These processes are necessary to attempt to achieve com-

pletelycleandata.Onlysuchcleandatacanprovidethefoun-

dationfromwhichaccurateriskanalysiscanbecarriedout.It

will also be essential that systems are able to interact with

eachotherandthatnocorruptionordiminutionofdataqual-

ityoccursduring transferofdatabetweensystems.Further,

dataprovidedforriskanalysismustclearlybeup-to-date.

Onapositivenote,althoughachievingBaselIIcomplianceis

undoubtedly onerous in terms of man-hours, it is generally

thoughtthatcompliancecanbeachievedthroughtheuseof

existingtechnologiesandavailableskills.Asdatagoingback

2-5yearsisrequiredforsomeformsofcreditriskcalculation

(AdvancedInternalRatingBasedor‘IRB’modelforexample),

prudentfinancialinstitutionshavealreadybeguntheprocess

ofBaselIIpreparation.

The complex web of data legislationBusinessleadersarebecomingincreasinglyconcernedabout

theever-growinglistoflegislativeandregulatorydatarequire-

mentswithwhichtheymustcomply.Ascanbeseenfromthe

table below, there are numerous pieces of legislation which

apply,orcanpotentiallyapply,wheredataisconcerned.Some

oftheseactsoverlapinareasandtheoverallsentimentfeltby

thebeleagueredcompanydataprotectionofficerisoftencon-

fusion. Various industry bodies and companies have made

theirfeelingsonthesubjectclearandhaveharshlycriticized

the principal piece of U.K. data protection legislation — the

DataProtectionAct1998.

In fact, even the U.K.’s Information Commissioner, Richard

Thomas,theheadofthebodywhichoverseesU.K.datapro-

tectionrecentlystated:

‘LastyeartheDataProtectionActwasdescribedinthe

CourtofAppealas‘acumbersomeandinelegantpieceof

legislation’.Iagree.’

The Information Commissioner regularly attempts to clarify

various data protection issues by issuing official guidance,

howeverashehasstateditisimpossibletoprovideguidance

foreverypotentialeventuality.AccordingtotheInformation

Commissioner:

‘Thedataprotectionprinciplesarelargelymattersof

commonsenseandfairnessbutdataprotectioncannever

beasetofdetailedDo’sandDon’ts.Organizationsmust

usetheirownjudgmenttobalancewhattheywantor

needtodoagainsttheneedtosafeguardtheprivacyof

individualsandtoensuretheirpersonalinformationis

handledproperly.’

This is perhaps not considered to be especially helpful by

largeorganizationsthatareinundatedwithdatarequestson

adailybasisandmust fund the resources tocopewith this

additionaladministrativeburden.

The U.K. Data Protection Act applies to any entity which

qualifiesasadatacontrollerordataprocessorandisbased

aroundeightgeneralprinciples.Theyare:

1.Personaldatashallbeprocessedfairlyandlawfully.

2.Personaldatashallonlybeprocessedforlimitedspecified

purposes.

3. Personaldatashallbeadequate,relevant,andnotexces-

siveinrelationtothepurposeorpurposesforwhichthey

areprocessed.

4.Personaldatashallbeaccurateand,wherenecessary,

keptuptodate.

5.Personaldataprocessedforanypurpose…shallnotbe

keptforlongerthanisnecessary.

6.Personaldatashallbeprocessedinaccordancewiththe

rightsofdatasubjects.

7. Appropriatetechnicalandorganizationalmeasuresshall

81

legislation/regulations comments

DataProtectionAct1998 Placessignificantobligationson

organizationswhichprocessdata.

ElectronicCommunicationsAct2000 ImplementspartoftheElectronic

SignaturesDirective99/93/EC.

ConsumerCreditAct1974 Thisprovidesthatcreditagreements

mustbeinwriting.Consultationis

underwaytoallowelectronic

communication.

FinancialServicesandMarketsAct2000 Provisionsrelatingtofinancialpromotion

arerelevanttoe-commerce.

RegulationofInvestigatory Thisregulatesinterceptionofemails

PowersAct2000 andotherformsofcommunication.

ComputerMisuseAct1990 Createsoffencesrelatingtousinga

computertoobtaindatawithout

authorization.

HumanRightsAct1998 Providesmoregeneralprovisionsrelating

toprivacy,etc.whichcanoperatein

conjunctionwiththeDataProtectionAct

1998.

CopyrightDesignsandPatentsAct1988 Protectsdataagainstcopyingand

unauthorizeduse.

FreedomofInformationAct2000 Thiscreatesageneralrightofaccessto

informationheldbypublicauthorities.

CriminalJusticeAct1988 Obligationstodisclosedatarelatingto

DrugTraffickingAct1994 suspicioustransactions.

TerrorismAct2000

InternationalFinancialReporting Financialdatamustbeprovidedinthe

Standards formatsetoutbythestandards.

FinancialServicesAuthorityOperational Policiesshouldbeinplacetoprotectdata.

RiskSystemsandControlGuidelines

InformationCommissioner’sCodeon AttemptstoclarifyaspectsoftheData

Employers’MonitoringPractices ProtectionAct.

BaselIIAccord Necessitatesadvancedsystemsfordeal-

ing

withdata.

PrivacyandElectronicCommunications Datasubjectsmustbemadeawareof

Regulations2003 theirdatabeingprocessed,amongother


u.s. issuesAlthough it isgenerally thought that theU.S.has lessstrin-

gent data protection controls than in Europe (indeed under

theU.K.DataProtectionActdatacannotbetransferredfrom

theE.C.totheU.S.unlesstoanorganizationcomplyingwith

‘safe harbor’ standards), there are some specific pieces of

legislationwhichbuckthistrend.

The sarbanes-oxley ActThislegislationwasenactedfollowingvariousfinancialscan-

dals intheU.S.,suchasEnron. It imposessignificantobliga-

tions on all companies listed on either the New York Stock

ExchangeorNasdaq,evenforEuropeancompanieswhichare

notbasedintheU.S..Publiccompaniesmustmeetthefinan-

cialreportingandcertificationmandatesforanyendofyear

financialstatementsfiledafterJune15th2004.Criminalpen-

altiesarepossibleforbreachesoftheAct.

Organizationsmustnowretaindata for longerperiods than

before and produce accurate reports more quickly than

before. Consequently these organizations must ensure that

theirprocessesrelatingtosecurityand integrityofdataare

updated.

The Gramm-leach-Bliley Act (Financial services modernization Act 1999)The Gramm-Leach-Bliley Act primarily sought to modernize

U.S.financialservicesbyendingrestrictionsontheabilityof

banks,stockbrokers,andinsurancecompaniestomergewith

one another. However, this led to the possibility of personal

databeingsharedaroundinstitutionswithouttheconsentof

thedatasubject.

Therefore,undertheAct,financialinstitutions,whetherthey

wish todisclosepersonaldataornot,mustdevelopprecau-

tions toensure the securityandconfidentialityof customer

records and information. They must also seek to protect

againstanticipatedthreatsorhazardstothesecurityorinteg-

rity of such records, and to protect against unauthorized

accesstooruseofsuchrecordsor informationwhichcould

resultinsubstantialharmorinconveniencetoanycustomer.

Someofthemainprovisionscanbesummarizedasfollows:

■ Cleardisclosureisrequiredbyallfinancialinstitutionsof

theirprivacypolicyregardingthesharingofnon-public

personalinformationwithbothaffiliatesandthirdparties.

■ Anoticetoconsumersandanopportunitytoopt-outof

sharingofnon-publicpersonalinformationwithnon-

affiliatedthirdpartiesisrequiredsubjecttocertainlimited

exceptions.

■ TheActaddressesapotentialimbalancebetweenthe

treatmentoflargefinancialservicesconglomeratesand

smallbanksbyincludinganexception,subjecttostrict

controls,forjointmarketingarrangementsbetweenfinan-

cialinstitutions.

■ TheActclarifiesthatthedisclosureofafinancialinstitu-

tion’sprivacypolicyisrequiredtotakeplaceatthetimeof

establishingacustomerrelationshipwithaconsumerand

notlessthanannuallyduringthecontinuationofsuch

relationship.

conclusionThemyriadoflegislativeandregulatoryrequirementsfinan-

cialinstitutionsmustcontendwithinrelationtodataleadsto

one general conclusion: the institutions must have in place

adequateprocedurestomanagedata.

Theimpetusforensuringtheseproceduresareimplemented

properly,carriedoutandadaptedtonewcircumstancesmust

come from the highest level of management within these

institutions. Recent corporate scandals both in the U.S. and

Europemustsurelyservetohighlighttheimportanceofany

organization applying very close scrutiny to the quality and

securityofitsdata.

With regard to Basel II compliance, financial institutions

shouldalreadybeconsiderablyadvanced in termsof imple-

mentingprocedurestoensuredataintegrityandongoingdata

management.


Financial

Data mining in finance: From extremes to realism1

Boris KovalerchukProfessor, Department of Computer Science,

Central Washington university

Evgenii VityaevSenior Scientist, Institute of Mathematics,

Russian Academy of Sciences

Abstract

This paper describes data mining in finance by discussing

financialtasksandspecificsofmethodologiesandtechniques

in this data mining area. It includes time dependence, data

selection, forecast horizon, measures of success, quality of

patterns, hypothesis evaluation, problem ID, method profile,

attribute-based,andrelationalmethodologies.

831 Thispaperisamodifiedversionofauthors’chapter‘Dataminingforfinancial

applications’fromtheforthcoming‘DataMiningandKnowledgeDiscovery

Handbook:ACompleteGuideforPractitionersandResearchers,’KluwerAcad.

Publ.(Eds.O.MaimonandL.Rokach).


Almosteverycomputationalmethodhasbeenexploredand

used for financial modeling. We will name just a few recent

studies:Monte-Carlosimulationofoptionpricing,finite-differ-

ence approach to interest rate derivatives, and fast Fourier

transformforderivativepricing.Newdevelopmentsaugment

traditionaltechnicalanalysisofstockmarketcurves[Murphy

(1999)] thathavebeenusedextensivelyby financial institu-

tions. Such stock charting helps to identify buy/sell signals

(timing‘flags’)usinggraphicalpatterns.

Dataminingasaprocessofdiscoveringusefulpatternsand

correlationshasitsownnicheinfinancialmodeling.Similarto

other computational methods almost every data mining

method and technique has been used in financial modeling.

Anincompletelistincludesavarietyoflinearandnon-linear

models,multi-layerneuralnetworks,k-meansandhierarchical

clustering,k-nearestneighbors,decisiontreeanalysis,regres-

sion(logisticregression,generalmultipleregression),ARIMA,

principalcomponentanalysis,andBayesianlearning.

Less traditional methods used include rough sets, relational

dataminingmethods(deterministic inductivelogicprogram-

ming and newer probabilistic methods [Muggleton (2002),

Kovalerchuk and Vityaev (2000)]), support vector machine,

independentcomponentanalysis,Markovmodels,andhidden

Markovmodels.

Bootstrapping and other evaluation techniques have been

extensivelyusedforimprovingdataminingresults.Specifics

offinancialtimeseriesanalyseswithARIMA,neuralnetworks,

relationalmethods,supportvectormachines,andtraditional

technical analysis are discussed in Kovalerchuk and Vityaev

(2000),Muller,etal.(1997),Murphy(1999),Tsay(2002).

Thenaiveoverlyoptimisticapproachtodatamininginfinance

assumesthatsomebodycanprovideacookbook instruction

onhowtoachievethebestresult.Somepublicationscontinue

to foster this unjustified belief, as is evident in commercial

dataminingsoftwareadvertisements.Thus,itisnotsurprising

that this overly optimistic cookbook extreme is a fertile

groundforanoppositeoverlypessimisticextreme—nothing

canbedone.

Infact,theonlyrealisticapproachproventobesuccessfulis

providingcomparisonsbetweendifferentmethodsconceptu-

allynotonlybasedbytheirperformanceonalimiteddataset.

Theusefulcomparisonshoulddescribedomainsofapplicabil-

ityofeachmethodwithstrengthsandweaknessesrelativeto

problemcharacteristics. In this framework,auser selectsa

method by matching the data mining problem with these

characteristics and task-specific circumstances. In essence,

thisapproachmeansaclearunderstandingthatdatamining

in general, and in finance specifically, is still more art than

hardscience.Fortunately,nowthereisagrowingnumberof

booksthatdiscussissuesofmatchingtasksandmethodsina

regularway[DharandStein(1997),KovalerchukandVityaev

(2000),Wang(2003)].Selectionofamethodisaverycom-

plextask.

Uncertaintyofproblemdescriptions(problemID)andmethod

capabilities(methodID)areamongthemostobviousdifficul-

tiesinthisprocess(Table1).DharandStein(1997)introduced

andappliedaunifiedvocabularyforbusinesscomputational

intelligenceproblemsandmethodsthatprovideaframework

formatchingproblemsandmethods.Aproblemisdescribed

using a set of desirable values (problem ID profile) and a

methodisdescribedusingitscapabilitiesinthesameterms.

Useofunifiedterms(dimensions)forproblemsandmethods

enhances capabilities of comparing alternative methods.

Introducing dimensions also accelerates their clarification.

Next,usersshouldnotbeforcedtospendtimedetermininga

method’scapabilities(valuesofdimensionsforthemethod).

Thisisataskfordevelopers,butusersshouldbeabletoiden-

tify desirable values of dimensions using natural language

termsassuggestedbyDharandStein(1997).

InTable1,weenhancedsuchproblemdescriptionwithdesir-

ablecharacteristicsbyincludingtheirnecessarycharacteris-

tics.Thesecharacteristicsshouldbetakenintoaccounttobe



85

‘October.Thisisoneofthepeculiarlydangerousmonths

tospeculateinstocksin.TheothersareJuly,January,

September,April,November,May,March,June,December,

August,andFebruary.’

MarkTwain,1894

Financial tasksForecastingstockmarket,currencyexchangerate,bankbank-

ruptcies,understandingandmanaging financial risk, trading

futures,creditrating,loanmanagement,bankcustomerprofil-

ing,andmoney launderinganalysesarecore financial tasks

for data mining [Nakhaeizadeh, Steurer & Bartmae (2002)].

Some of these tasks, such as bank customer profiling, have

many similarities with data mining for customer profiling in

otherfields.

Stockmarketforecastingincludesuncoveringmarkettrends,

planning investment strategies, identifying the best time to

purchase thestocks,andwhatstocks topurchase.Financial

institutionsproducehugedatasetsthatbuildafoundationfor

approaching these enormously complex and dynamic prob-

lemswithdataminingtools.Potentialsignificantbenefitsof

solving these problems motivated extensive research for

years.

specifics of financial tasks:

1 Multidimensional time series Y Y Y Y Y Y Y Y Y

3 Specific efficiency criteria Y Y Y Y Y Y Y Y Y

4 Multiresolution forecast Y Y Y Y Y Y Y

5 Explained forecast Y Y Y Y Y Y Y Y Y Y

6 Subtle pattern Y Y Y Y Y Y Y Y Y Y Y Y Y Y

7 Include complex relations Y Y Y Y Y Y Y

8 use of background knowledge Y Y Y Y Y Y Y Y Y

9 Significant level of noise Y Y Y Y Y Y Y Y Y

10 Control of overfitting Y Y Y Y Y Y Y

Data types:

1 Attribute-based Y Y Y Y Y Y Y Y Y Y Y Y Y

2 Relational data types Y Y Y Y Y Y Y Y

3 Mixed data Y Y Y Y Y Y Y Y Y

Hypotheses/models:

1 Functional Y Y Y Y Y Y Y Y Y Y Y

2 Symbolic Y Y Y Y Y Y Y Y Y Y Y Y

Assumptions:

1 Probability of events exists Y Y Y Y Y

2 Occam’s razor Y Y Y Y Y Y Y Y Y Y Y

Table1:FinancialproblemIDsanddataminingmethods’capabilities.

Inthistable,ILPstandsforInductiveLogicProgrammingmethods[Muggleton(1999,2002)].Forexplanationofsomeothertermsusedinthistable,seebelow.

Stock

mar

ketf

oreca

st-

Trad

ingfu

ture

s

Portfolio

man

agem

ent

Money

launder

ing

Credit

ratin

g

Neura

lnet

works

ARIMA

Fuzz

ylo

gic

Deduct

ivere

asonin

g

Statis

tical

met

hods

Decisi

ontree

s

Associa

tionru

les

Supportve

ctorm

achin

e

ILP

Proba

bilistic

ILP

PRoBlEm iD mETHoD cAPABiliTiEs

DimEnsions


ruleslessprofitableandeventfullyuselessorevendamaging.

Greenstone and Oyer (2000) examine the month by month

measuresofreturnforthecomputersoftwareandcomputer

systems stock indices to determine whether these indices’

price movements reflect genuine deviations from random

chance using the standard t-test. They concluded that

althoughWallStreetanalystsrecommendedtousethesum-

merswoonrule(sellcomputerstocksinMayandbuythemat

the end of summer) this rule is not statistically significant.

Howevertheywereabletoconfirmseveralpreviouslyknown

‘calendareffects’, suchas the ‘Januaryeffect’notingmean-

whilethattheyarenotthefirsttowarnofthedangersofeasy

dataminingandunjustifiedclaimsofmarketinefficiency.

The market efficiency theory does not exclude that hidden

short-term local conditional regularities may exist. These

regularitiescannotworkforever,andtheyshouldbecorrect-

edfrequently.

Ithasbeenshownthatthefinancialdataarenotrandomand

that theefficientmarkethypothesis ismerelyasubsetofa

largerchaoticmarkethypothesis[DrakeandKim(1997)].This

hypothesisdoesnotexcludesuccessful short-term forecast-

ingmodelsforpredictionofchaotictimeseries[Casdagliand

Eubank(1992)].

Dataminingdoesnottrytoacceptorrejecttheefficientmar-

kettheory.Dataminingcreatestools,whichcanbeusefulfor

discoveringsubtleshort-termconditionalpatternsandtrends

(Table 1) in a wide range of financial data. This means that

retraining should be a permanent part of data mining in

finance and any claim that a silver bullet trading has been

foundshouldbetreatedsimilarlytoclaimsthataperpetuum

mobilehasbeendiscovered.

The impactofmarketplayersonmarket regularitiesstimu-

latedasurgeofattemptstouseideasofstatisticalphysicsin

finance[Bouchaud(2000)].Ifanobserverisalargemarket-

placeplayerthensuchobservercanpotentiallychangeregu-

laritiesofthemarketplacedynamically.Attemptstoforecast

in such a dynamic environment with thousands of active

agentsleadstomuchmorecomplexmodelsthantraditional

data mining models were designed for. This is one of the

majorreasonsthatsuchinteractionsaremodeledusingideas

fromstatisticalphysicsratherthanfromstatisticaldatamin-

ing. The physics approach in finance [Voit (2003), Ilinski

(2001),andMandelbrot(1997)]isalsoknownas‘econophysic’

and ‘physics of finance’. The issue is that the data mining

approach is inessencenotaboutdevelopingspecificmeth-

ods for financial tasks, but the physics approach is. It is

deeper integrated into the finance subject matter. For

instance,Mandelbrot (1997) (knownforhis famousworkon

fractals)workedalsoonproving that thepricemovement’s

distributionisscalinginvariant.

Dataminingapproachcoversempiricalmodelsandregulari-

tiesderiveddirectlyfromdataandalmostonlyfromdatawith

little domain knowledge explicitly involved. Historically, in

manydomains,deepfield-specifictheoriesemergeafterthe

fieldaccumulatesenoughempiricalregularities.Weseethat

the future of data mining in finance would be to generate

more empirical regularities and combine them with domain

knowledge via generic analytical data mining approach

[Mitchel(1997)].Oneofthefirstattemptsinthisdirectionis

presentedinKovalerchukandVityaev(2000).

Time series analysisAtemporaldatasetT, calleda timeseries, ismodeled inan

attempttodiscoveritsmaincomponents,suchaslong-term

trend,L(T),cyclicvariation,C(T),seasonalvariation,S(T)and

irregularmovements,I(T).AssumethatTisatimeseries,such

as daily closing price of a share or S&P 500 index, from

moment0 tocurrentmomentk, then thenextvalueof the

timeseriesT(k+n)ismodeledbyformula(1):

T(k+n)=L(T)+C(T)+S(T)+I(T) (1)

TraditionallyclassicalARIMAmodelsoccupythisareaforfind-

ingparametersoffunctionsusedinformula(1).ARIMAmodels



surethatamethodcouldsolvetheprobleminprinciple.Ifa

method does not have necessary characteristics, but it has

somedesirableones(i.e.user-friendlyinterface)itwouldnot

besufficient.For instance, forstockmarket forecasting it is

necessary that a method is able to work with multidimen-

sionaltimeseries,tousespecificfinancialefficiencycriteria,

to predict market for different forecast horizon providing a

multiresolutionforecast,todiscoverandusesubtlepatterns,

andtoworkwithahigh levelofnoise.Thesecharacteristics

arepresentedinTable1.

Attribute-based learning methods such as neural networks,

thenearestneighborsmethod,anddecisiontreesdominatein

financialapplicationsofdatamining.Thesemethodsarerela-

tively simple, efficient, and can handle noisy data. However,

thesemethodshavetwoseriousdrawbacks:alimitedabilityto

represent background knowledge and the lack of complex

relations. Relational data mining techniques that include

Inductive Logic Programming (ILP) [Muggleton (1999),

Dzeroski (2002)] intend to overcome these limitations (last

twocolumnsinTable1).

Previouslythesemethodshavebeenrelativelycomputation-

ally inefficient and had rather limited facilities for handling

numerical data [Bratko and Muggleton (1995)]. Currently

these methods are enhanced in both aspects [Kovalerchuk

andVityaev(2000)]andareespeciallyactivelyusedinbio-

informatics [Muggleton (2002), Vityaev et al. (2002)]. We

believe thatnow is the time forapplying thesemethods to

financialanalysesmoreintensively,especiallytothoseanaly-

sesthatdealwithprobabilisticrelationalreasoning.

Variouspublicationshaveestimatedtheuseofdatamining

methods like hybrid architectures of neural networks with

geneticalgorithms,chaostheory,andfuzzylogicinfinance.

‘ConservativeestimatesplaceaboutU.S.$5billiontoU.S.$10

billionunderthedirectmanagementofneuralnetworktrad-

ing models. This amount is growing steadily as more firms

experiment with and gain confidence with neural networks

techniquesandmethods’[Loofbourrow(1995)].Table1con-

tainsonlybasemethods,nothybridmethodsthataretheir

combinations.However,Table1showsthatexistingmethods

donotindividuallymeetchallengesoffinancialtasks.Hybrids

arereallyneededtomeetthesechallenges.Inmoredetail,we

discuss this issue inKovalerchukandVityaev (2000).Many

proprietary financial applications exist and use hybrid data

miningmethods,butwithoutreportingpubliclyaboutmeth-

ods used as was stated in Von Altrock (1997) and Groth

(1998).

specifics of data mining in financeThechallengeforfinancialdataminingiscomingfromseveral

difficulttoaccomplishspecificrequests:

■ Forecastmultidimensionaltimeserieswithhighlevelof

noise.

■ Accommodatespecificefficiencycriteria(i.e.themaxi-

mumoftradingprofit)inadditiontopredictionaccuracy,

suchasR2.

■ Makecoordinatedmultiresolutionforecast(minutes,days,

weeks,months,andyears).

■ Incorporateastreamoftextsignalsasinputdataforfore-

castingmodels(i.e.theEnroncase,September11,and

others).

■ Beabletoexplaintheforecastandtheforecastingmodel

(blackboxmodelshavelimitedinterestandfuturefor

significantinvestmentdecisions).

■ Beabletobenefitfromverysubtlepatternswithashort

lifetime.

■ Incorporatetheimpactofmarketplayersonmarketregu-

larities.

These requests are major points for selecting data mining

methodasTable1summarizes.

The current efficient market theory/hypothesis discourages

attemptstodiscover long-termstable tradingrules/regulari-

ties with significant profit. This theory is based on the idea

that if such regularities exist they would be discovered and

usedbythemajorityofthemarketplayers.Thiswouldmake

87


randomly, but are produced by data snooping — checking

combinationsof industrysectorsandmonthsof returnand

then reporting only a few significant combinations. This

means that rigorous tests would require testing a different

nullhypothesisnotonlyaboutonesignificant combination,

butalsoaboutthefamilyofcombinations.Eachcombination

is about an individual industry sector by month’s return. In

this setting, the return for the family is tested versus the

overallmarketreturn.

Severaltestingoptionsareavailable.Sullivanetal.(1999)use

abootstrappingmethodtoevaluatestatisticalsignificanceof

suchhypothesesadjustedfortheeffectsofdatasnoopingin

trading rules and calendar anomalies. Greenstone and Oyer

(2000)suggestasimplecomputationalmethod—combining

individual t-test results by using the Bonferroni inequality.

Anotheroptionwouldbetotestwhetherthestatementsare

jointly true using the traditional F-test. However if the null

hypothesis about a joint statement is rejected it does not

identifytheprofitabletradingstrategies.

The sequential semantic probabilistic reasoning that uses

statisticalF-testaddressesthisissue.KovalerchukandVityaev

(2000) were able to identify profitable and statistically sig-

nificant patterns for the S&P 500 index using this method.

ThesetypesofmethodsareidentifiedinTable1asProbabilistic

ILP, where ILP stands for Inductive Logic Programming. We

alsowereabletodemonstratethatthisapproachcanbeben-

eficialforuncoveringmoneylaunderingschemesinforensic

accounting. [Klösgen and Zytkow (2002), Kovalerchuk and

Vityaev (2003)]. This technique that combines first-order

logicandprobabilisticsemanticinferenceisdiscussedinthe

nextsection.

Relational data mining in financeDecisiontreemethodsareverypopularindataminingappli-

cations ingeneralandinfinancespecifically.Theyprovidea

setofhumanreadable,consistentrules,butdiscoveringsmall

treesforcomplexproblemscanbeasignificantchallengein

finance.Inaddition,rulesextractedfromdecisiontreesfailto

comparetwoattributevalues,asitispossiblewithrelational

methods.

Itseemsthatrelationaldataminingmethodsalsoknownas

relationalknowledgediscoverymethodsaregainingmomen-

tum in different fields [Muggleton (2002), Dzeroski (2002),

Vityaevetal.(2002),Cowan(2002)].Belowweprovidesome

majorconceptsfromrelationaldatamining.

DatacanberepresentedbyattributesA1,A2,…,Anofobjects,

thatiseachobjectxisgivenbyasetofvaluesA1(x),A2(x),…

,An(x). The common data mining methodology assume this

typeofdata.Suchdataformthebaseofanattribute-basedor

attribute-valuemethodology.Itcoversawiderangeofstatisti-

calandconnectionist(neuralnetwork)methods.Examplesof

themostpopularattributesinfinancialtimeseriesareindex

valueatopen,indexvalueatclose,highestindexvalue,lowest

indexvalue,andtradingvolumeandlaggedreturnsfromthe

timeseriesofinterest.Fundamentalfactorsincludetheprice

of gold, retail sales index, industrial production indices, and

foreigncurrencyexchangerates.Technicalattributesinclude

variables that are derived from time series such as moving

averages.

Therelationaldatatypeisaseconddatatype,whereobjects

are represented by their relations with other objects, for

instance,x>y,y<z,x>z.Inthisexamplewemaynotknow

thatx=3,y=1andz=2,butweknowrelationsbetweenx,yand

z.Objectsmayhavedifferentattributevalues(i.e.,x=5,y=2,

andz=4),butstillhavethesamerelations.Alesstraditional

relationalmethodologyisbasedontherelationaldatatype.

Manydataminingmethodsassumeafunctionalformofthe

relationship. For instance, the linear discriminant analysis

assumes linearity of the border that discriminates between

twoclassesinthespaceofattributes.Oftenitishardtojus-

tifysuch functional form inadvance.Relationaldatamining

methodologyintendstolearnsymbolicrelationsonnumerical

data.Thefollowingtechnicalanalysisruleisinthiscategory.

To derive a conclusion it compares values of two attributes


12

j=1


arewelldevelopedbutaredifficulttouseforhighlynon-sta-

tionary stochastic processes that is typical in finance.

Potentially, data mining methods can be used to build such

modelstoovercomeARIMAlimitations.Theadvantageofthis

four-componentmodelincomparisonwithblackboxmodels,

such as neural networks, is that components in formula (1)

haveaninterpretation.

Data selection and forecast horizonData mining in finance has the same challenges as general

data mining has in data selection for building models. In

finance,thisquestionistightlyconnectedtotheselectionof

thetargetvariable.Thereareseveraloptionsfortargetvari-

able y: y=T(k+1), y=T(k+2),…,y=T(k+n), where y=T(k+1) repre-

sentsforecastforthenexttimemoment,andy=T(k+n)repre-

sents forecast fornmomentsahead.SelectionofdatasetT

anditssizeforaspecificdesiredforecasthorizonnisasig-

nificantchallenge.

Forstationarystochasticprocessestheansweriswell-known,

thatabettermodelcanbebuiltforlongertrainingduration.

Forfinancialtimeseries,suchasS&P500index,thisisnotthe

case [Mehta and Bhattacharyya (2004)]. Longer training

durationmayproducemanyandcontradictoryprofitpatterns

thatreflectbearandbullmarketperiods.Modelsbuiltusing

too short durations may suffer from overfitting and hardly

applicabletothesituationswheremarketismovingfromthe

bullperiodtothebearperiod.Alsoinfinance,thelong-hori-

zon returns could be forecasted better than short-horizon

returns depending on the training data used and model

parameters[KrolzigandToro(2004)].

Instandarddataminingitistypicallyassumedthatthequality

ofthemodeldoesnotdependonfrequencyofitsuse.Infinan-

cialapplicationthefrequencyoftradingisoneoftheparam-

eters that impact the quality of the model. This happens

because in finance the criterion of the model quality is not

limitedtotheaccuracyofprediction,but isdrivenbyprofit-

ability of the model. It is obvious that frequency of trading

impactstheprofitaswellasthetradingrulesandstrategy.

measures of successTraditionally, thequalityof financialdataminingforecasting

modelshasbeenmeasuredbythestandarddeviationbetween

forecastandactualvaluesontrainingandtestingdata.This

approach works well in many domains, but this assumption

shouldberevisitedfortradingtasks.Twomodelscanhavethe

samestandarddeviationbutmayprovideverydifferenttrad-

ing returns. The small R2 is not sufficient to judge that the

forecastingmodelwillcorrectlyforecaststockchangedirec-

tion(signandmagnitude).Moreappropriatemeasuresofsuc-

cess in financialdataminingaremeasures,suchasaverage

monthly excess return (AMER) and potential trading profits

(PTP)[GreenstoneandOyer(2000)]:

AMERj=Rij-βiR500j-(Σ (Rij-βiR500j)/12),

WhereRijistheaveragereturnfortheS&P500indexinindus-

tryiandmonthjandR500jistheaveragereturnoftheS&P

500inmonthj.TheβivaluesadjusttheAMERfortheindex’s

sensitivitytotheoverallmarket.Asecondmeasureofreturn

ispotentialtradingprofits(PTP):

PTPij=Rij-R500j

PTP shows investor’s trading profit versus the alternative

investmentbasedonthebroaderS&P500index.

Quality of patterns and hypothesis evaluationAnimportantissueindataminingingeneralandinfinancein

particularistheevaluationofqualityofdiscoveredpatternP

measured by its statistical significance. A typical approach

assumesthetestingofthenullhypothesisHthatpatternPis

notstatisticallysignificantatlevelα.Ameaningfulstatistical

testrequiresthatpatternparameters,suchasthemonth(s)of

theyearandtherelevantsectoralindexinatradingrulepat-

tern P, have been chosen randomly [Greenstone and Oyer

(2000)].Inmanytasks,thisisnotthecase.

GreenstoneandOyerarguethatinthe‘summerswoon’trad-

ing rule mentioned above, the parameters are not selected

89


extensive growth of hybrid methods that combine different

models and provide a better performance than can be

achievedbyindividuals.Insuchanintegrativeapproachindi-

vidual models are interpreted as trained artificial experts.

Therefore their combinations can be organized similar to a

consultationofrealhumanexperts.Moreover,theseartificial

experts can be effectively combined with real experts. It is

expectedthattheseartificialexpertswillbebuiltasautono-

mous intelligent software agents. Thus, experts to be com-

binedcanbedataminingmodels,realfinancialexperts,trader,

andvirtualexpertsthatrunstradingrulesextractedfromreal

experts.Avirtualexpertisasoftwareintelligentagentthatis

inessenceanexpert system.Wecoinedanew term ‘expert

mining’ as an umbrella term for extracting knowledge from

realhumanexpertsthatisneededtopopulatevirtualexperts.

Weexpectthatincomingyearsdatamininginfinancewillbe

shapedasadistinctfieldthatblendsknowledgefromfinance

anddatamining,similartowhatweseenowinbioinformatics,

whereintegrationoffieldspecificsanddataminingiscloseto

maturity.Wealsoexpectthattheblendingwithideasfromthe

theory of dynamic systems, chaos theory, and physics of

financewilldeepen.

References• Bouchaud,J.,andM.Potters,2000,TheoryofFinancialRisks:FromStatistical

PhysicstoRiskManagement,CambridgeUniv.Press,Cambridge,UK.

• Bratko,I.,andS.Muggleton,1995,“ApplicationsofInductiveLogicProgramming.

CommunicationsofACM,”38:11,65-70

• Casdagli,M.,andS.Eubank,1992,(Eds).Nonlinearmodelingandforecasting,

AddisonWesley

• Cowan,A.,2002,“Bookreview:DataMininginFinance,”InternationalJournalof

Forecasting,18:1,155-156

• Dhar,V.,andR.Stein,1997,Intelligentdecisionsupportmethods,PrenticeHall

• DzeroskiS.,2002,“Inductivelogicprogrammingapproaches,”In:KlösgenW.,andJ.

Zytkow,Handbookofdataminingandknowledgediscovery,OxfordUniversityPress,

348-353

• Drake,K.,andY.Kim,1997,“Abductiveinformationmodelingappliedtofinancial

timeseriesforecasting,”In:Nonlinearfinancialforecasting,financeandtechnology,

95-109

• Groth,R.,1998,DataMining,PrenticeHall

• Greenstone,M.,andP.Oyer,2000,“Aretheresectoralanomaliestoo?Thepitfallsof

unreportedmultiplehypothesistestingandasimplesolution,”Reviewof

QuantitativeFinanceandAccounting,15,37-55

• Huang,Z.,H.Chen,C.J.Hsu,W.H.Chen,andS.Wu,2004,“Creditratinganalysis

withsupportvectormachinesandneuralnetworks:amarketcomparativestudy,”

Decisionsupportsystems,37:4,543-558

• Ilinski,K.,2001,Physicsoffinance:Gaugemodelinginnon-equilibriumpricing,Wiley

• KlösgenW.,andJ.Zytkow,2002,Handbookofdataminingandknowledgediscov-

ery,OxfordUniversityPress,Oxford

• Kovalerchuk,B.,andE.Vityaev,2000,Datamininginfinance:Advancesinrelational

andhybridmethods,Kluwer,2000.

• Kovalerchuk,B.,E.Vityaev,andJ.F.Ruiz,2001,“Consistentandcompletedataand

‘mining’inmedicine,”In:Medicaldataminingandknowledgediscovery,Springer,

238-280

• Kovalerchuk,B.,andE.Vityaev,2003,“Detectingpatternsoffraudulentbehaviorin

forensicaccounting,”InProceedingsoftheSeventhInternationalConference

“Knowledge-basedIntelligentInformationandEngineeringonSystems,”Oxford,UK,

Sept,Part1,502-509

• Krolzig,M.,andJ.Toro,2004,“Multiperiodforecastinginstockmarkets:Aparadox

solved,”DecisionSupportSystems,37:4,531-542

• Loofbourrow,J.,andT.Loofbourrow,1995,“WhatAIbringstotradingandportfolio

management,”In:FreedmanR.,R.Klein,andJ.Lederman,Artificialintelligencein

thecapitalmarkets,Irwin,Chicago,3-28

• Mandelbrot,B.,1997,Fractalsandscalinginfinance,Springer

• Mehta,K.,andS.Bhattacharyya,2004,“Adequacyoftrainingdataforevolutionary

miningoftradingrules,”DecisionSupportSystems,37:4,461-474

• Mitchell,T.,1997,Machinelearning,McGrawHill

• Muller,K.R.,A.Smola,G.Rtsch,B.Schlkopf,J.Kohlmorgen,andV.Vapnik,1997,

“Usingsupportvectormachinesfortimeseriesprediction,”In:AdvancesinKernel

methodssupportvectorlearning,MITPress

• Murphy,J.,1999,Technicalanalysisofthefinancialmarkets:Acomprehensiveguide

totradingmethodsandapplications,PrenticeHall

• Muggleton,S.,2002,“Learningstructureandparametersofstochasticlogicpro-

grams,”12thInternationalConference,ILP2002,Sydney,Australia,July9-11.

LecturenotesinComputerScience2583,Springer2003,198-206

• MuggletonS.,1999,“ScientificKnowledgeDiscoveryUsingInductiveLogic

Programming,”CommunicationsofACM,42:11,42-46

• Nakhaeizadeh,G.,E.Steurer,andK.Bartmae,2002,“BankingandFinance,”In:

KlösgenW.,andJ.Zytkow,Handbookofdataminingandknowledgediscovery,

OxfordUniversityPress,771-780

• Occam’sRazor!,2003,http://www.commoditytradingadvisor.com/occams-razor.htm

• QuinlanJ.R.,1993,“C4.5:programsformachinelearning,”MorganKaufmann

PublishersInc.,SanFrancisco,CA

• Sullivan,R.,A.Timmermann,andH.White,1999,“Data-snooping,technicaltrading

ruleperformance,andthebootstrap,”JournalofFinance,54,1647-1691

• Tsang,E.,P.Yung,andJ.Li,2004,“EDDIE-Automation,adecisionsupporttoolfor

financialforecasting,”JournalofDecisionSupportSystems,SpecialIssueonData

miningforfinancialdecisionmaking,37:4,559-565

• Tsay,R,2002,Analysisoffinancialtimeseries,Wiley

• VityaevE.E.,L.OrlovYu,O.V.Vishnevsky,B.Ya.Kovalerchuk,A.S.Belenok,N.L.

Podkolodnii,andN.A.Kolchanov,2002,“Knowledgediscoveryforgeneregulatory

regionsanalysis,”In:Knowledge-basedintelligentinformationengineeringsystems

andalliedtechnologies,KES,(Eds.)Damiani,E.,R.Howlett,L.Jain,andN.

Ichalkaranje,IOSPress,Amsterdam,Part1,487-491

• Voit,J.,2003,Thestatisticalmechanicsoffinancialmarkets,Vol.2,Springer



such,as5and15daymovingaverages(ME5andME15)and

derivatives of moving averages for 10 and 30 days

(DerivativeME10,DerivativeME30):

IfME5(t)=ME15(t)&DerivativeME10(t)>0DerivativeME30(t)>0,

thenbuystockatmoment(t+1).

Thisrulecanbereadas‘Ifmovingaveragesfor5and15days

areequalandderivativesformovingaveragesfor10and30

daysarepositivethenbuystockonthenextday.’

Datamininginfinanceleadstheapplicationofrelationaldata

miningformultidimensionaltimeseries,suchasstockmarket

timeseries.A.Cowan,aseniorfinancialeconomistfromU.S.

DepartmentoftheTreasury,noticedthatexamplesandargu-

ments available in Kovalerchuk and Vityaev (2000) for the

applicationofrelationaldataminingmethods,suchasMachine

Method for Discovering Regularities (MMDR), to financial

problemsproduceexpectationsofgreatadvancementsinthis

field in the near future for financial applications [Cowan

(2002)].

It was strengthened in several publications by suggestions

thatrelationaldataminingareaismovingtowardprobabilis-

tic first-order rules to avoid the limitations of deterministic

systems[Muggleton(2002)].Relationalmethods in finance,

such as MMDR, are equipped with probabilistic mechanism

that is necessary for time series with high level of noise.

MMDRiswellsuitedtofinancialapplicationsgivenitsability

to handle numerical data with high levels of noise [Cowan

(2002)].

Informally, the ideaofsemanticprobabilisticreasoningused

in MMDR method is coming from the principle of Occam’s

razor(alawofsimplicity)inscienceandphilosophy.Fortrad-

ingitwaswrittenasfollowsinOccam’srazor!(2004):

■ Whenyouhavetwocompetingtradingtheorieswhich

makeexactlythesamepredictions,theonethatissimpler

isthebetterandmoreprofitableone.

■ Ifyouhavetwotrading/investingtheorieswhichboth

explaintheobservedfactsthenyoushouldusethesim-

plestoneuntilmoreevidencecomesalong.

■ Thesimplestexplanationforacommodityorstockprice

movementphenomenonismorelikelytobeaccuratethan

morecomplicatedexplanations.

■ Ifyouhavetwoequallylikelysolutionstoatradingorday

tradingproblem,pickthesimplest.

■ Thepricemovementexplanationrequiringthefewest

assumptionsismostlikelytobecorrect.

conclusionTobesuccessfuladata-miningprojectshouldbedrivenbythe

application needs and results should be tested quickly.

Financial applications provide a unique environment where

efficiencyofthemethodscanbetestedalmostinstantly,not

onlybyusingtraditionaltrainingandtestingdatabutmaking

realstockforecastandtestingitthesameday.Thisprocess

can be repeated daily for several months collecting quality

estimates.

The relational data mining methods outlined in this paper

advancespatterndiscoverymethodsthatdealwithcomplex

numeric and non-numeric data, involve structured objects,

text,anddata inavarietyofdiscreteandcontinuousscales

(nominal,order,absolute,andsoon).

Currently the success of data mining exercises has been

reportedinliteratureextensively.Typicallyitisdonebycom-

paringsimulatedtradingandforecastingresultswithresults

ofothermethodsandrealgain/lossandstock.For instance,

recentlyHuangetal.(2004)claimedthatdataminingmeth-

ods achieved better performance than traditional statistical

methods in predicting credit ratings. Much less has been

reportedpubliclyonsuccessofdatamininginrealtradingby

financialinstitutions.Itseemsthatthemarketefficiencythe-

ory is applicable to reporting success. If real success is

reportedthencompetitorscanapplythesamemethodsand

the advantage will disappear because in essence all funda-

mentaldataminingmethodsarenotproprietary.

Thenextstepisforthedevelopmentofpracticaldecisionsup-

port software tools thatmake iteasier tooperate inadata

mining environment specific for financial tasks, where hun-

dredsandthousandsofmodels,suchasneuralnetworksand

decision trees, need to be analyzed and adjusted every day

withanewdatastreamcomingeveryminute.

Inside of the field of data mining in finance we expect an 91

Enterprise

The legal assault against marketing

who owns the customer? who owns the data?

Privacy challenges

Data quality management

The fourth leg of the stool – Data protection

steady progress – But could do better

The shift to web services

A user-centric approach to effective enterprise data services

Extracting the business value of iT

Taking snapshots of the internet

The informational role of financial analysts

1 Source:AlternativeInvestingReport2003,GoldmanSachs/Russell

The legal assault against marketing

Kirk HerathChief Privacy Officer, Associate General Counsel, Nationwide Insurance Companies

or service under that specific type of communication.

Companiescanquicklylosetheveryabilitytosellanythingin

the future to that customer, if they offend him or her and

elicitanoptout.

For example, in the financial services industry, a company’s

breadandbutterproductswilloftenmeanseveralthousand

dollarsperyearinincomeorpremiumfromanindividualcus-

tomer. Yet, that same customer is often in the bull’s-eye of

severalothermarketingcampaignsforancillaryorunrelated

productsorservicesasaresultofbeingacustomerofthat

same company. Under the FCRA, the original company may

sharethecustomer’s informationwithaffiliatestosell them

newproductsandservices.Theymayalsoshareinformation

withunaffiliatedcompanies,underGLBA,throughjoint-mar-

keting arrangements. However, the original company’s cut

fromtheancillaryaffiliatecross-sellorjoint-marketingactivi-

ty with a third-party partner may be very small, such as

U.S.$50toU.S.$100.Bynotmanagingaccesstothecustomer,

youraffiliateandthird-partymarketingactivitiesmayresultin

thecustomeraskingto:

■ Opt-outofallaffiliatesharingformarketingpurposes.

■ Opt-outofallthird-partysharingformarketingpurposes.

■ Opt-outofallcommerciale-mailsolicitationcampaigns

nowandinthefuture.

■ Beplacedonthecorporatedonotcalllists,whicheffec-

tivelyprecludesanyandallcontactwiththatcustomerby

telephoneonanynewproductorservice.

Lookatitthisway.Youhaveafiniteamountofgoodwillfrom

eachcustomer.Assumingthattheyarealreadycustomersof

yourcoreproductsorservices,theyalreadyrepresentaposi-

tivestreamofincomeorpremiumtothecompany.Obviously,

companies must maximize the amount of money that they

receivefromanysinglecustomer.However,unlikeinthepast,

whenyoumayhavehadmanyopportunitiestoreachoutto

an existing customer in an attempt to up-sell or cross-sell

themotherproductsorservicesfromanassortmentofaffili-

atesorthird-partybusinesses,thecustomercannowessen-

tiallyprecludeorturnoffanynewmarketingtothem.Thekey

is to maintain the current income stream (core products),

whilegentlyenticingthemtopurchasenewproducts,without

offendingthemtothepointwheretheyuseoneorallofthe

legaltoolsavailabletothemtoopt-outoffuturemarketing.

new strategies for marketingWhen developing marketing strategies, companies must

remember that each contact with existing customers may

result in that customer opting out of all future contacts by

thatmeanwiththecompany.

Nooffensetotelemarketers,butthedataclearlyshowsthat

consumers find unsolicited telephone sales calls to be the

mostoffensivemeansofmarketingcommunications.Thedin-

nertimesalescallmaywellhavebeenthegenesisoftheentire

modernprivacymovementintheUnitedStates.Over75mil-

lion phone numbers, representing well over half of the U.S.

population,nowresideonthemanystateandfederaldonot

call lists.Companiesalsohaveadutytocomplywithacon-

sumer’swishesiftheyasktobeplacedonacompany-specific

list,butformostcustomersthisonlyresultsfromaphonecall

from the company or its representative that the consumer

findsoffensive.Nevertheless,regardlessofthereason,once

the customer’s phone number is on a company’s corporate

do-not-call list, the EBR exception is over, and no one from

thatcompanyoranyof itsmarketingpartnerscancall that

customer in the future to solicit a new product or service.

Therefore,companiesshouldsparinglyuse the telephoneto

communicatewithcustomersandonlyusethis‘chit’onprod-

ucts or services that would truly generate good cash flow

opportunities.Otherwise, ifyoutrytosellsomeonealower-

valuedproductorservicethatgeneratessmallincometothe

companyand,asaresult,theconsumeraskstobeplacedon

thecompanydo-not-calllist,evenyouragentswillnotbeable

tocommunicatewiththatcustomeraboutnewhigher-valued

andhigher-incomegeneratingproducts.Thisisthetextbook

definitionofcuttingyournoseofftospiteyourface.

Thesamelineofreasoningholdstrueforcommerciale-mail

94

95

Legislators have heard the loud cries of their constituents:

‘Don’t callmeduringdinner!—Don’te-mailme!—Don’t fax

me!’Andtheyhaverespondedwithamyriadoflawstoprotect

theirconstituentsfromwhattheyperceiveastheonslaught

ofmarketing.Inthisnewmarketingenvironment,companies

have one shot to get their message out or risk losing the

opportunityaltogether.

Privacy vs marketingPrivacy law is quickly becoming the law against marketing.

Just a few years ago, the environment was quite different.

Companies contacted prospective customers whenever and

howeverpossible tomarketproducts.Theyboughtandsold

customerliststomarkettonewconsumers.Affiliatedcompa-

niesfreelyshareddatatocross-sellproducts.

Today, laws such as the CAN-SPAM Act and Do-Not-Call

restrict how companies can solicit individuals. The Gramm-

Leach-Bliley Act (GLBA) restricts how a business can use

customer information with third parties for marketing pur-

poses.Restrictionson theuseofcustomerdata foraffiliate

sharingarenowthecenterpieceoftheFairCreditReporting

Act(FCRA).Aplethoraofstateandfederal lawshaveeffec-

tivelycurbedtheuseofthetelephoneformarketingpurpos-

es,evensweepingintheuseoffaxmachinesintheprocess.

Withalloftheselawsrestrictinghowandwhenonecaneither

usecustomerinformationtosellproductsandservicesorthe

meansofcommunicatingwithprospective(andevenexisting)

customers,companiesmustclearlythinkthroughtheirmar-

keting campaigns to determine whether the risk of losing

customersisoutweighedbyalargereturn.Otherwise,compa-

nieswillfindthattheyarebeingpenny-wiseandpound-fool-

ish.

opting out of marketingIn general, the new privacy laws require companies to give

customersanopportunitytooptoutofspecifictypesofmar-

keting. The laws vary, but generally, when a customer opts

out,theyrevokethepowerofacompanytosharetheirinfor-

mationorcontactthemformarketingpurposes.Someopt-out

choicesliterallyforceacompanytooptthatpersonoutofall

futurecommunicationsfromaspecific‘meansofcommunica-

tion’, such as by telephone, fax, direct mail, or commercial

e-mail. Thus, the law allows companies one opportunity to

market to a consumer — even when the company is cross-

sellingtheirownproductstoexistingcustomers.Companies

nowhaveafiniteabilitytomarkettheirproductsandservices

toanyindividual.Anymarketingpracticesthatupsetanindi-

vidualwilleffectivelyremovetheindividualpermanentlyfrom

thepoolofprospectivecustomers.

marketing and established business relationshipsManyoftheprivacylawsofferasafeharborforexistingbusi-

nessrelationships(EBRs).Whatthismeansisthatacompany

canstillsolicitexistingcustomersforanotherproduct,evenif

thecustomerhasalreadyplacedhisorhernameonastateor

federaldonotcalllist.UndertheCAN-SPAMAct,acompany

canessentiallysendanyone—currentorprospectivecustom-

ers—onecommerciale-mailsolicitation—buttheyhavetobe

able torespondtoarecipient’srequest tooptoutof future

commerciale-mailsolicitations.

Theexistenceofanestablishedbusinessrelationshipdoesnot

guarantee unlimited marketing ability to current customers.

Existingcustomerscanaskcompaniesnottomarkettothem.

Theymaytellthecompanyoritsrepresentativetoaddthem

tothecompany’sdo-not-calllistortostopsendingthemcom-

merciale-mailsolicitations.Theymayalsochoosetooptout

ofthirdpartyandaffiliatesharingofinformationformarket-

ingpurposes,whicheffectivelyremovesthemfromdirectmail

solicitations. Additionally, if you really upset them, they will

contactconsumerreportingagenciesandasktoberemoved

fromallmarketingfromanyone.

Asyoucansee,eventheabilitytomarkettoexistingcustom-

ersisbecomingrestricted.Eachtimeacompanycontactsan

existingcustomer,thatcustomermaychoosetoexercisetheir

righttooptout.Oncethecustomerispushedtooptout,com-

panieseffectivelylosetheabilitytomarketanynewproduct

solicitations.Usethemsparinglywithyourexistingcustom-

ers.Ifusedproperly,e-mailprovidesanexcellentopportunity

toenterintoarelationshipwithyourcustomerbase.Discover

waystogetyourcustomerstoconsent,oropt-in,tocommer-

ciale-mailsfromyou.Newslettersthattellthemhowtobet-

ter use the products or services they currently have, while

concurrently showing them glimpses of new products that

couldbeusefultothem,areaneffectivemeansofcommuni-

cationbye-mail.But,becarefulhowyouusethismeansof

communication,becauseyouareonlyonenegativeconsent

option away from not ever being able to send someone a

commerciale-mailsolicitation.Therefore,managethismeans

ofcommunicationwellanddonotoffendyourcustomersby

overusingit.

Third-partyandaffiliatemarketingcanalsohelpacompany

getalargerpieceofanycustomer’swallet,butthistooshould

bemanaged.Youcannotgo to thecustomerwell toomany

times,orelseyouwillforcenegativebehaviorandfindyour

potentialmarketingpoolofcustomersquicklydwindling.

conclusionAsshownthroughout thisarticle,privacy lawsbytheirvery

natureposeadirectthreattoacompany’sabilitytomarket

newproductsandservicestoitscustomers.Therefore,before

attempting tosell customerseveryconceivableproductand

service, regardlessof thepotential return, it is important to

coordinatemarketingactivitiesandlimityourmarketingcon-

tactstoonlythoseproductsorsetsofproductsthatprovide

thegreatestreturn.


who owns the customer? who owns the data?Keith MacDonald, Partner, CapcoMark Dynes, Managing Principal, Capco

chaincomplexity.

supply chain complexityInrecentyears,wehavebeenwitnessingan increase in the

complexityofthefinancialservicesvaluechain.Thedecision

to enter into these types deals is, in most circumstances,

driven by an internal assessment of core competencies, in

effect theclassic ‘makev.buy’manufacturingmodel.Whilst

this process may justify the deal, we believe that too many

organizations have not yet developed the competencies to

maximizethereturnsfromtheserelationships.Inessence,the

moreparties involved, theharder it is toensure thatobjec-

tives are the same. There are numerous examples of the

increasing complexity within the financial services supply

chain:

Joint ventures–Theentryofretailersandsupermarketsinto

financial services has spawned many joint ventures. Well

established players in the U.K., such as Tesco Personal

Finance (Tesco/RBS) and Sainsbury’s Bank (Sainsbury’s/

HBoS), are now competing with newer rivals, such as Post

Office Financial Services (Post Office/Bank of Ireland). We

have also seen the creation of distribution only JV’s being

attempted,albeitwithmixedsuccess(i.e.Abbey/CostaCoffee

co-location).Wehavealreadywitnessedthesecondroundof

supply chain complexity with some organizations already

having changed JV partners on more than one occasion

(TescoPersonalFinancewasoriginallywithHFCBank, then

NatWestbeforemovingtoRBS)orbeingboughtout(Goldfish

fromCentrica).

white labeling/Affinities –Again,therearemanywellestab-

lishedexamplesofwhite labelingandaffinities,particularly

withinthecardmarket(i.e.storecards,suchastherecently

announcedJohnLewis/HSBClaunch).MBNAhavebeenpar-

ticularlyaggressivewithin theU.K.marketplace,buying the

cardbooksofAbbeyandAlliance&Leicester.Thereareother

non-card examples, such as HBoS’s deals with trade bodies

andtradeunions.Thelifeandpensionsmarkethasalsopro-

ducedmanydeals(i.e.BarclaysandAlliance&Leicesterwith

Legal&General).Wearealsobeginningtoseethecreationof

othercorporaterelationships,suchasworksitedealsallowing

abankaccesstoacompany’semployees,whichaddfurther

complexitytothetraditionalwhite labeling/affinityrelation-

ships.

multi-brand loyalty schemes – One of the most successful

multi-brand loyalty schemeshasbeenAirmiles,whichhasa

rangeofpartnerorganizationsacross financialservicesand

non-financial services companies. However, we have seen

multi-brandloyaltyschemestakentothenext levelwiththe

creationofNectar,intermsofthebrandstrengthoftheorga-

nizations involved and customer acceptance of the scheme.

However, these multi-brand loyalty schemes bring potential

complexitiestoowningandmanagingthecustomerrelation-

shipasthepointofcontactfortheschemeisnolongerthe

hostorganization—andtherefore,customersmaybereceiv-

ing conflicting or competing marketing from rival partners

(forexample,Barclaycardarethefinancialservicesprovider

to Nectar, but Sainsbury’s only offer Nectar points as the

supermarketpartner,despiteco-owningSainsbury’sBank).

Retailing competitors’ products –Whiletheirmodel isnow

changing,Bradford&Bingley’sMarketplaceintheU.K.wasan

attempttocreateashop-frontforthebestdealsirrespective

of product manufacturer. The general insurance market is

anotherwherepanelsofsuppliersareofferedtocustomersto

decideontheirpreferredchoicebasedonpriceandproduct

features. The question for the provider of these services is

whetherornotthestrengthoftheirbrandisenoughtopro-

tecttherelationshipwiththecustomer.

outsourcing –Outsourcing,beitonshore,nearshore,oroff-

shore, can now mean initial customer contact through to

eventualproductsupplycanbedeliveredbyfirmsotherthan

thebrandowner.Again,thisraisesfundamentalquestionson

whatitisusedforandhowbothorganizationsensurethatall

revenueopportunitiesaremaximizedthroughprovidingrele-

vantcustomerinformationandthepointof interaction.This

becomesevenmoredifficult,becauseagain, typicaldeals in

thisspacefocusoncostreductionandefficienciesratherthen


99

Much has been written about Customer Relationship

Management (CRM)and itsmyriadofhybrids. Indeed,many

organizationsarealreadyencounteringCRMfatigueasthey

struggletosuccessfullyusecustomerdatatodeepenrelation-

shipsandincreaseshareofwallet.However,therealityisthat

from a financial services perspective CRM has often been

associatedwithamisplacedarroganceabouttheimportance

andqualityofthecustomerrelationship,overlookingthefact

that people do not want to buy financial services products,

theyarejustameanstoamorecompellingend(ahousenot

amortgage,acarnotaloan,ahappyretirementnotapen-

sion).

Howtocollectcustomerdata,turnitintoinformation,anduse

it, has been the challenge that FS organizations have been

strugglingtoaddress.ThefactisthatmostFSorganizations

haveamassofdataontheircustomers,butitisverydifficult

tofindsuccessfulexamplesofinstitutionsthathaveturnedit

intoprofits.Although,weshouldsaythattherearesomeearly

examplesofsuccesswithorganizationslikeLloydsTSBinthe

U.K.providing tailoredcustomer informationat thepointof

customer interaction and some of the monoline providers,

suchasCapitalOne,usingsophisticatedcustomerdataseg-

mentation to develop tailored propositions to support cus-

tomerrequirements.However,thesesuccessesarefewandfar

between.OneofthekeyreasonsforthisisthatFSorganiza-

tionsarecomplex;consequentlysuccessfulimplementationof

CRM solutions is hard to achieve. Indeed, not many large

banks have instituted the full end to end operational CRM

systemsdue to themassivecost involved; insteadopting to

implement point CRM solutions where the business case

appears to justify the investment. Even today, despite the

tremendous investments made, many organizations do not

havethecapabilitytogenerateasingleviewofthecustomer

acrossthewholeorganization.

Justas financial institutionsarebeginning tocometogrips

withturningcustomerdataintosomethingthatcanbeused

bytheorganizationtogenerateinsight,thepictureisbecom-

ingevenmorecomplicatedbytheincreasedcomplexityofthe

FS supply chain. The problems of generating a single cus-

tomer view, which was already difficult to attain within the

traditionalsinglesuppliersituations(customerreticence,sys-

temsinconsistenciesetc),hasbecomeexacerbatedbymoves

todevelopnewbusinessmodelsandtheresultingrapidfrag-

mentationofthesupplychainfromasinglesuppliermodelto

multiplesuppliers.Asaresultofthisincreaseinsupplychain

complexity,dataisdispersedacrossdifferententities,poten-

tiallyacrossdifferentgeographies,andmostprobablyacross

differentcustomercontactpoints.Thefundamentalquestion

thatneedstobeaddressedishowcanthisnewsupplychain

complexitybesuccessfullymanaged?

The traditional focus of organizations entering into a new

venturewithapartnerhasbeenonthelegalandcontractual

issues.Once this step is completed the focus thennormally

shifts to the implementation and development of perfor-

manceservicelevelagreementstomanagethenewventure

on an ongoing basis. We would argue that this traditional

approachmissesa fundamentalpoint,which is thedevelop-

ment of the detailed understanding of the operational pro-

cesses thatwill beused inmanagingdatasuccessfully.This

understanding of the operational processes would need to

providetheframeworkforhowdataisgoingtobegathered,

managed,anddeployedacrossthenewentity.Webelievethat

it is imperative that this should be in place at the point of

enteringintothecontractualrelationship,sinceitwillhelpthe

organization to determine how each point of the customer

propositionwillbedeliveredandsupportedbyeachpartyand

makesitpossibletoinstituteacohesiveandcoordinatedsales

managementapproach.Asaresult,thecustomerproposition

doesnotgetconfusedandallrevenuegeneratingopportuni-

tiesareseized.Additionally,thefocusontheoperationaluse

ofdatawillalsoallowthedevelopmentofappropriateincen-

tivesforbothpartiesinthecrosssaleofeachother’sproducts

andservices.

Thefocusontheoperationaluseofdatawhilstimportantfor

traditionalsinglesupplierorganizationsbecomesfundamen-

tal tosuccesswhere thesituation iscompoundedbysupply

thebusiness.

conclusionSupplychainswillcertainlygetmorecomplex,andthepres-

sureonFSorganizations tomaximize thenewrelationships

willintensifyasotherareasofbusinesscomeunderincreas-

ingcompetition.Technologywillprovidesomeassistance to

organizationstomakeuseofthedatatheyarecollectingon

customersbutunlessall thepartieswithin thesupplychain

understand their proposition with respect to the customer,

anddeterminewhoneedswhatdatatosupportthisproposi-

tion,technologyspendwillbeduplicatedandwasted.

Theanswerisnotnecessarilycomplicated,buttakingaview

across the supply chain may not be straight forward. It will

increasinglybecomeimperativefororganizationstoco-ordi-

natetheireffortsbothinternallyand,increasingly,withsuppli-

erstoensurethattheyarecapturingandutilizingcustomer

datainthemosteffectiveway.Byfarthebesttimefororga-

nizations to crystallize their approach to operationalizing

customerdataiswhenthenewventureisformed.

For the banks and financial services institutions, some soul

searchingmayberequiredtorecognizethattheirbrandmay

notbekeyone,andthatthenon-financialservicesproductis

actually the one the customer really wants. In essence, FS

organizationsmustdecide if thedatatheyhavesoughtand

squanderedforyearsisreallycoretotheirsuccessinthenew

worldofmultiplesupplychainsorwhethertheyarewillingto

sacrifice customer data to generate benefits from essential

partnerships to create propositions that customers really

want.Formanyorganizationsthesoulsearchingmaybelong

andpainful.


revenuecreation.

conflicting objectivesEvenfromtheoutset,somerelationshipssufferfromalackof

clarity on the desired strategy and objectives. For example,

the idea of giving several million supermarket customers a

creditcardmaybeappealingtoaretailer,butunconstrained

itwouldgenerateriskissuesforthebankorcardsupplier.For

thebankthedealmaysimplybeabout increasingsalesvol-

umes,ineffectusingtherelationshipasjustanotherchannel

to market. The responsibility for developing the customer

relationship can also be confused, with respective partners

seeingitastheirroletodevelopthecustomerpropositionon

theirownbehalf,notnecessarilyfortheadvantageofallthe

partiesinvolved.Oftenthisisdowntothelinkedpropositions

notactuallybeinglinkedinthecustomers’minds,theAbbey/

CostaCoffeetie-upintheU.K.beingarecentexampleofcus-

tomerconfusion.

Ascustomerpropositionsgetmorecomplex,someofthesup-

plychainrelationshipsmayalsoconstrainthepartiesinvolved.

For example, smartcard use across products and channels

(potentially a significant development given EMV and the

pendingspreadofchip/PIN)couldberestrictedifthecardis

white-labeled.Ifthecardisfromamonolinesupplier,suchas

MBNA or Capital One, then their concern may be increased

competitionfromlinkingthecardtoproductstheydonotsup-

ply.Anotherexampleisthecurrentaccountmortgages,where

ifthemortgageprocessingisoutsourcedfuturedevelopment

maybeconstrainedbyselfinterest.

who really owns the relationship?Thelowestcommondenominatoroftheconflictofobjectives

is theownershipof therelationshipand itsassociateddata.

Thejointventuremodelisagoodexample.Thethreeparties

involved in a retail/bank joint venture (i.e. the retailer, the

bank,andthejointventureitself)willhavedifferentperspec-

tivesoncustomerdata,evenifitispurportedlycoveredinthe

jointventureagreement.Evenfromtheoutsetpotentialcon-

flictscanbeseen,butcertainlyastimegoesbyandstrategies

change,commonboardsandothergovernancemeasuresmay

notkeepobjectivesaligned.

Inmostsituationsallthreepartieswillholddataonthesame

customer,whomayholdthebank’scurrentaccount,shopat

theretailer,andtakeoutaloanwiththejointventure.Inthis

case,allthreewillbelookingtodevelopthatrelationship,and

will collect data to support this development. Will all three

partiesattempttocross-sellacreditorstorecard?Iftheydo,

willtheyallsetthesameparameters(creditlimit,interestrate,

etc)basedonthedatatheyhold?

Technology can only do what the data allows it toTechnology, whilst providing part of the answer, is not the

entiresolution.Substantialimprovementsinstorageandpro-

cessingcapacityarenowallowingforthekindofdataanalysis

thatprovides insightandunderstandingatacustomer level

thatmarketersonlyusedtodreamabout.However,thisisonly

part of the story. If organizations have not developed an

appropriate operational data strategy then technology will

not be able to bridge the gaps in available customer data.

Technologycanonlyprocessdatawhichhasbeengathered.

The real trick for new partners is to quickly determine the

information that needs to be collected which allows real

insightintocustomerbehaviors.Webelievethatthisneedsto

bearticulatedatthepointofenteringintotherelationshipto

allow the new entity to build a deep understanding of what

datawillbegathered,whyitisbeinggathered,andhowwillit

beused.

We are also beginning to see many of the behavioral disci-

plinesemployedwithriskmanagementbeingusedformarket-

ing,butthiswillonlywork if thedatarequiredtosupporta

decision isactuallycollected.Forexample,someinstitutions

areexperimentingwithpointofsalepricingbasedonpropen-

sities to buy, as well as risk (put simply, to charge a higher

pricefor,say,anunsecuredloan,thantheriskadjustedprice

withoutlosingthebusiness).Thisrequirescustomer‘fail’data

(aswellasdataonsuccessfulsales),whichisoftennoteven

collected.Itisthistypeofoperationaldatathatneedstobe

addressed during the establishment of the new entity to

ensureitbecomeshardwiredintotheoperationalprocessesof

101

Privacy challengesRay Everett-Church Principal and Chief Privacy Officer, ePrivacy Group, LLC

Forresterincludedadditionalpressandpublicrelationsactivi-

ties, audits and review of practices, customer service costs

arisingfromincreasedcomplaintandcustomerinquiries,and

eventhetimeandexpenseoftakingexecutivesoffmorepro-

ductivetaskstomanagethedamagecontrol.Andletusnot

forget the cost of fixing whatever went wrong in the first

place!

Butthebiggerimpact,andonemuchlesseasilymeasured,is

the harm to a company’s brand name and corporate image

whentheyareaccusedofprivacybreaches.Amajorprivacy

debaclecanundoyearsandmillionsofdollarsworthofbrand

promotion and hard-earned goodwill. If a company fails to

implementexplicitprivacyprogramstoprotectagainsteither

deliberate or unintentional privacy breaches, that lapse can

placeacompany’sbrandnameinperil,resultinginthelossof

customerloyaltyandrepeatbusiness.Thoselossesarehard

toquantify,andevenhardertoremedy.

Whiletherearesignificantcosts,companiesmustalsoremem-

ber that good privacy management can also have positive

bottom line implications. Companies that build a solid trust

relationshipwith customerswill gain competitiveadvantage

andimprovedeffectivenessofloyaltyofcustomers.Moreover,

if done correctly, respecting consumer privacy and making

investmentsinprivacy-sensitivebusinesspracticescanhavea

discernablereturnoninvestment.

Forexample,thefindingsofaconsumerstudyconductedby

HarrisInteractivein2003showedthatwhile83%ofrespon-

dentswouldstopdoingbusinesswithacompanytheylearned

hadbeenusing information improperly,91%woulddomore

businesswithcompaniesthathadprivacypoliciesthatwere

soundandverifiedbyindependentthird-parties.

Often,withnewtechnologiesandmarketingtechniques,there

isalearningcurvethatcompaniesandindustriesgothrough

inlearninghowtomakemostefficientandeffectiveuseofthe

new tools.But in thishighly-chargedatmosphereofprivacy

issues,thedangersofmiscuesareamplifiedtooftenabsurd

proportions.Therefore,companiesdonothavetheluxuryof

toomuchexperimentationandmustplantheirprivacystrat-

egywithaneyetowardsnotmerely learningfrommistakes,

butavoidingtheminthefirstplace.

Puttingtogetheracorporateprivacystrategyrequiresconsid-

erable familiarity with the policies and procedures in place

withinacompany.Simplyput,youcannotknowwhatpromises

youcanmakeunlessyouknowwhatyourcompanyisdoing

already.Sothefirststepinbuildingacoherentprivacystrat-

egymustnecessarily includeassessingyourcurrentprivacy

posture.Fromthere,youcanthendecidewhatpromisesyou

canmake,andwhatstepsmustbetakentoensurethatyou

canliveuptothosepromises.

Dealingwithprivacyrequires leadership thatcancutacross

allaspectsofcompanyoperations,fromITtomarketing,legal

to PR, front-line staff to upper management. This is why at

many Global 2000 firms, a senior executive — sometimes

styledaChiefPrivacyOfficer,orCPO—hasbeenappointedto

manage privacy issues across the enterprise. With privacy

impactingsomanyareasofanorganization,theCPOneedsto

beatapositionwithinthecompanythatheorshecansurvey

theentirecompanyandhavetheauthoritytoinfluencepolicy

andpracticewhereneeded.

CreatingaCPOcanrequireasubstantialcommitmentonthe

partofanorganization,butwhencompared to thedamage

that canoccur fromevena singleprivacydisaster, thecost

can be tiny by comparison. Like the costs of insurance or

disasterrecoveryplans,thecostsofmanagingtheriskassoci-

atedwithprivacy issuescansometimesbehardto justify in

standard return-on-investment terms. But calculating that

returnisseldomaproblemafterdisasterhasstruck,because

hindsightisalwaysdepressinglysharp.


103

Intoday’sbusinessenvironment,concernsrelatingtoprivacy

are increasingly an important area of risk management.

Myriad regulations affecting the privacy of consumer and

employee information are deeply affecting how businesses

planforandexecutetheirday-to-dayactivities.

Financial services firms are finding new privacy-impacting

regulationsespeciallychallenginggiventhatmanyofthenew

rulesseemoccasionallycontradictory.Forexample,numerous

privacy regulationsare restricting theuseanddisclosureof

consumerinformation,whileotherrulesrequiremoreformal-

ized monitoring and retention of communications between

customersandbusinesses.Andstillmoreregulationsevolving

out of post-September 11 anti-terrorism measures are now

requiringunprecedented levelsof trackinganddisclosureof

sensitivefinancialdata.Withtheseseemingcontradictionsin

howprivacyistreated,itislittlewonderthatcompaniesfind

themselvesinaprivacywhip-saw.

But ifyouthinkthearrayofnew legalrequirements isabit

paradoxical, that is nothing compared to the paradox that

arises when you look more closely at how consumers view

privacy in the context of the relationships they have with

thosecompanieswithwhomtheydobusiness.Consumersare

asdemandingasever,butincreasinglytheirdemandsregard-

ingprivacyarerivaledonlybytheirdemandsforthekindof

personallytailoredservicesthatrequiredetailedprivateinfor-

mation.

Wehavebeenstudyingthisprivacyparadoxformanyyears

andhaveseen first-hand thechallenges thatarise fromthe

conflictingdesiresofconsumerswho,ononehandwanttheir

personal information kept private, and on the other hand,

demand more personalized services. Under some circum-

stances, consumers are eager to divulge personal data and

deepen their relationship with a business. Yet, under other

circumstances,theyarenot,andtheconsistencyandlogicare

notimmediatelyapparentorpredictable.

Agoodexampleofthisparadoxcanbeseenintherecentlegal

disputes over airlines which disclosed passenger records to

companies developing security programs for the U.S.

Department of Homeland Security. Both Northwest Airlines

and JetBlue have faced investigations and private legal

actionsbecausetheysharedpassengerdatainamannerthat,

on itsface,appearedtogoagainsttheirstatedprivacypoli-

cies.Yet,whileconsumershavecomplainedthatthosedisclo-

sures were violations of promised privacy protections, it is

entirelypossiblethatthoseverysamepassengerswouldhave

joined in a lawsuit if those very same airlines had failed to

credit their frequent flyeraccounts formiles flownonaffili-

atedairlines.

Businessesandmarketersaredoingtheirbesttobesensitive

toconsumerconcerns,evenastheyworktoleverageonline

andoff-lineinformationforuseincustomeracquisition,cross-

sellingandup-selling,and inbuildingmorerobustcustomer

retention and personalization services. The results of these

efforts add to the confusion in some respects: Customers

often respond very positively to more personalized experi-

ences and availability of services through multiple channels

enabledbybetterdatasharing.

Furthercompoundingtheironyofthisprivacyparadoxisthat

bettertargetingandbetterpersonalizationeachprovidecon-

sumerswithmorerelevant informationatmoreappropriate

times,andcanevenresultingreatersavingsbecausecompa-

niescantightlyfocustheiroutreacheffortsandsignificantly

reducecosts.

Sohowarecompaniestocopewiththemixedmessageson

privacy?First,companiesneedtorecognizethatprivacyhas

adirectbusinessimpact.

Certainlytherisksofbeingprosecutedbylawenforcementor

being sued for violations of law are impacts that are easily

understood.Butyoudonotneedtofacealawsuitinorderto

facebigexpenses.A2001studybyForresterResearchpegged

theseadditionalcostsatgreaterthanU.S.$1millionforasin-

gleprivacyincidentofmoderateseverity.Thecostscitedby

Data quality management: How to produce high quality reports for risk management

Barbara Boos IT Manager, Risk Applications, European Investment Bank

interpretations of the term ‘U.S.’. While one of the reports

was based on counterparties located in the U.S., the other

report was aggregating all deals that were booked in the

bank’sU.S.subsidiaries.

Furthermore, one of the reports was based on S&P ratings

manually obtained from the Bloomberg system (manually

meansactuallytypedinbyanassistantofthecreditcommit-

tee just for this particular report), the other was based on

bank’sinternalratings.Thereportusinginternalratingshad

aproblemwithdataavailability,asoneoftheU.S.subsidiar-

ies does not maintain the rating information electronically.

This information could not be retrieved from the database

when generating the report. Consequently the respective

deals had to be excluded from the average rating and risk

capitalcalculation.

Themethodusedtocalculatecreditriskcapitalwasadjusted

twice within the previous twelve months. While one of the

reportswasonlycomparingtheabsolutefigures(lastyear’s

figurescalculatedby theoldmethod, today’s figurescalcu-

latedaccordingtothemostrecentmethod),theotherreport

accounted for the methodology changes and therefore re-

calculatedthecreditexposurefromlastyearwiththecurrent

methodtodistinguishbetweentheeffectsofmodelchange

andthechangeoftheactualportfolio.

Theaboveexampleonlyshowsasmallsubsetofallpossible

reasonsthatcanberesponsibleforradicallydifferentresults

for an apparently straightforward request for information.

Why?Itisnotatallaquestionofwhichinterpretationofthe

actualrequestforinformationisrightorwrong.Itisaques-

tion of proper definition and organization of the processes

behindreportingtasks.Sowhatarethenecessaryactionsfor

instituting the prerequisites for a sound reporting environ-

ment?

consistent reporting measuresUp front, different reports on the same subject will always

show inconsistent results if reporting measures are not

clearly defined or properly understood. Quite often it is as

simpleasreportingmeasureshavinginconsistentnamesdue

to historical reasons. This obviously needs a final clarifica-

tion.Iftwosimilarbutdifferentmeasuresarereallyunavoid-

able,thetwomeasuresneedtobenameddifferentlytomake

it apparent that theyare representingdifferent things.The

definitionofallreportingmeasuresistobedocumented,for

example,asanappendix intheriskpolicyor inadedicated

reporting handbook. To ensure that definitions are not

unnecessarily complicated and therefore are really used in

dailybusiness,itisusefultopinasinglepagesummarizingall

definitions available to every desk of the reporting team

members.

Defining reporting measures is, despite general practice,

definitelynotaone-off task,butrequirescontinuousatten-

tion to ensure all measures are kept up to date. To list an

obviousexample,therapidexpansionofthetelecommunica-

tionsindustryinthelate1990srequiredtheintroductionof

new industry sectors, such as telecom-fixed and telecom-

wireless.Anindustrysectordefinitioncommitteethatmeets

regularlytwiceayearoratexplicitrequestcouldbethebind-

inglinkbetweenthebusinessunits,riskmanagement,andall

organizational units that are impacted by a change of the

standard.

It isanabsolutemustforunambiguousreportingmeasures

thattheperson inchargeofproducingareport isawareof

thebackgroundofthequery. Incaseofanyuncertainty(‘is

the reporting recipient interested in the risk profile of the

bank’sU.S./energyportfolio,ordoeshe/sheneedsomeback-

groundinformationtorestructurethesector-basedset-upof

thecreditbusinessintheU.S.offices’),thereport-produceris

obliged to ask further questions ensuring all requirements

are understood (‘If you say region, do you mean the geo-

graphicalregionofthecustomerorthelocationofourbook-

ingoffices?’). Insomecases itmaybeannoyingtoasenior

managerhavingtoanswersuchapparentlybasicquestions,

but if corporate culture or management behavior prohibits

thesequestions,thereportingrecipientcannotbecertainto


105

Globalplayers in the financialmarketshavediscoveredthat

excellentknowledgeoftheportfoliostructureandtheassoci-

ated risks is a clear competitive advantage, and some have

evenstartedtofocustheirbusinessonmanagingriskyportfo-

lios — not without considerable success, as this knowledge

enforces flexible and market based decision making and

enablesstrategicportfolioalignment.

One key ingredient for success is decision-making based on

informationpresentedinriskmanagementreports.Thusthe

focus of these reports is to supply management with high

quality and reliable information. As reporting means aggre-

gatinglargedatasetsandperformingvariouscalculationson

a lot of time series and factor data, the quality of the final

report,andthereforethequalityofmanagementdecisions,is

directly and strongly reliant on the quality of all underlying

data.

Unfortunatelymanyreportssufferfromdataqualityproblems

anddirectlyincreasetheoperationalriskconnectedwithinap-

propriatedecisionsorevenareresponsible formissedbusi-

nessopportunities.Theimportanceofdataqualityandgood

datamanagementprocessesiswidelyunderestimatedinthe

bankingindustry.Thosewhohavediscoveredthatgooddata

management is just as important as good methodology are

alreadyonestepahead.Thisarticleoutlines themostcom-

mon reasons for data quality problems in risk management

reports and suggests improvements to data management

processes in order to achieve high quality reports for risk

management.

A real life exampleDuetoanarticleinadailyfinancialnewspaperonthenegative

developmentoftheU.S.energyindustry,theChiefRiskOfficer

of a large European investment bank contacted the head of

risk management and asked for a report about the bank’s

creditexposuretotheU.S.energyindustry.Hiscontactinrisk

managementpromisedtoprovideareportwithinthenexttwo

hours.AfterwardstheChiefRiskOfficerhadlunchwithamem-

berofthecreditcommitteeandaskedthesamequestion.

Rightafterlunch,thememberofthecreditcommitteecalled

theCROandstatedthattheU.S.energyexposurehadslightly

decreasedwithinthelast12monthsfrom9.2blnto7.9bln,

whilethequalityoftheexposure improvedfromanaverage

rating of AA- to AA+. The credit risk capital consequently

decreased by 10%. Two hours later, the CRO was caught by

surprise while reading the report provided by risk manage-

mentshowingtheU.S.energyexposurehadincreasedby8%

from6.4blnto6.9bln.Thereportalsostatedthatthecur-

rentaverageratingwasA+,butthisinformationwasmarked

as‘notfullyreliable’,astheinternalratingsofsomecustom-

ers were not available in the reporting system and the U.S.

officescouldnotbecontactedyetduetothetimedifference.

The report also stated that the credit risk capital remained

almoststableduringtheprevious12months.

what went wrong?Itmightseemimpossiblethattworeportsonthesamesub-

jects fromwithin thesamebankcanprovidesuchdiffering

perceptionsofthesameevent.But,itisnot!Amoredetailed

analysisrevealsnotjustonebutalistofreasons:

Amajortaskperformedbythecreditcommitteeisapproving

creditlimits.Thusthereportprovidedbythecreditcommit-

tee member was based on committed limits. The second

approach reported the current credit exposure from a risk

perspectivebyshowingthecurrentdrawdownplusabuffer

for potential future limit utilization for on-balance-sheet

products and a simulated exposure for off-balance-sheet

ones.

Identifyingallcustomersattributedtotheenergysectoralso

differedinthetwoapproachesofthetwodifferentreporting

teams. While the first report used the U.S.-standard (SIC)

classificationassignedonthecounterpartylevel,thesecond

wasapplyingtheEuropeanstandard(NACE)classificationon

thegrouplevel,thereby,forexample, includinganexposure

toafinancialservicesaffiliateofanenergyprovider.

Selecting the ‘U.S.’ region was also based on two different

overtimethatprovidestheactualinformationratherthanthe

exactfigureatapointintime.

The best way to compensate problems arising from model

changes is to re-validate the historic portfolio with the new

modeland,dependingonthepurposeofthereport,usethese

figureswhenevershowingdevelopmentofquantitativemea-

suresovertime.This,ofcourse,requiresthatallmodelinput

dataareavailableonahistoricbasis,andthatthereporting

systemallowsstoringseparateversionsofthesamequantita-

tive measure. This short exempt shows that model changes

havetobemadewithgreatcautionandwithaneyeonthe

potential side effects, risks of the model change itself, and

costsresultingfromthechange.

Ifmaintainingdifferentquantitativemeasuresresultingfrom

different versions of a model is not feasible, it is absolutely

necessarytoprovideadetailedtransitionanalysis fromone

versionofthemodeltotheother,allowingaqualitativeexpla-

nationofthedevelopmentofthemodeloutput.Insuchcases,

a respective comment in the report, such as a footnote, is

mandatory.

Thereisnoreasontostatethatquantitativemodelsmustnot

beimproved,butitisimportanttostrikethebalancebetween

consistencyandcorrectness.

Final quality controlBeforebeingdistributed toabroaderaudience,eachreport

hastofaceafinalsanitycheck.Forstandardreportsproduced

ataregularfrequency,thisisastraightforwardtaskandcan

evenbeautomatedtoacertainextentwithinanITsystem.For

example,byoutliningchangesofmeasuresfromonereportto

the next that exceed a pre-defined threshold. All significant

deltasinaspecificreportingmeasureshouldbeexplainable.

Theanalysisanddocumentationofsuchchanges ismanda-

torywithinthequalitativepartofastandardreportanyway.

Foranon-standardreportresultingfromanad-hocrequest

forinformation,astheonedescribedintheexampleabove,a

sanitycheckisnotatalleasytospecifyandcannotfollowa

set of fully pre-defined rules. Rather, it requires a devoted

reportingcultureaswellasagreatdealofexperiencetochal-

lengefiguresprior totheirsign-offandtoaskthequestion

‘canthisbe?’overandoveragain.Wecancontinuouslyseea

similaritywithanITsecurityofficerwhowillalwaysanticipate

security leaks when introducing a new system. Similarly, a

final quality control of a non-standard report must always

take into consideration possible reasons for causing false

figures.

why is this so difficult?Allmeasuresdescribedinthisdocumentarestraightforward

and follow a common sense methodology rather than any

sophisticated approach. They also do not necessarily raise

highrequirementstowardsITsystems.So,whyhavetheynot

been in place in all financial institutions for years? Maybe

becausetheyaretoosimple!

Reportingpeopleareanalysts.Theyreadpublicationsabout

thedevelopmentofthecreditqualityintheenergysectorin

generalorperformhigh-levelanalysesofthedevelopmentof

someportfoliomeasures.Addingdataqualitytoananalyst’s

jobdescriptionwillmostlikelybefollowedbyunpleasantdis-

cussionsabouttheanalystbeingover-skilledforthishands-on

workandthatafteradegreeinBusinessManagementhe/she

is not prepared to work on comparing data represented in

long lists and to tick each row individually after having

checkedthecorrectnessofthefigures.

Asecondpossibleansweristhattheanalystmightbewilling

toworkonthissubjectbutdoesnothavesufficientITskillsto

getintodataqualityandthatitwouldthereforebemuchmore

efficienttoassignthisjobtosomebodyelse.

Indeed,thisimportanttaskcontainsasoundelementofIT,as

dataqualityrequiresthatwelookupfiguresintheITsystems

andunderstandhowtheyaremaintainedandstored.However,

ITisofcoursenotthedriverfordataquality.Aslongasthe

business units do not see a need to put down a detailed

requestforimprovingsystemsorformaintainingworkflows,it


gettheinformationhe/she(implicitly)expected.

Data maintenance processesFurtherreasonsfor inconsistentreportsarisefromthedata

maintenanceprocesses.Whileforindividualanalyses(i.e.,as

partofthecreditapplicationprocess)itmaybesufficientto

fall back on paper-based information combined with expert

knowledge, risk management usually deals with a large

amountofdataassumingthatthequalityofthedataatlarge

isatanacceptablelevel.

The first issue that banks are facing is the quality of data

maintained intheall theirdifferentsystems.Dataquality is

anissueacrossthewholeorganizationmainlybecauseofthe

fact that the units maintaining the data do not necessarily

benefit fromgoodbutdosuffer frombaddataquality.The

secondproblemisimminentassoonasanorganizationand

itsmembersstartacceptingbaddataqualityasaGod-given

fact and therefore start to allow maintaining data for their

personal needs in separate systems, bypassing the main-

streamarchitecture.‘Weknowthattheratinginformationin

thecreditapplicationsystemsarebadastheyarenotupdat-

edonratingchanges,sowekeepourownlistthatweregu-

larly update from Bloomberg and feed manually into our

reportingtool’.Acceptingthisopensthedoortoinconsisten-

ciesalloverand,althoughgivingtheimpressionofbeingthe

easier and quicker solution, this is not at all cost-efficient

fromabank-wideperspective.

If data quality problems are presented to the data mainte-

nance teams, they frequently assert that it is impossible to

maintain a certain class of information properly. Such mes-

sagesshouldbetakenveryseriouslyinsteadofbeingignored.

Ifthecapabilitiesofthesystemreally limitthedatamainte-

nanceprocess,acommonsolutionhas tobe found immedi-

ately.Forexample,byadaptingthesystemorbyadjustingthe

maintenanceprocessassoonaspossibleifaquickfixforthe

problem cannot be supplied. However, in many cases such

messagesareputforwardduetoalackofknowledgeofdiffer-

entsystems—inthiscase,atrainingsessionwillhelpout—or

lackofinterestofthemaintenancestaff.Thiscaserequiresa

clearmessagefromthemanagementtothemaintenanceunit

that ignorance is not tolerated. Also management will be

requiredtodoregularfollowuponthedatacompletenessand

quality.

Apossiblesolutiontoensureall relevantreportingdataare

maintainedatanacceptablequalitylevelistoestablishadata

qualityofficerwhowillestablishandmonitorstandardsand

will be the central point for co-ordination and escalation in

case of incidents. The role of a data quality officer can be

compared toan IT securityofficer, and it isofequal impor-

tance.

Everybodyinthebankingindustryhasacceptedthefactthat

ITsecurityproblemscancausemajorlossesforabank,butit

has not been appreciated yet that huge operational risks

resultfrombaddataquality.Thepurposeofreportsistouse

themasabasisforbusinessdecision,butifthesereportsare

based on incomplete, inaccurate, or even false data, wrong

decisions may even have a higher impact than IT problems

causedbyanITsecuritybreach.Henceadataqualityofficer

needs tohaveabroadbacking fromseniormanagement to

ensure his enquiries are followed thoroughly by all parties

involved.

change of quantitative modelsImproving models to accurately reflect the risk profile is

essential. Unfortunately the impact of model changes on

reportingisrarelyfullyanticipated.Wheneveramodelchang-

es it becomes impossible to immediately isolate the effect

causedbythemodelchangefromtheactualchangesinthe

portfoliostructure.

Themajorobjectiveofquantitativeriskmodelsisnotneces-

sarilytoshowabsolutefiguresbuttoshowthedevelopment

offiguresovertime.Showingacreditriskcapitalfigureofx

onareportstateslessaboutaportfoliothanareportshowing

thattheriskcapitalhasreducedfromytozwhileincreasing

boththeoverallexposureandthetotalcreditspreadearned

within thesameperiod. It is thedevelopmentofameasure

107

willcontinuetobechallengingfortheITstafftohaveasystem

basedonthelatestversionof javaorusingthemostrecent

version of an interface technology instead of dealing with

hands-onproblemslikea15-year-oldASCII–basedinputscreen

providingadrop-downmenutoensureonlyvalidfigurescan

beenteredasindustrysectorcodes.

Financial institutions also spend a considerable amount of

moneyonquantitativestaffthathaveadegreeinmathemat-

ics or physics, but when designing models, they simply

assumethatallnecessaryinputdataisavailable.Aquantita-

tivemethodsteamdoesnotnecessarilycoverpracticalques-

tionslike‘whatifdataismissing’or‘howtomakedataavail-

able’.

Allthesefactsshowthatagooddataqualitymanagerisnei-

ther flesh nor fish. He/she needs a little bit of everything:

business analyst, IT engineer, and quantitative engineer. On

top of this, he/she needs a sound understanding of process

managementandsomeinterestinwhatthedailyworkofthe

data maintenance teams looks like. To act as a mediator

betweenallparties,he/shealsoneedsexcellentcommunica-

tionskills,and,mostimportantly,he/sheneedsamanagement

that is aware of the importance of this role and therefore

providethenecessarysupportincaseofescalation.

conclusionItisaseeminglyunattractivebutobviousfactthatimproving

thequalityofbasicdataandthequalityofthedatamanage-

ment processes is more important to achieving reliable

reports than using the newest, the most exact, or the most

sophisticatedmodelandmethods.

Respecting the checklist of fairly straightforward corner-

stonesthe listbelowwillprovidean importantsteptowards

improvingthequalityofdataandreportsandwillsignificantly

increasethereliabilityofreports,andsubsequentlythequal-

ityofbusinessdecisions,inyourorganization.

■ Defineandmaintainyourreportingmeasures.

■ Installadatamanagementprocessandensurediscipline.

■ Dataqualityisnotaone-offtask.Regularlymonitorthe

qualityandensurethattheprocessesreflectthebusiness

reality.

■ Recognizetheimportanceofadataqualityofficerand

enforceacceptanceinyourorganization.

■ Manageyourmodelchangescarefully—consistencyvs

correctness.

■ Introduceareportingculturetoperformconsistencyand

plausibilitycheckspriortoreportdistribution.

■ Clarifytheresponsibilityforreportinginyourorganization

—dospreadreportingrequestsacrossdifferentunits.

109

The fourth leg of the stool — Data protectionJohn P. RosatoCEO, CS Technology

Theabilitytosuccinctlyarticulateadatacenterstrategythat

ensures congruency between business goals, technology

advances, regulatory compliance, implementation options,

andcapitalexpenseissurelyvaluable.While,short-termuses

ofdataand informationmaygiveyouan immediateadvan-

tageintermsofaccessingthebusinesslandscapeorcreating

newproductsandservicestobringtomarketorhandlingyour

riskmanagement,thatadvantagecanbefleeting.Withouta

long-term protection plan for your data, without a disaster

recoveryandbusinesscontinuityprogramthatensuresthat

youwillalwayshavethenecessarydatatoachievetheshort-

termadvantage,yourlong-termgainisatperil.

Thestool’sfourthlegiswhatgivesitstability.Itisthelegthat

fortifiesyourdatastrategy,providesacheckandbalanceto

youroperations,andallowsyoutomaximizeyourapproachto

datawithintechnologyandfinancialconstraints.


111

Financialexecutivesfocusondatacreation,analysis,anddis-

seminationasthelifebloodoftheirbusinessstrategy.Atthe

creationphase,multipleinputsprovideanenormouswealthof

data.Attheanalysisphase,databecomesinformation,driving

everything frombusinessstrategy tomarketingstrategy.At

thedisseminationphase,informationbecomescorporatewis-

domasitissharedacrosstheenterprise,andthecorporation

as a whole takes a step forward. This three-legged stool is

sturdy,yethas itsweaknesses,especially if the legsarenot

wellpositioned.Whatisneededisafourthlegofthestool—an

enterprise protection strategy that safeguards the data to

makeitavailableandaccessiblewhenneeded.Atthecoreof

anyfinancialfirm’sdiscussionaboutdatashouldbetheques-

tions:Howcurrentdoesmydataneedtobe?WhatamIdoing

toprotectandmaintainmydataacrosstheenterprise?

In building the stool’s fourth leg, the enterprise must step

backandgainahealthyunderstandingofitsrequirementsfor

data, information,andwisdom.Whatdataarerelevanttoall

business lines,whatare important toonlya few?Wheredo

the lines of information security reside and how does that

relatetodataaccessanddissemination?

Inthepast,businessrequirementshavedrivendatarequire-

mentsthatwereaddressedsingularlywithinthebusinessunit.

Asaresult,data,information,andwisdomlanguishedinsilos,

unable to be shared across the enterprise. Hamstrung by

siloed applications and business line objectives, financial

enterpriseshavestruggledtoleveragethebenefitsandsyner-

giesofenterprise-widedataaccessanddissemination.

Alongtheselines,financialexecutiveshavebeenloathingto

embracebroad-baseddisasterrecovery,datareplication,and

storagestrategiesduetocostconstraints.Manyresortedto

tape back-up as the protection of choice against data loss.

Timeandrecentcriseshaveprovedthattaperecoveryisweak

atbest—withupto25-30percentdataloss.Thislossisunac-

ceptable,especiallyifthatpercentagelostismissioncritical.

Whileapplicationsoftwareisrelativelyeasytoreplace,data,

oncelost,isgoneforever.Thisfourthlegofthestooliswhat

allowsustotalkaboutthefirstthreelegsinrelativecomfort,

knowingthatifweachievetheoptimumincreation,analysis,

anddissemination,wearewellprotected.Withoutthisfourth

leg,withoutthesecurityofthedataitself,thestoolteeters.

Fortunately,datastorage,replication,andrecoverycostsare

comingdownandmost industryplayersandenterprisesare

embracingautilitymodel.StorageAreaNetworks(SANs)are

becoming a reasonable solution and there is industry-wide

acknowledgementthattapecannolongersupportthedisas-

terrecoveryfunction.

Itisatexactlythistimethatfinancialexecutivesneedtoseize

theopportunitytotakeastepbackandanalyzetheirdata—

fromcreation toprotection—at theenterprise level.Knock

downthesilos,findtheareasofsynergyandleverage,answer

thequestionsrelatingtoavailabilityanddatalosstolerance,

andevaluatetheimpactoffederalregulationsandlegislation.

Fromthere,formulateadatacenterstrategythatbuildsthe

foundation of acquiring, analyzing, disseminating, and pro-

tectingyourdata.

Thedatacenterstrategy—theprotectionofyourdata—istoo

often overlooked in the three-legged stool model until the

otherareasofdatacreation,analysis,anddisseminationare

addressed.Iarguethatthedatacenterstrategy—howyouwill

protectyourdataonceyouacquireit,howandwhereyouwill

houseyourdata,andtowhatlevelsofreplication,mirroring,

andsynchronizationyouneedtogotodeliveronyourbusi-

nessobjectives—shouldbefirstandforemost.Becauseyour

datacenterstrategyisanenterprisestrategy,beginningwith

theenterpriseviewwillallowyoutobetterleveragethecom-

monalitiesandneedsattheenterpriselevel.Datacreationis

often tactical.Dataanalysis is strategic,but can languish in

silos. Data dissemination can be both tactical and strategic.

Dataprotectionisstrategicandimpactsallareasoftheenter-

prise–technology,operations,andfinance.

Yourdatacenterstrategyhastheabilitytodefineyourorga-

nizationasawinneroraloserbecauseitisforthelongterm.

1 Surveymethodology—InFebruaryandMarch2004SASandOperationalRisk

magazinecontactedorganizationsworldwide,receivingover250responses(70%

fromfinancialinstitutions,7%fromregulators).28%ofrespondentswerededicat-

edoperationalriskteammembers.Thesampleconsistedmainlyofmediumand

largebusinesses(41%withturnoverinexcessofU.S.$1billion).Nearlyhalfof

respondentswereEuropean.

steady progress — But could do betterPeyman MestchianDirector, Risk Management Practice, SAS uK

operationalriskmanagement.Respondentsfeltthatonaver-

age,theycouldachievea10%reductiononeconomiccapital

asaresultofanoperationalriskprogram.Foramedium-to

large-sized high street bank with 20% of its U.S.$10 billion

economiccapitalallocatedtooperationalrisk,thistranslates

intoaU.S.$200millionreduction.Applyingastandardrateof

costofcapitalof10%,thismeansthattheirexpectedbenefit

fromenhancedoperationalriskmanagement isU.S.$20mil-

lionperannum.

In addition, there are of course significant benefits to be

securedasaresultofactuallossreduction.Onaverage,survey

respondentsexpectedthesetobemorethan17%.Evenifwe

apply this to reported losses (which fall well short of actual

losses) thiswoulddeliveranetbenefit ranging fromabouta

milliondollarsperyearforthesmallerfinancialinstitutions,up

totensofmillions,orgreater,forthelargerEuropeanandU.S.

firms.

I would however add a word of caution here based on per-

sonalexperience.Whenfinancialinstitutionsimplementoper-

ationalriskmanagementprograms,performancemayappear

to get worse before it gets better, because companies will

startreportinglossesthatpreviouslywentunreported.Sodo

notexpectthat17%reductioninlossestobeachievedimme-

diately,andmakesureseniormanagementarepreparedfor

some bad news or you could be setting unrealistic expecta-

tions.Howbigthetimelagisbetweentheinitialdipinperfor-

manceand theexpected improvement isdifficult topredict,

andwill varyenormouslybetweenorganizations,depending

onhowaccuratelytheyarecurrentlymeasuringrisk.Justdo

notbetoosurprisedifittakesacoupleofyears.

major obstacles to successful operational riskmanagementPerhapsthemoststrikingfindinginthesurveyistheidentifi-

cationofthekeyobstaclestoeffectiveoperationalriskman-

agement. The number one issue was ‘Difficulty in collating

sufficientvolumeofhistoricaldata’(Figure3).BasedonSAS’

experience,thisisabsolutelycorrect.Youcanhavethemost

sophisticatedanalyticaltoolsintheworld,butifyouarenot

workingwithcomprehensive,real-worlddata,youarelikelyto

misstherealdangers.Moreover, thethirdbiggest issuewas

closely related: ‘Difficulty in ensuring data quality.’ Again, if

you are working with inconsistent and inaccurate data, you

aresuretorunintoproblemsanddisagreements.Secondon

the list was the people issue of poor overall awareness of

operationalriskissues.Thisisaculturalchallengeandisgoing

to take time. The full survey report explores some of these

issues in more detail. The bottom line is that organizations


Figure1:Maturityofoperationalriskprogram.

Doesnotcurrentlyhavesuchaprogram

Morethan5years

3to5years

2to3years

Between1and2years

Lessthan12months

0% 5% 10% 15%

12%

14%

17%

22%

16%

19%

Prominentaccountingscandalsandregulatoryresponse

(eg.Sarbanes-Oxley)

Increasedshareholderpressureforoperationalriskmanagement

anddisclosure

Terroristattacksandrelatedbusinesscontinuityissues

Industry/associationtechnology/opriskinitiatives

(eg.straight-through-processing)

Concernoverlevelsofinternallosses

Internalbestpracticesbenchmarkingexercises

BaselIIandrelateddomesticregulation

1 1.5 2 2.5 3

2.318181818

2.550561798

2.594285714

2.726256983

2.859550562

3.142857143

2.212290503

Figure2:Factorsdrivingdevelopmentofoperationalriskprogram.

How long has your firm had a coordinated set of operational risks policies and

procedures in place? (184 responses)

Rate the following factors in terms of their impact on the development of your

operational risk management program.

113

The economic returns of a successful operational risk pro-

gram are significant, running into tens of millions of dollars

annually for large financial institutions, according to a new

surveyconductedbyOperationalRiskmagazineandSASUK1.

But while there has been progress over the past 12 months

thereisstillalottodo.Incredibly,afifthofglobalrespondents

donotevenhaveaprograminplace.Difficulties incollating

clean data and poor awareness among staff are the major

obstacles,andcompanieshavenotyetworkedout thebest

organizationalframeworkforaddressingrisk.

Financial institutions regard operational risk as a rather

ambiguousareacomparedwithcreditandmarketrisks. It is

less well defined and potentially a greater challenge. Given

thatitisanewerdiscipline,oursurveyaskedaboutthematu-

rityof theriskmanagement functionwithin theseorganiza-

tions, but we were startled to learn that 19% still have no

operational risk program whatsoever (Figure 1). Now you

might think this was purely a matter of geography, but you

wouldbewrong.A full 17%of thosewithoutanoperational

riskprogramareinEurope,20%inAmericaand24%inAsia

Pacific.

Wealsoasked,whatarethekeyfactorsdrivingyourprogram

(Figure2)?BaselIIwasclearlyakeyreasonbutalotofthe

drivers are to do with business. Internal benchmarking was

one,andtherewasalsowidespreadconcernabouttheimpact

of internal lossesonbusinessperformance.Recentaccount-

ing scandals and the regulatory response have raised con-

cerns.Terrorismanditspotentialimpactonbusinesscontinu-

ityarenotmajorconcerns,thoughslightlymoresothaninour

2003survey.Thelargerthecompany,thegreatertheworry

aboutterrorism.

Whenaskedwhattheyregardastheirmainsourcesofopera-

tionalrisk,respondentsputITsystemsfailuretopofthelist,

whichisconsistentwithwhatwefoundlastyear.Thebiggest

moveriscustomerrelationshiprisk—uptosecondfromsev-

enthplace in2003— though it seemstomeagreyarea. Is

customer relationship management an operational risk or a

business risk? Regulatory and compliance issues (including

taxation)comethird.Definitionalissuesaside,insurveyslike

this,companiesoftenreportwhattheycanmosteasilyiden-

tifyandquantifyasarisk.Iwouldarguethatlossofkeyper-

sonnel inabankisamuchgreaterriskthanITsystemsfail-

ures,whicharealreadysubjectedtoallsortsofcontrols,such

asdisasterrecoveryandbusinesscontinuityplanning.Inreal-

ityifyouloseoneofyourtoptradingteamsthelosswillprob-

ablybemuchgreaterbecauseitisdifficulttoplanaroundit.

But it ismuchmoredifficult tomeasurethe impactofsuch

humanlossesonKPIs,soitisoftenacaseof‘outofsight,out

ofmind.’

losses — and opportunitiesAccordingtothesurvey,theaveragetotalofreportedlosses

peryearisU.S.$18.8million.Asexpected,themostcommonly

occurring events are the smaller ones, in the order of

U.S.$100,000.Lossesdiminishinfrequencyaccordingtosize,

and it is those in theU.S.$5million toU.S.$10million range

thatseemtohavethegreatestcumulativeimpact.Ofcourse,

thelossesincreasesignificantlywiththesizeofthecompany.

Americancompaniesexperiencethehighestaveragelevelsof

loss(U.S.$23.4million)andAsiaPacificcompaniesthelowest

(U.S.$11.5million).Europeancompanies reportedanaverage

annuallossofU.S.$20million.

We asked companies to quantify the economic rewards of

Figure3:Obstaclestosuccessfuloperationalriskmanagement.

2.55

2.58

2.59

2.68

2.72

2.73

2.8

2.81

2.81

2.82

2.89

2.94

Difficultyinintegratinginternalandexternallossdata

Accesstooperationalriskexpertise/talent

Difficultyinaccessing/reportingoperationaldata

Systemintegrationissues

Difficultyinmixingqualitativeandquantitativeinformation

Inadequatemanagementbuy-in

Lackofclarityandbestpracticefromregulators/professionalbodies

Difficultyinmodellingoperationalrisk

Costandtimeofimplementation(thesheersizeoftheproject)

Difficultyinensuringthequalityofthedata

Overallawarenessandknowledgeofoperationalriskissuesamongstgeneral

staff

Rate the following areas as potential obstacles to successful implementation

of your operational risk management system.

The shift to web servicesKurt Gilman, Partner, PricewaterhouseCoopersShawn Connors, Senior Manager, PricewaterhouseCoopers

nectionorientedandsupportsecurityimplementationsat

theconnectionlevel,webservicesaremessageoriented

anddonothavetheguaranteeofadirectconnection

betweenserviceproviderandconsumer.Inadirect-con-

nectionscenario,datatravelingbetweensystemscanbe

securedbytheapplicationsorbythenetwork.

■ system coordination–Thewebservicesarchitecture

requiressignificantcoordinationamongdifferentsystems.

Forexample,awebservicesimplementationmayinclude

accessingolderapplicationsandexternalwebservices,

andinterfacingwithotherenterpriseapplicationsusing

webservicestechnologies.Thecomponentsoftheimple-

mentationarelikelytohavedifferentsecuritymecha-

nisms,complicatingboththecoordinationofauthentica-

tionandauthorization,andthemaintenanceofintegrity

andconfidentialityacrossthewebservicesinterfaces.If

securitylapsesoccurinanyofthecomponents,thevul-

nerabilitycouldputallotherparticipatingsystemsatrisk.

■ machine-to-machine interaction–Webservicesopera-

tionsarepredominantlymachine-to-machineinteractions.

Consequentlycreating,federating,andmanagingdigital

identitiesandentitlementsacrosssecuritydomainsrepre-

sentanotherchallenge.

■ interoperability–Webserviceswillalsoincreasethecom-

plexityoftesting,changemanagement,andtroubleshoot-

ingofapplicationcomponentsbecauseofthedynamic,

looselycouplednatureofthewebservicesrun-timeenvi-

ronment.

web services and conventional infrastructuresecurity technologiesSecuritydisciplinesthatareroutinelyusedintypicalenviron-

ments — authentication, authorization and access control,

encryption,anddataintegrity—alsoplayanimportantrolein

providingbasiclevelsofsecurityforwebservicescommunica-

tions. By adequately implementing controls within each of

thesesixdisciplines,weareconfidentthatanenterprisecould

todayimplementasecureandmanageablewebservicesenvi-

ronment.

Authentication–Webservicesinteractionsrequiretwokinds

of authentication. In addition to authenticating the service

consumerandprovider toeachother,principal (that is,end

user)authenticationisalsoneeded.Thefirsttypeofauthenti-

cation, between the consumer and provider, is generally

accomplishedusingconnection-orientedsecuritysuchasSSL;

principal authentication is more difficult and requires some

type of message embedded authentication token. When an

organizationadoptsawebservicesapproach,itmightrequire

additionalsecuritymeasurestoprotectandauthenticateoth-

erwiseexposedwebservicescomponents.

Authorization and access control–Authorizationiscritical

becausewebservicescanintroducecomplexlevelsofaccess.

Web services are programmatic interfaces, therefore they

maybemoredifficulttomonitorforsuspiciousactivitythan

standard applications. The Security Assertion Markup

Language (SAML) is designed to facilitate the management

and interoperability of security credentials across security

domains, but if the SAML payload is not itself protected it

couldpresentasecurityvulnerabilitythatcouldbeexploited

by an attacker. Web SSO and credential mapping solutions

thataredesignedtomaketheseenvironmentseasiertoman-

ageandsimplerforparticipantstousemaypresentanadded

securityriskifpropersecuritymeasuresarenotinplace.

session management–Webservicesaregenerallystateless.

Servicecomponents,therefore,needtoauthenticatethecli-

entoneveryaccess,unlessthepropersecuritymeasuresare

in place. Web services components may leverage web SSO

technologiesandusecookiesorSSLsessionstogranttrust

for a specific period of time. Alternatively, some solutions

offer implementations of state machines that help alleviate

thisproblem.

Encryption and data privacy – Typically, standard SSL

encryptionprovidespoint-to-pointdataprivacybetweenthe

end points of service requestors and service providers.

However, given the disconnected, loosely coupled nature of

web services, the service provider may not be the ultimate


115

First,thereweremainframearchitectures;thenclient/server

modelsevolved intoweb-basedapplications.Today,thenext

generation of innovative and large-scale architectural infra-

structures are embracing web services. It is open, intercon-

nected,andoperatesonastandardinterchange;butthereare

issues,webservicesareseeminglydesignedtobeinherently

insecure. This has led, mistakenly we believe, a number of

enterprisestodelaywebservicesimplementations.Despitean

innateopenness,thereareseveralwaystomitigatethesecu-

rityconcernssurroundingwebservicesdeployments.

Whyshouldanenterprisegetonthewebservicesband-wagon

sooner rather than later? For the best of business reasons:

webserviceswill increaseefficiency in certainkeyareasby

enabling better business processes, and implementing web

servicescouldpotentiallyreducethecostsoftechnologyinte-

grationprojects.

Thesecurityconcernspreventingtheimplementationofweb

servicesinclude:

■ conventional infrastructure security technologies–

Theabilityoffirewallsandintrusiondetectionsystemsto

protectresourcesaccessiblethroughwebservices.

■ Prevention of malicious attacks–Thepossibilitythat

newwebservices-specifichackerattacksmaybedifficult

todetectanddefeat.

■ Existing security protocols–Theinadequateprotection

forwebservicesprovidedbyexistinginfrastructuresecu-

rityprotocols,suchasSSL,andthefactthatnewwebser-

vices-specificsecurityprotocolsarestillindevelopment.

These concerns are amplified by the additional architectural

and implementation considerations introduced by web ser-

vices.

six steps to successful deploymentTheanswertodeployingwebservicesinasecurefashionisno

different than deploying security in the previous web-based

generationofcommonapplicationsandservices.Theenter-

prise will need a robust security strategy encompassing

authentication, authorization and access control, session

management,encryptionanddataprivacy,dataintegrityand

confidentiality,andsharedcontext.

The premise of web servicesThewebfollowsasimplepremise:Usestandardmethodsto

encodeinformation—HypertextMarkupLanguage(HTML)—

andtoaccessit—HypertextTransferProtocol(HTTP)—from

anywebbrowser-equippednetworkedcomputer,regardlessof

itsoperatingsystem.Asimilarnotionunderlieswebservices

andExtensibleMarkupLanguage(XML):Standardizetheway

software components and applications communicate with

eachotherovertheInternet,oracrossanynetwork,regard-

lessoftheirhostplatformsorthesoftwareenvironments in

whichtheyrun.WebservicesandXMLspecifyasimplifiedway

for systems to interoperate over the Internet, allowing the

informationresources,applicationsanddata,tobesharedand

facilitatingthedevelopmentofsoftwarethatcanaccessand

manipulatetheinformation.

Whilestandardweb-basedtrafficinvolvesHTMLfromserver

tobrowser,webservices trafficcan involveapplicationpro-

gramming interfaces (APIs) that send data back and forth

overavarietyofprotocols, includingHTTPandSMTP.Each

web service application interface may have hundreds of

operationsthatcanbeaccessed,providinghackersandother

unauthorized users with new opportunities to compromise

systems.Forexample,abankloanapplicationforarealestate

transactionmaybeexposedasawebservice.Onceinvoked,

thisapplicationmayhavetoperformanumberofoperations,

suchasverificationof identity, credithistory,andappraisal

value,aswellaspaymentcalculationorfundstransfer.Each

subsequent operation may involve other, more granular

operations.Compromisingoneofthecomponentoperations

mayprovideanattackerwithanopportunitytoaccessother

applicationsandtheirrelateddata.

Features of web services■ message based–Whiletraditionalapplicationsarecon-

ofprocessingcomponentswhosesecurityisonlyasstrongas

theweakestlink.Thislinkisthemostvulnerabletoattackand

cancompromiseothercomponentsinthechain.Exploitation

ofvulnerabilitiescanoccurinavarietyofways:

■ Datathatflowstoorfromtheweak-linkcomponentmay

beintercepted,allowinganinterlopertoacquiresensitive,

personal,orvaluableinformation.

■ Thestreamsofdatatravelingamongthecomponentsmay

bemanipulatedtoalterthedata,redirectthedata,orto

useunsuspectingserverstomountaDoSattack.

■ Acomponentmaybeshutdowncompletely,denyingits

functionalitytotheothercomponentsthatdependupon

it.Thiscaneffectivelydisruptusers’activitiesfrommany

differentaccesspoints.

conclusionTherearesomesecuritychallengesinawebservicesdeploy-

ment. However, if an enterprise approaches each of those

areas using the six-step process outlined above, it can miti-

gatethoseissuesandbenefitfromthoseefficienciesthatweb

servicesaredesignedtodeliver.


destinationforthetransactionmessage,andmayevenactas

aservicerequestoraspartofamultistagebusinesstransac-

tion.BecauseSSLencryptionterminatesataweborapplica-

tionserver,relyingonSSLforend-to-endprotectionmaybe

insufficient. Additional protection, such as using the XML

Encryptionstandard,wouldpermitencryptionofportionsof

theSOAPmessage,offeringgreatersecuritythroughoutthe

processingcycle.

Data integrity and confidentiality – An organization that

exposesan internalapplication, suchasawebservice,may

alsoexposesupportingdatastores,suchasdatabases,regis-

tries, or directories. This data must be protected, either by

encryptionor,iftheperformanceimpactofencryptionisnot

an acceptable option, by providing resource-based access

controls(accesscontrolbasedonevaluatingresourcenames

suchasthenamesofdatabasesagainsta listofauthorized

users).

shared context–Thisreferstothe informationthataweb

serviceneedstoknowaboutaserviceconsumertoprovidea

customized, personalized experience. Shared context data

may include the identity of the consumer, the consumer’s

location, and any privacy constraints associated with the

consumer information. When several discrete web services

areaggregated tocreateacompositebusiness service, the

participatingservicesneedtosharethecontextinformation.

Toadequatelyaddresstheconsumer’sconvenienceandpri-

vacy concerns, the services must employ a complex set of

rulesandsafeguardstoensuretheintegrityoftheuser’sdata

andidentity.

web services security frameworkFor effective and acceptable business use of web services,

authentication and non-repudiation must be applied to its

messagesatagranular level.Thisrequirementcouldpoten-

tially involve many users across disparate organizations in

situationswhereitmaybenecessarytoencryptandauthen-

ticateatransactioninarbitrarysequences.Also,theabilityto

provide digital identity management that can span multiple

organizations isessential forhigh-levelbusiness-to-business

transactions.Thesecomplexitiescanmakeitmoredifficultto

make enterprise-wide decisions and verify the compliance

withenterprise-widesecuritystandards.Therefore,acritical

aspect of any web services architecture must be a security

managementframeworkthatwouldallowcentralizedorgani-

zation and coordination of different security systems in an

interoperable and managed fashion. Such a framework may

notbeexclusivelypositionedtosupportwebservices,butmay

be focused on addressing broader security concerns. To

ensure that a security management framework can deliver

the desired functionality it must be implemented as an

instanceofaserviceorientedarchitecture(SOA).

Such a security management framework would extend the

notion of policy-based management to enable setting and

enforcingsecuritypoliciesacrossallwebservicespresentin

the organization. A security management framework would

include,butisnotlimitedto,thefollowing:

Trusted interoperable identities

■ Identityfederation–linkingidentitiesacrossvarious

securitydomains.

■ Authenticationsharing–exchangeoftheauthentication

states.

■ Attributesharing–exchangeofinformationabout

attributes/roles.

interoperable credentials – Issuance, exchange, and valida-

tionofdigitalcredentials.

interoperable policies –Operational, resourceaccess,confi-

dentiality,andprivacypolicies.

Trust models–Businessandcryptographictrustmodels,and

messageexchangeintegrityandconfidentiality.

web services applications and malicious attacksAswithanycomponentizedapplication,webservicesapplica-

tionsarevulnerabletoattackbecausetheyrepresentachain

117

Enterprise


Gopi chelliahChief Information Officer, Cross-Business

Technology & Operations, Deutsche Bank AG

Abstract

Transforming raw data into information and then to intelli-

gence is crucial to effective business strategy and can add

substantialvaluetobusinessoperations.Intherapidlychang-

ing financial services environment, higher decision intelli-

genceenablesoptimumuseofcapitalandimpactsprofitabil-

ity.Asregulatorsscrutinizedataintegrity,andBaselIIimposes

capitalchargescommensuratewiththecapabilitiesandeffi-

cacyofrisksystems,technologycapabilitiesmeritconcernat

thehighestlevelsoftheorganization.

ThroughitsemergingBusinessServiceModel(BSM),Deutsche

Bank is seeking to prove that the intelligent and innovative

useofdatacanbeakeydifferentiator.Contrarytothewidely

accepted view, the measure of success should not be data

deliveryalone,buttheongoingconsumptionofdatabybusi-

ness users, which can be accelerated by the refinement of

dataanditsdeliverybasedonusers’experience.

119

1 Lenson,M.,2002,‘2001:Atransformationodyssey,’JournalofFinancial

Transformation,6,73-81

2 FinancialTimes,29January2004,p.17quotingRichardWinter.


scrutinyongovernance,havecomeregulatoryandaccounting

changes.Forsometimeinternalandexternalauditors,aswell

asratingagencies,haveusedacompany’saccountingandrisk

systemsandassociateddatabases todetermine its financial

strengthandinthecaseofSAS70reports,itscontrols.What

hasincreaseddramaticallyinthelastthreeyearsisregulation

with requirements and penalties if data are not stored and

maintained with absolute integrity. While enterprises have

become proficient at converting data to information to be

usedbystandardtransactionprocessingapplications,suchas

accounting, trading, etc., they are now required to demon-

strate to regulators that they have in place suitably robust

levelsofcontrolsonhowthattransformationiscarriedout.If

basedata lackproven integrity, the firmmaynow face— in

additiontotheimmediatecosts,suchasclientcompensation,

undercharging, rerun/reproductioncosts,andmissedoppor-

tunities—the increasedriskofregulatorycensureandeven

lossoflicensetoconductbusiness.TheU.S.PatriotActneces-

sitatesthatfirmsmustbothknowtheircustomersandmoni-

toractivitiesthatcouldindicatepotentialmoneylaundering.

TheSarbanes-OxleyAct requiresseniormanagers tocertify

that,sofarastheyareaware,theirperiodicreportsincludeall

of the information considered ‘important to a reasonable

investor’andplacesapremiumoninternalcontrols,transpar-

ency,andtraceabilityofdata.

Forfinancialservicesfirms,itisprobablytheBaselIICapital

Accordthatisprovidingthebiggestexternalimpetustodata

and systems evaluation. It requires, among other things,

enhanced data and specific risk-management calculation

tools.Systemsarchitectureisamajorissueasdatadefinitions

and quality must be consistent across various systems and

businesses.Whateversystemsanddatasolutionsarechosen,

theymustbeflexibleandrobust to incorporaterefinements

to Basel II and other standards, such as the International

AccountingStandards.Whilemostbanksalreadyrealizethat

thetimelinessandaccuracyofabank’sValue-at-Riskcalcula-

tionaredeterminantsofitsabilitytooptimizeitsuseofcapi-

tal, Basel II will reward banks that can demonstrate high

standardsthroughreducedcapitalcharges.Thusdata integ-

rityandsystemssophisticationwill impact the financialser-

vicescompetitiveplayingfield.

The Business service model (Bsm)DeutscheBankdiffersfrommanyfirmsinboththeapproach

takentomeettheenterprisedatachallengeandthemeasures

ofsuccessused.CBTOisadoptingnewstrategies,butworking

withproven,standardizedbuildingblocks,anddesigningsolu-

tions for longevity, economies of scale, high quality, fit for

purpose,and lowcost.Aswithother ‘change-the-bank’pro-

grams,theapproachisarchitecture-driven,whichmaximizes

thevalueofbusinessdeliverieswhileensuringahighdegree

of standardsconformance, industrystandardization, compo-

nentization,andreuse.This isparticularly important forthe

enhancedandefficientcommunicationofITstrategyamong

all internal stakeholders, resulting in deeper alignment of

technologywiththebusinessesandwithexternalpartners.

In Issue 8 of this journal, ‘More than offshoring: Smart-

sourcing,’3MichaelBaldwindescribedDeutscheBank’ssourc-

ing strategy and effectiveness in working with third-party

partners.Thesmartsourcingapproachisacriticalingredient

thatenhancesthescalabilityoftheorganizationintermsof

expertise,speed,andcostvariability,andprovidestheparam-

etersforsignificantcommercialmodelstobedevelopedand

utilized.

DeutscheBankhasalreadydevelopedits‘functionaldomain’

modeluponwhichoperationalprocessesandfunctionalcom-

ponentscanbemapped. Indevelopment isacorresponding

modelfordatacalledthe‘datataxonomy’model,whichcap-

tures the key relationships between golden sources of data

andimplicitlyestablishestherulesofcompositionandgover-

nancenecessarytodefine,build,anddeployenterprisegrade

dataservices.

Thestandardizationoftheseservicesintermsoftechnology

andprocessistheBusinessServiceModelorBSM.Ofparticu-

lar importance is the concept of ‘consumer processes’ in

whichthebusinessusersofanyBSMserviceprovideconcrete



1213 Baldwin,M.,2003,‘Morethanoffshoring:SmartSourcing,’JournalofFinancial

Transformation,8,95-102

In ‘2001: A transformation odyssey,’1 Deutsche Bank Group

ChiefInformationOfficerMitchelLensondescribedamassive

change in the management of the firm’s technology and

operationsorganization,as itmovedfromaseriesofdispa-

rate teams with significant duplication to a single function,

Group Technology & Operations (GTO), with common goals

andobjectivesalignedtothoseofbusinessclients.

AyearagothistransformationledtothecreationwithinGTO

of Cross-Business Technology & Operations (CBTO) to pro-

mote cross-business leverage and the ‘One Bank’ vision of

synergiesincross-businessdivisionalfunctions.Putconcisely,

CBTO’sobjectiveistoenablesmarteruseofdata,information,

and ultimately business intelligence, to help Deutsche Bank

make better decisions in all areas of the firm. However, the

complexityofthechallenge,andeventhecommunicationof

thestrategiesisininverseproportiontothesimplicityofthe

objective.Thisarticlewillexplorewhythismattersmorethan

ever,anddescribetheapproachandtheprogressmadeinthis

endeavor.

The evolving role of data in the enterpriseFinancial services enterprises have substantial needs for all

typesofdata:reference(or‘static’)andtransactiondata,and

multiplelevelsofderivedinformation.Frequently,thisencour-

agesever largerdatabasesoftencontainingduplicatedcon-

tent,mistakenlyputtingtheemphasisonstorageratherthan

easyaccess, retrieval,dataqualitymanagement,andusage.

Technologicaldevelopmentsindatastorageandadownward

trend in storage costs further encouraged the belief that it

wasbothfeasibleanddesirabletostoredata in increasingly

greater volumes and in multiple copies. The range of data

types and sources (from reference data, trade-related and

transaction data, research and legal documents, to images,

audio, scanned news feeds, and up-to-the-minute market

data)alsomushroomed.Itisnolongerevenpossibleforone

organizationtooriginateandmaintainallthedataitneeds.

Richard Winter suggested that ‘…any fool can build a big

database.Thetrickisbuildingonethatisrobustandflexible

enough to be used for its intended purpose. Once the data

have been turned into information, however, the question

remainswhetheritisactedon.Databasesgetbiggerandbig-

ger,moreandmoreinformationisproduced,butthehuman

capacitytoabsorbinformationremainsunchanged.’2

Today, enterprises both generate and acquire enormous

quantitiesofdatawhosevalue,inmostcases,remainsunder-

appreciated.Theattractionofbeingabletomineenterprise

datatoachievecompetitiveadvantagehasbeenrecognized,

but despite major investments in applications, such as

CustomerRelationshipManagement(CRM),successhasbeen

elusive.Andyettheenormouslyincreasedlevelsofdetailed

informationaboutcustomersandthemarketplace,combined

withtheimmediacyoftheworldwideweb,providethefunda-

mentals to enable businesses to provide products that are

personalized to an unprecedented degree and brought to

marketincreasinglyquickly. Italsoshouldgivebusinessesa

vastlyimprovedunderstandingofthemselves,theircostsand

theirdrivers.

Meanwhile, financial service firms are under massive cost

pressure, and the speed of change is accelerating. Cost of

entry to most marketplaces is coming down and product-

basedcompetitiveadvantagelastsforevershorterperiods,at

leastintheserviceindustries.Infinancialservicestherehas

been a rise in high decision intelligence instruments and

opportunities, for those with the technology to compete.

Companies need to become increasingly nimble and that

inevitably increases the rate at which organizations change

theirstructures(includingthroughmergersandacquisitions).

Thisdemandsincreasedflexibilityinhowdataarerepresented

(ormodeled),managed,andstored.Thisisespeciallytruefor

theorganizationalreferencedatathatdrivecompanies’inter-

naladministration,risk,andcontrollingsystemsandprocess-

es.Costcentersrequireconstantrealignment—peoplemove

between departments, risk systems need to recognize new

producttypes,andsoon.

Amidst consolidation in the banking sector and increased


andmeaningfulfeedback.Thefeedbackisenabledbyspecific

workflowinteractionpatternsthatultimatelyresultinadjust-

ments in the originator process or system, improving data

quality,andoveralluserexperience.

TheBSMvaluepropositionistoenablereductioninsomeof

the common or ‘reusable’ technology base by focusing on

data provisioning across silos for controlling risk, and other

aggregateorcommonfunctions,andmakingthedatahighly

accessibleviastandardizeddeliveryandintegrationinterfac-

es. This drives down costs, improves investment efficiency,

andalsoenhancescapabilityandservicequality.

At a technology level, BSM is implemented as a ‘Service

OrientedArchitecture,’whichisorchestratedbyrelevantbusi-

nessworkflows.Itisfocusedonmakingreferenceandtransac-

tion data re-usable across the enterprise, and higher-order

information and decision intelligence services built up in an

architecturallyconsistentandstreamlinedmanner.Themain

servicecategoriesoftheBSMare:

■ Data integrity services–focusondeliveringclean,

re-usabledataas‘rawmaterial’forotherapplications.The

processesforthisincludereceiptorcapture,validation,

cleansing,repositorymanagement,dataqualityfeedback,

andaccesscontrol.

■ information management services—focusondelivering

datathathaveundergoneoneormoreprocessingsteps

resultinginvalue-addedinformationthathasoperational

andanalyticalrelevancetotherunningofthebusiness.

Processesincludetransformation,enrichment,operational

modeling,analysis,versioning,dataqualityfeedback,and

deliverytousers.

■ Decision intelligence services–focusondeliveringdata

andinformationthathaveundergonemultiple,complex

processingstepsresultingintheemergenceofhighvalue

knowledgeanddiscoveryofunderlyingpatternsthatcan

providestrategicuplifttotherunningofthebusiness.

Processesincludestrategicmodeling,strategicanalysis,

scenariogenerationandtesting,patterndiscovery,trend

detection,anditerativerefinementofmodels.

Thecasestudybelowshowshowthisisworkinginpractice:

Adoption drives success via consumer processesWhile data quality and flexibility of the business model are


Enablemulti-faceted

leverageopportuni-

tiesbyharnessing

cleandatatofuel

business-alignedmgt.

andcontrolsystems.

Discoverpatternsof

businessintelligence

fromInformationand

deliverthesein

successivelevels

ofrefinement.

information mgt. Decision intelligence

Assureintegrityin

thecreationandcon-

sumerprocessesfor

reference,peopleand

orgdata,financial

controlandriskdata,

andotherre-usable

transactiondata.

Data integrity

consumer processes & production

Figure1:TheBusinessServiceModel(BSM)

In2002,DeutscheBankrecognizedtheneedtoaddresskey

issueswiththehandlingofreferencedata—coveringclient,

instrument, and organizational data. At the same time an

integratedframeworkwasneededtoensureefficientdistri-

butionofdataacrossDeutscheBank.Toachievethesesig-

nificantrequirements,thedb-ReferenceandAi2programs

wereestablishedandhaveprogressivelyevolvedtoaddress

thechallenges.Thesechallengesledtomakingchangesand

revising approaches, which required significant innovation

andleadership.

The db-Reference program addresses some of Deutsche

Bank’skeyhistoricchallenges,suchasoptimizingtheinvest-

mentinreferencedataprojectsandtechnology,eliminating


123

the duplication of reference data sources, transferring the

organizationtoasingle,logical‘goldensource’ofcorerefer-

ence data and ensure adoption of that golden source by

downstreamsystems,andreducingthehighcostsofchang-

es in responding to changing business and regulatory

requirements.

Todothisrequiredstandardcorporatearchitectureencom-

passingfourcomponents:

■ Establishinggoldendatasources.

■ Providingfora‘staging’technologytoaddressadiverse

rangeofdatapackagingrequirementsinastandard

manner.

■ Enablingreliableandeasyaccesstothedataviastan-

dardizedinterfaces.

■ Implementingconsumerprocessesincorporatingasetof

integrated,bank-wide,controlledworkflowandgover-

nancemodelstoensuredataquality,integrity,andcon-

trol.Thisisfundamentaltoassuringintegrityinthecre-

ationofconsumerprocessesfortheuseofreference,

people,andorganizationaldata,financialcontrolandrisk

data,andotherre-usabletransactiondata.

Thedb-Referenceproductsuiteconsistsof:

db-client – Handles client adoption for major offices and

growing electronic adoption of the data by applications.

Over2300users in41countriessofar.Designedtodeliver

threestrategicgoals:

■ Golden source of client data–commonentrypointfor

requestingnewclientsoradditionaltradingaccountsfor

existingclients.

■ core details service–masteringoflegalentity(party)

details,internallyassignedmasteridentifier,andastan-

dardized,integrated,cross-bankworkflowtoimplement

KnowYourClient(KYC)complianceprocesses.

■ cross-reference service–masteringofthepartyto

tradingaccountcross-reference.

db-instrument–Deliverstimelyandconsistentinstrument

datatomultipleconsumersviathestandardcorporatearchi-

tecture.

db-organizational –Delivers timelyandconsistentorgani-

zationaldataviathestandardcorporatearchitecturefroma

singlebank-widestore.

db-client Documents (in development)–Addressesdocu-

mentmanagementandimagingrequirements.

Ai2 – The db-Reference program is synergistically coupled

with an enterprise-wide messaging framework called Ai2

(Application Infrastructure Integration) which brought

togetherexistinginitiativeswithinGTOtocreateastandard,

secure technicalapplication integrationplatform.Theplat-

formcanserveasafoundationforapplicationintegrationat

themessagelevel,providingloose-couplingandXML-based

representationsthatcanbewidelydisseminatedandutilized

underarationalizedsecuritymodel.Thesefeaturesofcon-

sistency and rapid adoption capability are fundamental to

establishingthe‘OneBank’visionbyfacilitatingcross-busi-

nessintegrationinamassiveway.

Thecommercialmodelsofdb-ReferenceandAi2havebeen

integraltohowchallengeshavebeenaddressed.Bothpro-

gramshavehadtoleadprocessandstructuralchangewithin

thebank(acrossthebusinessandfunctionallines)andhad

todevelopnewapproachestofacilitateandsustaindelivery

overmultipleyears.Thisinvolveddevelopingextensiveand

dynamic governance models to maintain the drive of the

programs—structuresthatattimesarelargerthantheproj-

ect delivery team. In addition Ai2 developed a federated

modelofcontributionwhichevolvedintosubsequentgover-

nance and project delivery models to sustain success

through the multi-year program. To support the expertise

required in both programs, new partnerships with vendors

wereforgedthatprovidecentersofexcellenceinareassuch

as application development and support, business process


Atthestart,amajorchallengefor theprojectwasthediffi-

cultyofprovingthebusinesscaseforacostcentersuchas

CBTOonthetraditional1-3yearhorizon,giventheexponential

effects of successful adoption. Key to the project's success

has been starting with achievable objectives such as stan-

dardizingnewclientset-upwithdb-Client,andunderstanding

the requirements at a low level with applications that are

simpletouse,ratherthangrandioseplansoftacklingallthe

company'sdataproblemsatonce.AsCBTOdeliversandthe

datamarketplacegrows,BusinessServiceProvisioning(offer-

ing these services on full commercial basis, with its smart-

sourcedpartners,tootherfirms)isfirmlyontheagenda.

conclusionFinancialservicesfirmsfacecostpressure,increasedcompeti-

tion,andattentiontogovernance.Thereisalsoahighdegree

ofskepticismabouttheabilitytodeliverlargeprojects,given

experiences with STP, CRM, and others. Many organizations

arefocusingondataqualityanddatadelivery,assumingthat

havingbuiltabetterdatabase,theuserswillembraceit.

Deutsche Bank’s experience has been that successful data

programs are architecture-driven to ensure standardization,

improvedcommunication,andmaximumleverage,butneedto

be user-centric with continual feedback via well-structured

consumer processes. Accelerated delivery and much of the

business risk and costs can be mitigated by choosing and

managingtherightvendorpartners.

Anoften-ignoredaspectofdataprograms is intensivecom-

munication. Users need to be involved from the start, and

much of the progress comes from iterative efforts.

Standardizationleadstobettercommunicationwithallinter-

nalstakeholdersandexternalvendorpartnersandcutscosts

through automation where possible. With projects of such

magnitude,externalexpertiseshouldcertainlybeconsidered


andoperationalsupport,anddatacleansingandsynchroni-

zation tool setswhichcanscaleupordowndependingon

thescaleofactivityordeployment.

Theseprogramshavebecomea ‘strategiccorporateasset’

for the firm, in the view of the COO for the Institutional

ClientGroupofDeutscheBank’sGlobalMarketsgroup,one

ofthepioneerusersofdb-Client.Forthesalesfunction,the

benefitsofstandardizedsetupforclientsareenormous,and

helpfullyclientsarenowmoreforthcomingwiththerequired

extensivepersonal informationas theyhavebecomemore

awareofregulatorymovestocombatterrorism,moneylaun-

dering,etc.andascompetitorsaskforsimilar information.

Usefulenhancementstodb-Clientincludeapop-upwindow

remindingthesalespersonofallthenecessaryclient infor-

mationforcompliancewhenopeninganaccountindifferent

jurisdictions.Withamulti-productsalesforce,havingacom-

prehensive view of each client is facilitating cross-selling,

andGlobalMarkets isbuildingitsowntoolsonthebackof

db-Clientfortradecapture,clientprofitability,etc.Theprog-

ressmadewithstandardizingnewclientsandtowardachiev-

ingagoldensourceofstaticdataprovidesimpetustoward

thecontinuingchallengeofretrofittingexistingclientsand

cleaningupolddatatointerfacewithdb-Client.Thebenefits

oflookingatcustomersacrossthefirminamoretimelyand

accurateway,andthespeedwithwhichthefirmcanrespond

toregulatorychanges(forexamplearecentchangeinpri-

oritiesnecessitatedbyamendmentstotheU.S.PatriotAct)

willcontinuetomultiply.

InarecenttradeshowofDeutscheBankCIOs,thedb-Refer-

enceandAi2programswonawardsfor‘Strongestalignment

withGTOstrategicgoals.’(Thesearethe4Csofconsistently

highclientsatisfaction,best-in-classcostperformance,scal-

ableandresilientcapability,risk-weightedcontrol.)


importantaspects,thecruxofsuccessisreallyhow,andhow

widely,theBSMserviceisused.Atmanyfirms,theattention

is focused on the database, perhaps with accompanying

efforts toward straight-through processing (STP). In CBTO’s

view,thatisonlyoneaspect;theoverallchallengemostlikely

splitsintothreeequallyweighteddimensions:

■ Makingthedatabase‘goldensource’(accurateand

reliable).

■ Providingcertificationofdata.

■ Ensuringadoptionbybusinessusers.

Whilethedatamustbegolden,itisequallyimportanttocre-

ate the workflows to get the data to everyone certified to

receiveit(fully-fledgedcertification).Crucially,successisnot

thedeliveryofdata,it isbusinessusersconsumingthedata

andaskingforchangessothattheycanconsumeitevenmore

effectively.

Amajorchallengeisobviouslydeliveringgoldendata,which

is often constructed by merging corporate and local data

sources,toline-of-businessapplications.Itisatthisveryearly

stagethatattentionneedstobepaidtotheconsumerprocess

andfeedbacksolicited.Asnotedearlier,associatedwitheach

BSMserviceisaconsumerprocesswhichprovidesaqualita-

tive measure (against an internal data benchmark) and

definestheworkflowtoenabledataquality improvement. In

effect the consumer process implements a feedback loop

between the application consuming the data from the BSM

serviceandeachofthecomponentsourcesofthedata.

Theconsumerprocessis invokedwhenanerror isdetected;

manyofthese(i.e.dataomissions)aredetectableatthestruc-

tural level and therefore can be automated. More general

errordetectionhappensintheconsumingcontextofanappli-

cation, and is typically found by human users and manually

expressed through an exception reporting screen. The con-

sumerprocessmodeloffersasemanticframeworkfordefin-

ingtheactualerror,throughwhichthemodelcancomputethe

potential component source. The error report is routed and

subsequently tracked against an owner for the potential

cause.

The consumer process model allows basic metrics of data

qualityatthedeliverypointtobedynamicallymaintained.In

addition,itprovidesahighqualityrepositoryoferrorsforroot

causeanalysistoolsandtoolsforlong-termkeyperformance

indicatorsondataquality.Thisiscertainlyacaseoflearning

andimprovingfromerrorsastheyoccur,andgivesdepthto

data quality by ensuring that root causes are addressed,

eliminatingrepeatedoccurrencesofthesameerrorevents.

Insomecasesconsumerprocesseswillaffectprimaryorigina-

tion workflows. For example, within the db-Reference pro-

gramdescribedinthecasestudy,db-Clientcoredataarethe

result of a global corporate process called Know Your

Customer (KYC), implemented across Deutsche Bank to

ensurebestpracticecompliance.KYCdecomposes intosub-

processesownedbycreditrisk, legal,sales,compliance,and

otherindependentfunctionswhichallhaveaninterestinthe

db-Client model. The consumer process associated with db-

Client has knowledge of and can challenge, from a quality

perspective, any or all of the sub-processes that generate

componentsofadb-Clientpayloadofdatadeliveredtoacon-

sumingapplication.

‘Viral effect’ and data marketplace creationTheBSMmodelaimstoinvolveitsconsumers(businessusers)

inaprocessofdiscovery,andthisleadstotheviralnatureof

itsadoption,characterizedbyacceleratedgrowthovertime.It

isanintriguingbalanceofsupplyanddemand,asCBTOmust

doalltheworktosupplygoldendata,butitsusewilldepend

on success of the consumer processes and feedback from

consumerstoreachimproveddataqualityandultimatelycer-

tification. When business users perceive the data is golden

andcertifiedandcanbeinputintohigherintelligenceusage

scenarios,thedemandshouldspreadinaviralfashionanda

datamarketplacewillhavebeencreated.Thedifficultymay

becomekeepingupwithdemand.

125


and can bring scalability, accelerated delivery, mitigation of

businessrisk,andmorecontrolofcosts.

The benefits of intelligent and innovative use of data have

neverbeengreater;indeed,manywouldarguetheyareessen-

tial for survival. With a more competitive environment and

increased regulatory scrutiny, particularly post-Sarbanes-

Oxley,countlessexecutivehourswillbewastedunlessusers

areassuredofreliablegoldendatawhichITiswillingtocer-

tify.Onlythencanbusinessusersactuponwhatthenumbers

tellthemtoachievebetterbusinessdecisionsandcompetitive

advantage.


Enterprise

Extracting the business value of iT: it is usage, not just deployment that counts!

Donald A. marchandProfessor of Strategy and Information

Management, IMD and Chairman and President of enterpriseIQ®

Abstract

BusinessManagersdevote90%oftheirtimeandattentionto

ITinvestmentsanddeploymentthatonlyaccountfor25%of

the business value of IT. The residual business value of IT

(75%) resides in increased usage of information and IT by

managersandemployeesinternallyandwithcustomers,part-

ners,andsuppliersexternally.Seeing,measuring,andimprov-

ingtheinformation,people,andITusagepracticesinacom-

pany impacts directly effective information and knowledge

use and business performance. For some companies, their

InformationOrientation is thedifferencebetweendeploying

andusinginformation,people,andITforcompetitiveadvan-

tageversuscompetitivenecessity.Knowingthedifferencecan

makeabigdifferenceincompanyperformance.

127

1 Carr,N.2004,‘ITdoesn’tmatter,’HarvardBusinessReview,May,41–49.

Itisinterestingthatthebookisentitled‘DoesITmatter?’(HarvardBusiness

SchoolPress,2004),whichappearstoleaveopentheviewthatITmaydeliver

valueincompaniesundercertainbusinessconditions.

2 Brynjolsson,E.,andL.Hitt,1996,‘Paradoxlost?Firm-levelevidenceonthereturns

ofinformationsystemsspending,’ManagementScience,42:4,541-558.And

‘Computingproductivity:Firm-levelevidence,’MITSloanSchoolofManagement,

CenterforeBusiness,WorkingPaper139(June2003).


suredintermsofprofitability,thusleadingsometoconclude,

likeCarr,thatthereisnorealrelationshipbetweenITspend

anddeploymentwithselectedcompaniesachievingcompeti-

tiveadvantageinindustrysectors2.Itwouldappearthatboth

the business commentators and researchers are correct —

thatthemaineffectofITinvestmentsanddeploymentisits

contributiontocompetitivenecessity.Youcannot‘not’spend

onITasacompany,butyoudonotwanttospendanymore

thantheminimumnecessarytodeployandrunITefficiently

inyourcompany.

Thus,ourdisenchantedbusinessmanagersareledtobelieve

that IT is a commodity and deploying it should be done as

economically and efficiently as possible — the partly ‘right’

elementof IT’sbusiness impact.However, formostbusiness

managers, the actual usage of information and IT in their

companies isa ‘blackhole’, consuminghumanenergy, time,

attention,andresourceswithoutanyvisiblefocusbybusiness

managersonitsimportancetorealizingthebusinessvalueof

informationandIT.

AmajorreasonwhyusageofITandinformationinacompany

islargelyignoredoroverlookedbybusinessmanagersisthat

ITinvestmentsanddeploymentarethemostvisibleandhard

factors to measure and manage. Business managers spend

90%oftheirattentiononplanningITprojects,aligningITwith

thebusinessneeds,budgetingandinvestinginIT,anddealing

withthe IT functionandexternalsuppliers.Othermanagers

naivelyseektosolvebusinessproblemswithITthinkingthat

theseproblemswillgoawayiftheyimplementIT‘solutions.’A

very common reason why companies have invested in the

deployment of Customer Relationship Management (CRM)

systemsoverthe last5yearstosolvetheperceived lackof

‘customer orientation’ of their companies. What most busi-

nessmanagersdidnotseeclearlyisthatCRMsystemshave

to be accompanied by significant behavioral and cultural

changesinthewaysinformationandITareusedbypeoplein

business functions such as sales, marketing, and servicing.

Thesesoft factorsofhowpeople inacompanybehavewith

information and use IT have not been perceived, measured

verywell,ormanagedconcurrentlywith the introductionof

CRMsystems.Thishasleadtobusinessdisappointmentsover

theresultsobtainedfrommajorCRMinvestmentsanddeploy-

ments.

Incontrast,academicresearch,asFigure1suggests,pointsto

only 20-25% of the business value of IT being linked to

deploymentandinvestments,whereasresearchsuggeststhat

75-80%ofthebusinessvalueofITandinformationislinked

toso-calledsoftfactorshavingtodowiththeusageofinfor-

mation and IT by managers and employees in the company

andexternallybycustomers,partners,andsuppliers3.Sohere

wecometoan importantdisconnect in theperceptionsand

mindsetsofbusinessmanagers.Mostmanagersdevote90%

oftheirtimeandattentiontoITinvestmentsanddeployment

thatonlyaccountfor20-25%ofthebusinessvalueofITand

information! The focus on the effectiveness of information

and ITusagebypeoplereceivesmuch lessmanagerial time

and attention. Thus the potential business value residing in

increased usage of information and IT in the company by

people is largely lost. The result is that business managers

often overlook the largest potential payoff from IT by not

focusing on how effectively information and IT are used by

themselvesandtheirpeopleintheircompanies.

iT deployment differs from iT usageThe deployment and usage of IT differ in managerial focus,

mindset,andmeasuresofperformance(Figure2).Ontheone

side, IT deployment is concerned with the governance and

provisionofITservicesinmostcompaniesfromnetworks,to

ITprojects,tothedailymanagementoftheITfunction.Onthe

other side, IT usage is concerned with the organization of

decision rights in a company around lines of authority,

accountability,andexpertisesothatideallyinformationandIT

canbeusedeffectivelyindecisionmakingatallappropriate

levels.CaninformationandITbeusedbypeopleintheopera-

tional and business processes of a company as well as for

managerialdecision-making?Herethebehaviorsandvalues

— such as integrity, sharing, and transparency — that lead

peopletouseinformationandITeffectivelyintheirworktake

128


1293 Brynjolsson,E.andL.Hitt,2002,‘Intangibleassets:Computersandorganizational

capital,’MITSloanSchoolofManagement,CenterforeBusiness,WorkingPaper,

(October2002).AndMarchand,D.A.,W.J.Kettinger,andJ.D.Rollins,2000,

‘Informationorientation:People,technologyandthebottom-line,’Sloan

ManagementReview,41:4,69–80.

If you ask senior executives and managers as I do in my

executive programs at IMD whether their companies are

extractingtheexpectedbusinessvalueoftheirinvestmentsin

IT,theoverwhelminganswerbyalargemarginis‘no’!There

areanumberof reasons for this.Some focuson thedisap-

pointments their companies have experienced with imple-

menting Enterprise Resource Planning (ERP), Customer

Relationship Management (CRM) and various ‘e-systems’

internallyandexternallyduringthe lastseveralyears,espe-

ciallythroughtheboomandbustofthedot.com‘e-everything’

era.Manybusiness-orientedITprojectshavefailedoutrightor

not livedup toexpectations.Blame isoftensharedonboth

sidesbetweenthebusinessandIT.

Others point to a disconnect between what the IT industry

promised them, namely that IT investments would lead to

‘competitive advantage’, and what IT in their industries and

companies has delivered — ‘competitive necessity’ — where

mostfirmscompetinginthesameindustryaredeployingthe

same IT and using it more or less for the same purposes.

ExecutivesknowthattheircompaniesmustspendonIT,but

want to invest no more than is necessary to keep up with

competitors — a major reason why the IT function is under

continuouspressuretoreducecostsanddeploystandardized

systems and processes more efficiently. The competitive

necessityargumenthasrecentlyrisentoprominencethrough

the May 2004 Harvard Business Review article by Nicholas

Carr,provocativelyentitled‘ITDoesn’tMatter?’1Carrargues

thatsinceITisequallyavailabletoallfirmsinanindustry,and

evenacrossindustries,tobedeployedinsimilarways,itisnow

moreofacommoditythanvaluedeliveringtechnology.

What is interesting about the ‘competitive necessity’ argu-

mentbyCarrandothercommentatorsisthatitispartlyright

and partly wrong! Academic researchers have found during

thelasttenyearsthatITinvestmentsbycompanieshavecon-

tributed to their increased productivity and also increased

price competition in industries. The chief beneficiary of IT

deploymentshavebeencustomerswhohavebeenabletobuy

moreproductsandservicesfromcompanieswithless,andthe

IT industry itselfwhichhasdirectlybenefited from these IT

investments.BrynjolssonandHitthavefoundnocorrelation

between IT investmentsandcompanyperformanceasmea-

Visible hard factors

ITdeployment(25%)

invisible ‘soft’ factors

InformationandITusage

bypeople(75%)

Managers

Employees

Customers

Partners

Suppliers

Figure1:ThebusinessvalueofITvsdeploymentofIT

Deployment focuses on:

• ITgovernance

• ITservicesandprocesses

• ITinfrastructure

• ITapplicationsand

datamanagement

• ITresources

(peopleandexpertise)

• ITinvestments

iT deployment measures focus on:

• ITavailabilityandaccess

• QualityofITservices

• Usersatisfaction

• Costreductionand

standardization

• EfficiencyofIT

• ITROIforprojects

• ITorganizationmaturity

(e.g.GartnerModel)

Figure2:ITdeploymentdiffersfromITusage

usage focuses on:

• Organizationandgovernanceof

decisionrightsinabusiness

• UsageofITandinformationprocesses

formanagementdecision-making

• UsageofinformationandITin

operationalandbusinessprocess

management

• Thebehaviorsandvaluesofpeople

thatleadthemtouseITand

informationintheirwork

iT usage measures focus on:

• InformationandITusageinthe

business

• Effectivenessofinformationuseby

people(InformationOrientation

Maturity)

• Contributiontotoplinegrowthrelative

to:• Profitability• Marketshare• EBIT• EVA• Innovationinproductsandservices• Companyreputation


tion, and focus are on the implementation of the large IT

project.TheROIanalysisfortheprojectshowsapositivenet

returnbyassumingthattheintendeduserpopulationwilluse

thesystemanddataafterimplementationiscomplete.TheIT

projectgoesforward.Thesystemanddataareimplemented,

but no real training of users occurs and senior managers

assume that because the project is implemented it will be

used.SinceITdeliveredtheproject,theirjobisdoneandthey

declaresuccess,whileweeksandmonthsgobywithnovisible

or deliberate attention to who is or is not using the actual

systemanddatadeployed.Over time, the intendedusersof

thesystemrealizethatusageispoor,butthereisnoproject

orbudgettofixtheseproblemsrelatedtohowpeoplearecol-

lecting or maintaining the information after the project is

deployed.Overtime,userstrusttheinformationinthesystem

lessandlessandrelyoninformalhumannetworksorinvent

their own approaches to information use using spreadsheet

tools.Theresult: ITdeploymentwasOK,butusagedeclined

over time and so did the expected business value of the IT

systemtothecompany.

Inthethirdcase,therelationshipbetweenITdeploymentand

usagemaybeadditivewheretwoplustwoequalsfour.Here

theresultsareOK,andarebetterthantheprioralternatives,

sobusinessmanagersinthecompanysettleforcompetitive

necessity indeployingandusing IT.Therearemanycompa-

nies today where business managers have low expectations

abouthoweffectively their companycandeployanduse IT,

eitherduetothetorturedhistoryoftheITfunction’scapabili-

ties inthecompanyorthemixedresultsfrommanyITproj-

ectsthatwerepoorlyimplementedorfailedtomeetexpecta-

tions for value. In these companies, managers realistically

assumethat if they lowerexpectationsabout ITanddeploy

fewerorsimplerprojectsusing IT, then theycan followand

perhapsevenkeepupwiththeircompetitorsintheindustry,

butnotaspiretocompetitiveadvantage.Inthiscase,business

realism dictates doing the basics well, but not aspiring to

achieveinformationcapabilitiesthatthecompanyhasneither

theinternalexpertisetodeploy,orthebusinesscapabilitiesto

use.Settling foranOKbusiness result is seenasa realistic

positiontotake.

Inthefourthcase,therelationshipbetweendeploymentand

usage may be a multiplier of value, since deployment of IT

inside and outside the company is leveraged with excellent

usageonthepartofmanagersandemployeesinternallyand

perhaps with customers, partners, and suppliers externally.

Again, researchers have identified companies in different

industriescapableofachievingamultipliereffectwithgoodIT

deploymentandwidespreadusageof ITand informationby

peoplethatleadstoexceptionalandsustainablegainsinbusi-

ness performance. What do these managers know about IT

deployment and usage that others do not or cannot imple-

ment?

iT deployment enables, but does not drive effec-tive information and iT use in a companyOurresearchhasdemonstratedthatbusinessmanagerssee

thebusinessvalueof informationandITasgoingbeyondIT

deploymentandthe ITfunctiontotheknowledgeand infor-

mationembedded in theirpeople4.Businessmanagerspos-

sessabroaderviewofeffectivenessininformationandknowl-

edgeusethat incorporatesthehumanbehaviorsandvalues

relatedtoinformationandITusageandpractices.Wecallthis

theInformationOrientation(IO)(Figure4)ofthecompany.We

haveproventhatmanagersbelievethateffectiveinformation


Figure3:KeystrategicchoicesaboutachievingthebusinessvalueofIT

Deployment ? usage = Business value ?

Deployment - usage = Dilutes value

Deployment + usage = oK return

Deployment x usage = multiplier effect

DeploymentofIT

insideandoutside

thebusiness

Fordrivingbusiness

performance

UsageofITby

managers,employees,

suppliers,customers,

andpartners


oncriticalimportance.

Similarly,themindsetofdeploymentismoretechnicallyori-

ented and concerned with aligning IT to the business, the

developmentofITstrategy,andthemanagementoftheport-

folioofITprojects.Ontheotherhand,themindsetofITand

informationusageismorebusinessoriented,concernedwith

businessfacingprojectsthatenhancecustomervalue,leadto

product innovations, or improve customer loyalty through

improvedknowledgeandinformationusage.

Finally,ITdeploymentmeasuresthequalityandavailabilityof

ISservices,ITcosts,andROI,aswellasontheefficiencyofIT.

Noneofthesemeasureshavebeenfoundtobedirectlycor-

relatedtobusinessperformance.Incontrast,thefocusonIT

and information usage is concerned with measuring IT and

information’scontributionstothebottomlineandtheeffec-

tivenessof informationusebypeople inthebusinesstothe

achievementofbusinessresults.

ThesedifferencesinITdeploymentandusageraisestrategic

choices by business managers that have very different

impactsonachievingthebusinessvalueofIT(Figure3).

InthefirstcaseinFigure3,dobusinessmanagersinacom-

panyreallycareaboutorwanttoknowtheimpactofdeploy-

mentandusageofITonbusinessresults?Duringperiodsof

rapidgrowththroughmergerandacquisitionorafterperiods

ofrestructuringanddownsizing,manybusinessmanagersare

lessworriedaboutITdeploymentandusagethanconfronting

the challenges and chaos of rapid change in the scale and

scopeofthecompany.Aftercompleting30plusacquisitions

toredefineitsbusinessoverthelast7-8years,managersofa

large specialty company had to redefine the structure, pro-

cesses,andcultureofthetheircompany—literallyre-invent-

inganewcompany.Ithastakenthesemanagers2-3yearsto

movebeyondbasic restructuringandrationalizationofpeo-

ple,processes,and ITresourcesto lowercostsandtheyare

onlynowbeginningtoseekmorebusinessvaluefrominfor-

mationandITusagebytheirpeopleforgrowthandnewbusi-

nessdevelopment.

Inothercompanies,ITdeploymentisconsideredbybusiness

managers to be synonymous with usage, so that the usage

dimensionisinvisibletothem.Inthesecases,ITprioritiesand

deploymenttendtodrivethebusinessusesofITratherthan

the other way around. A common view among managers is

thatinformationmanagementactivities—suchascollecting,

organizing, and maintaining information about customers,

products,andoperations—isboringandisusuallydelegated

tolowerlevelsinthecompany.People,inturn,attheselower

levels,suchassalesoroperationalstaff,recognizethatsenior

managers rarely pay much attention to these information

managementpracticesand thusconclude that they toocan

performtheseresponsibilitieswithlesscareandquality,since

theyarerarelyrecognizedorcompensateddirectlyforthese

activities. Thus, these less visible information practices

obscuretheirrealimportancetothequalityandvalueofinfor-

mationfordecisionmakinginthecompany.Nowonderman-

agers insuchcompaniestreatdeploymentanduseof infor-

mationandITassynonymous!

Inthesecondcase,therelationshipbetweenITdeployment

andusagemaydilutebusinessvalue,sinceneitherITdeploy-

mentnorusageisthedirectfocusofmanagerialattentionor

they are poorly implemented. There are companies today

wherebusinessmanagershaveabdicatedtheir involvement

in business and IT decisions to such an extent that the IT

function drives IT deployment and indirectly usage in the

business.Inthesecases,ITdrivenprojectsseldommeetbusi-

ness criteria for functionality, relevance, and payback, and

maythereforedilutevalue inthecompanywhere ITcannot

win business credibility with its initiatives and the business

managerscannotunderstandwhyITimpactissolowintheir

company.

Instillotherinstances,ITdeploymentmaybedonewell,but

noonepaysattentiontousagesoitsbusinessvalueisnega-

tive.Forexample,afinancialservicescompanycommitsitself

to a large IT project for managing the company’s financial

reportingandaccounting.Allavailableexecutivetime,atten-

1314 Marchand,D.A.,W.J.Kettinger,andJ.D.Rollins,2001,Makingtheinvisiblevisible:

Howcompanieswinwiththerightinformation,peopleandIT,JohnWileyand

Sons,LondonandNewYork

5 Marchand,D.A.,W.J.KettingerandJ.D.Rollins,2001,Informationorientation:The

linktobusinessperformance,OxfordUniversityPress,Oxford

6 TheIODiagnosticandBenchmarkaretrademarkedproductsofenterpriseIQbased

inLausanne,Switzerland.Seewww.enterpriseIQ.com.



Figure4:TheInformationOrientationMaturitymodel

Source:Marchand,D.A.,W.J.KettingerandJ.D.Rollins,2001,MakingtheInvisible

Visible:Howcompanieswinwiththerightinformation,PeopleandIT,NewYorkand

London:JohnWileyandSons

information orientation (io)

information technologypractices

ITformanagementsupport

ITforinnovationsupport

ITforbusinesspro-cesssupport

ITforoperationalsupport

information management

practices

Sensing

Processing

Maintaining

Organizing

Collecting

information behaviors and values

Proactiveness

Sharing

Transparency

Control

Formality

IntegrityMa

turi

ty


andITusagebypeopleislinkedtobusinessperformance5.

Inaddition,wealsoknowthatbusinessmanagersseevaluable

informationandITasembeddedinthebusinesscapabilitiesof

their companies. Thus, business managers seek business

valuefromthemarketplacewiththeirbusinessstrategiesand

thebusinesscapabilitiesrequiredtoexecutethosestrategies.

Some business managers go beyond this view of value cre-

ationtodeveloptheinformationcapabilitiesofthecompany

requiredtoextractanduseknowledgeandinformationbetter

than their competitors. This process of building the people,

information, and IT usage capabilities of a company we call

theinformationorientationmaturityofacompany.Moreover,

since the IO maturity of a company can be measured and

benchmarked,businessmanagerscanexplicitlyevaluatetheir

progress indevelopingtheircompany’s informationcapabili-

tiestobuildbusinessvalueovertime6.

Thus,ITdeploymentcanenable,butnotdriveusageofinfor-

mation and IT by people in a company. Creating value with

effectiveusageofinformation,people,andITinacompanyis

the multiplier of IT deployment. Moreover, the information,

people,andITcapabilitiesneededforhighIOmaturityaredif-

ficult to replicate. Companies can enjoy competitive advan-

tageinhowtheyintegratetheirinformationcapabilitiesinto

their business models and strategies over time. Some firms

can be better positioned than others in achieving both the

outputandproductivityimpactsofgoodITdeploymentwith

themultipliereffectofinformationorientationmaturitythat

leverageshoweffectivelyinformation,people,andITpractic-

esworktoimpactcompanyperformance.

iT does matter, but effective information and iT use by people matter more!Effective management of IT deployment and function in a

companydoesmatter.CompaniesmustdeployITat leastas

effectivelyastheircompetitorsandtheyshouldseektodoso

asefficientlyaspossible. Ifacompany’sbusinessmanagers

donotseektoalignITstrategyandprojectswiththeirbusi-

nessneedsanddevelopanoperationallyeffectiveITfunction,

thentheycannotplayatthetableofcompetitivenecessityin

theirindustryorglobally.However,settlingforgoodITdeploy-

ment without concurrent focus on usage means that these

managerswillfallfarshortoftheobjectiveofoptimizingthe

businessvalueofITintheircompany.

Ironicallyforbusinessmanagerswhoseektoleveragethefull

valueofITintheirbusiness,theyneedtofocusontheeffec-

tiveuseof Information,people,andIT intheirbusinessfirst

and then align IT deployment with their business strategies

andcapabilities.Thisimportantchangeinmindsetisrequired

so that the linkagesbetween informationcapabilitiesof the

companyareembeddedinitsbusinessmodelorwayofdoing

business. Then, it is possible to guide IT deployment to

achievethemultipliereffectaroundeffectiveinformationuse

inthecompanyandnottheotherwayaround.

Competitiveadvantagegoestothosebusinessmanagersand

companies that deploy and use information, people, and IT

moreeffectivelytoimpacttheirgrowthandbusinessperfor-

mance in their industry and globally. They strive to extract

100% of the business value of effective information and

knowledge use in their business through improving the IO

maturity of the people, information, and IT practices. They

seektocapturenotonlythe25%ofthebusinessvalueresult-

ingfromefficientITdeployment,butalsogoafterthe75%of

thebusinessvalueresultingfromeffectiveuseofinformation

andknowledgebytheirmanagers,employees,customers,and

partners.AchievingthemultipliereffectfromhighIOmaturity

in theirbusiness isastretchobjectiveworthmotivating the

leadersofthesecompaniesthatareseekingtoextractmaxi-

mum value from the three most expensive resources in a

company—theirpeople,informationandknowledge,andIT!

133

Enterprise

Taking snapshots of the internet: new database of insider transactions and liquidity

steven m. BenvenisteResearcher, The Harold Price Center for

Entrepreneurial Studies, The Anderson School, uCLA

Duke K. BristowFinancial Economist,

The Harold Price Center for Entrepreneurial Studies, The Anderson School, uCLA

Alfred E. osborne, Jr1Director of The Harold Price Center for

Entrepreneurial Studies and Associate Professor of Business Economics,

The John E. Anderson Graduate School of Management, uCLA

Abstract

Inaworldawashwithdata,itissurprisingthatimportantdeci-

sionswouldeverbemadeintheabsenceofdata.However,the

paucityofwidelyavailableresearch-orienteddatabasesofour

rapidlychangingfinancialmarketscriticallylimitsacademical-

ly-rigorouspreemptiveinvestigations.Toooftenpublicpolicy

decisionsconcerningthetransformationofourfinancialmar-

kets,forexampletheSarbanes-OxleyActof2002,aremade

intheabsenceofscientificevidence.Lesscostlyandtimelier

databasesareneededtobetterevaluatecapitalmarketsleg-

islationandregulations—beforetheybecomelaw.Thispaper

examinesanewdatabaseofoveronemillioninsiderfilingsfor

13,000 U.S. securities and a new methodology for rapidly

buildingotherdatabases.Animportantnewclassofdatabas-

es termed ‘snapshots of the Internet’ is described. These

databasesmayhelpfillcriticalgaps inourknowledgebases

and improve the quality of national and international eco-

nomicpolicydecisions.

1351 TheauthorsthankthecomputersupportgroupattheUCLAAndersonSchool,

especiallyMr.WesleyWong,andthehelpdeskatNasdaq.comforexcellenttechni-

calassistance.TheNasdaqStockMarketisthankedforaccesstothedataforthe

non-commercialpurposesofresearchandscholarship.Thedatabasedescribed

hereinwasfirstusedinBristowandOsborne(2002)entitled‘Rule144Twenty

YearsLater:ConstraintsonEntrepreneursandtheEconomy.’LauraBristowand

DebbieDutraarethankedforproofreading.Theremainingerrorsareourown.


liquidity,thispaperoffersalow-costsolutiontoalargerprob-

lemconfrontingallfinancialresearchers.Inthegeneralcase,

as increasingnumbersof investorsget theirprimary invest-

mentandcompanyinformationfromtheInternet,itisincreas-

inglyimportanttohavearecord,anarchive,ofexactlywhat

wasavailabletoinvestorsonagivenday.TakingwhatInternet

programmersrefertoas‘snapshots’oftheInternet,weoffer

a step toward solving this growing problem. Unlike financial

publicationsofthedistantandevenrecentpast,suchasdaily

andweeklynewspapers,withtheirstockpricetables,dividend

tables, insider sales reports, tender offer announcements,

etc.,theInternetisahighlydynamicdatasource.Lastyear’s

newspapers can be re-examined. Last quarter’s dividend

tablescanbere-checked.However,yesterday’sInternetcan-

notbere-checked.

Eveniftime-consuming,thedatabasesonwhichthefinance

andaccountingliteraturewerebuilt,theCRSPandCompustat

databases,4canbehand-checkedagainsttheoriginalpaper

sources.Theresultisthatthedataareeitherwhattheauthor

says theyareor theyarenot.With the Internet it isnot so

easy.Asourfinancialmarketscontinuetobetransformedby

theInternet,itisincreasinglydifficultifnotimpossibletogo

backtoonehistory.Thedatathemselvesmaybealteredeven

for historical records. And the Internet lacks any hint of an

audittrailthatwouldallowonetoseethatthepostedclosing

priceforXYZCorporationfromtwoquartersagowaschanged

—justyesterday.Theresearcherandhismanager,orhisjour-

naleditor,facesthesameproblem:theregressionyoureport-

ed using historical data downloaded from Nasdaq.com does

notgiveyouthesameresultitgaveyoulastweek.Thebest

andtheworstcharacteristicoftoday’severchangingInternet

is that informationonwhich investorsrely isever-changing.

And those data are subject to such rapid change without

notice, caution, or footnote. Even the most professionally

operatedwebsite,Nasdaq.com,Finance.Yahoo.comorBloom-

berg.comisnoexceptiontothisrule.Iftheresearcherisusing

information from the Internet, he or she is compelled to

archiveitlessheviolatesthefirstruleofresearchandthatis

keepingyourdataforotherstocheckyourresults.

Theremainderofthepaperisorganizedasfollows:Thenext

sectionexplainstheregulatoryoriginoftheexampleinsider

filings database, Rule 144, which provides the source filings

for the Internetsnapshots.Thiswillbe followedbyanover-

viewoftheexperimentalexperiencewithtakingsnapshotsof

theInternetandthemethodologyneededtotakesnapshots

of the Internet.Thepenultimatesectionanswers tensimple

butnovelresearchquestionsand, indoingso,describesthe

natureof thesedataandextremesizeof thisdatabase, fol-

lowedbyaconclusion.

Regulatory origin of the example databaseTheregulatoryoriginofthefilingrequirementsfortheexam-

pleinsidertradingdatabasedatesback70yearstolegislation

passedduringtheGreatDepression.ThefirstoftheActswas

passed on May 27, 1933 and is known as the Securities Act

of1933(’33Act)orthe‘truthinsecurities’law.Followingon

theheelsofthe’33Act,theSecuritiesExchangeActof1934

(’34Act),passedonJune6,1934.Forgoodorforbad,nothing

sotransformedtheU.S.financialmarketsasthe’33and’34

Acts. The ’34 Act established the Securities and Exchange

Commission (SEC) and under Section 16 required directors,

officers,andprincipalstockholderstoreporttheirownership

andchangesinownershiponSECForms3,4,and5.Forrea-

sons that are not completely clear, beyond these insiders,

the’33Actrequiresallthosewhoacquirestockwhileafirmis

privatetobeextremelyrestrictedorevenprohibitedintheir

ability to sell their unregistered stock via the public stock

marketaftertheIPO5.Theregulationthuspreventsthemar-

ketfromefficientlypricingtheliquidity.Thislimitationstands

in addition to other restrictions on whether the seller has

material non-public insider information or not. In fact, this

restriction binds all persons holding unregistered stock,

whethertheyareinsidersornot.Thiscanbeachillingrestric-

tion on angel investors, and other early investors, thereby

reducingtheinnovationstheyfund.

SECRule144providesasanctionedprocedure,describedasa

‘safeharbor’,foranyonesellingstockafteranIPOwhichwas

acquiredbeforethefirmwaspublic.Partofthesafeharbor

1362 Writinginthecontextofthefutureofthemuch-lessregulatedhedgefundindus-

try,seeScholes(2004)intheJournalofFinancialTransformation.

3 ReadersoftheJournalofFinancialTransformationknowthatneitherthetopicof

theInternet’simpactonfinancialmarkettransformationnorthestudyoffinancial

liquidityarenewtothisJournalasthefirstandsecondissuesofthisJournalwere

focusedonthosetwotopics.SeearchivesoftheJournalofFinancial

Transformation,IssuesNo.1andNo.2.


1374 CRSPstandsfortheCenterforResearchinSecurityPricesattheUniversityof

ChicagoGraduateSchoolofBusinessandisthesameacronymusedtodescribe

theirdatabasesofhistoricalstockandbondprices.Compustatreferstoasetof

databases,thebestknownofwhichcontainquarterlyaccountingdata,managed

bytheStandard&Poor’sdivisionoftheMcGraw-HillCompanies.

5 WhiletheSECnearlyalwayspreferentiallyseparatesandprotectsthewidowsand

orphansfromtheswindlersandthieves,withRule144theyaretreatedequally.

Thepaucityofwidelyavailableresearch-orienteddatabases

ofinsidertransactionslimitsacademically-rigorousinvestiga-

tions of insider trading, entrepreneurial liquidity, corporate

governance,and related topics.The result is that toooften

publicpolicydecisionsconcerningthetransformationofour

financialmarketsaremade in theabsenceof scientificevi-

dence.Onesuchexamplemightbetheone-size-fits-allnature

ofthenewinsiderreporting,thenewprohibitiononloansto

management,andthenewCEO/CFOcertificationregulations

stemming from Sarbanes-Oxley Act of 2002 (Sarbanes-

Oxley). In Sarbanes-Oxley, the requirements thrust on the

board of directors at giants General Electric and General

Motorsarethesameasthoseburdeningtheentrepreneur’s

teamatthesmallestIPO.Withitsshareofunintendedconse-

quences,thefullcostoftheenormousnewlegislativeinitia-

tiveinSarbanes-Oxleywillnotbeknownforyears.Andata

minimum,itislikelythattheeconomicrecoverywasdelayed

bythisact.

YettheverysamescandalsatArthurAndersen,Enron,Tyco,

WorldCom, and others which motivated Sarbanes-Oxley,

reminduswhycriticalanalysisofpolicyquestions involving

suchmattersremainsimportant.Withoutthoughtfulanalysis,

basedon largeandwidely-availabledatabases, asa society,

weriskdamagingoursmallestandmostentrepreneurialfirms

whiletryingtobettercontroltheactionsofafewofourlarg-

estcorporations.Thistypeofoverreactionintheabsenceof

extensivedataisequivalent,inaneconomicsense,tothrow-

ingthebabyoutwiththebathwater.Regulatorylimitsonthe

rateatwhichentrepreneurs,venturecapital investors,angel

investors,andotherinsiderscanobtainliquidityviathefinan-

cialmarketsshouldbeofsignificantandcontinuinginterestto

policymakersand investors.However,thesedataaresimply

lacking. Less costly and timelier databases are needed on

whichbetter capitalmarkets legislationand regulationscan

bebased.

Recent theoretical contributions in Kahl, Liu and Longstaff

(2003)indicatecontinuinginterestintheeffectsofilliquidity

infinancialmarketsandtheirextremecoststoentrepreneurs.

Prof. Scholes, writing previously in this Journal, reminds us

thatwhenfinancialmarketsareallowedtofunction,

‘Liquidityisthepriceofimmediacy,thecostofconvertingan

assetintocashinashortperiodoftime.Thepriceofliquidity

does not remain constant. It increases as investors become

pessimisticabouttheeconomicoutlookorattimesofcrisis.2‘

Whenregulationsprohibitsalesthenthefullcostsofilliquidity

areborneby theentrepreneur therebyreducing innovation,

economicgrowth,and itsbenefitstothepublic.Yettodate,

the time series and cross sectional databases which would

allowthoughtfulanalysisofthisimportantarearemainallbut

nonexistent. While there are a few commercially available

insider trading databases, these were not built with the

researcher in mind, but rather they were built primarily for

stocktraders.Thispaperexaminesanewdatabaseandanew

methodologyforrapidlybuildingotherdatabases.

Thenewexampledatabase,itself,isjustthat—anexample.It

isanexampleofaverylargedatabase,onewithinexcessof

one million data elements and one which can be built with

modest investmentof timeand labor.Wewill investigatesix

research questions that can be answered by this example

database.Whilethedatabaseisonlyintendedasanexample,

itmaybeofgreatinteresttoboththeacademicandtheprac-

titioner seeking sounder investigations into insider trading,

entrepreneurial liquidity, and corporate governance. The

methodologyusedtobuildtheexampledatabasewillleadto

othercompletelyunrelateddatabases.Thelowcostofthese

databases will allow researchers to investigate public policy

issues that previously saw rapid action without reasoned

analysis.Thismethodologyyieldsan importantnewclassof

databases which can be termed ‘snapshots of the Internet.’

Thesedatabasesmayhelpfillcriticalgapsinourknowledge

bases and improve the quality of national and international

economicpolicydecisions.3

Unrelated to the specific case of improved investigation of

insider trading, corporate governance, and entrepreneurial


the securities for more than two years, you are free to sell

without any Rule 144 restriction. Current work found in

BristowandOsborne(2002),utilizesthedepthoftheexample

databaseandfurtherexaminestheregulatorylimitsfoundin

theRule144safeharborandfindsevidencethatthesemaybe

furtherrelaxed.

Experimental experience with very large downloadsAswithmostoriginalresearch,theexperimentalexperience

acquiredbytakingsnapshotsoftheInternetwashumblingto

say thevery least.Workbegan inOctoberof2001with the

foolhardybelief thatbyyear-end2001,note2001, the team

would be developing its first snapshots. The team had the

year-endpartcorrectbutgottheyearpartwrongas itwas

year-end2002beforeaconsistentstringofsuccessfulsnap-

shotswerebeingtakenonaroutinebasis.Afterbeinghum-

bled so frequently and so completely, the team has yet to

regain the hubris it had when the project was undertaken.

However, itwouldbeprudentlysafetosaythatbyfollowing

thesemethodsnewdatabasescontainingmillionsofelements

ofdatacannowbebuiltinamatterofman-daystoweeks.The

samesizedatabaseswouldrequireman-yearstodecadesby

previousmethods.These innovativemethodsaccount forat

leasta1,000-foldincreaseinproductivity.

Theoriginalsubjectofthesnapshotwastheverypopularsite

Yahoo!Finance,andtherewassomesuccessingettingasnap-

shotofthatsitecompletedin2001.However,afterverycare-

ful comparison of snapshots taken a few days apart, it was

determinedthatthe insiderdataonthissite issummarized;

thesearenottheoriginalfilings.Thetradedates,whichwere

quotedonthissite,arethelatestdatesandmayconcealsev-

eralearliertradingdates.Neitherfactisobviousfromthesite

orthenoteswhichexplainthesiteandwasonlydetermined

afterseveralweeksoflaboriouslyhandcheckingthesedata.

The combining of shares data and price data from multiple

formsandusingonedatetorepresentastringofdatesare

unacceptable features foradatabase intended for scholarly

financeresearchincludingtheconstructionofpeer-reviewed

event studies. For these reasons, Yahoo! Finance was aban-

doned for the NASDAQ Stock Market website, and it is on

Nasdaq.comthattheteamhasfocusedtheireffortssincethe

springof2002.

During the testwecaptured insider trading records totaling

1,092,929 Form 4s and Form 144s containing approximately

tenmilliondataelementsinthefirstsession.Eachsnapshot

capturesatmost20insidertradingrecords.Therefore,there

areover50,000snapshotspersession.Snapshotsaretaken

sequentiallysoastohaveessentiallyzeroimpactonNasdaq.

com;minimizingtheimpactmeansthatthesesnapshot-taking

sessionsrequiredaystorun.Eachindividualsnapshothasa

dateandtimestamprecordedtothesecond.Thefirstcom-

pletesetofsnapshotsofinsidertradingdataonNasdaq.com

wascompletedonJune28,2002andittook4.47(24-hour)

days to complete. That is 107.28 hours, 6,436.8 minutes or

386,208secondscapturingroughlythreetradingrecordsper

secondonanultrafastInternetconnectionavailableatUCLA.

IfoneweretotranscribefromtheInternettoanexcelsheet

this same information by hand at the blistering rate of one

record per minute (which the authors cannot do), the first

downloadwouldhaverequired2.08man-yearstocomplete.

Nearly 13,000 securities were examined beginning on June

28,2002,and7,825(60.3%)ofthemhadcollectivelyoverone

million insider transactions filedonanSECForm4orForm

144. The snapshots are begun at 9:01 p.m. EST after the

NASDAQStockMarketcloses11onthelastbusinessdayofa

givenmonth.Essentially,everysecuritylistedonNasdaq.com

isexaminedtodetermineiftherearefilingsfortheirinsiders.

Thesizeofthedownloadwasreduced29.2%overthefirstsix

monthsprimarilyasimprovedparsingtechniquesweredevel-

oped,butdownloadtimeseemstoalsovarybynetworktraf-

fic.Thegoalistoretainonlytheinsidertradingdataandnot

otherhypertextmark-uplanguage(HTML)characters.Agreat

dealofeffortwasinvestedinfindingthemostefficientways

todecodetheHTMLtables.

138

6 RegisteredstockisstockregisteredwiththeSECundertheSecuritiesActof1933.

Unregisteredstockisalsocalledletterstockandlegendstockreferringtothelet-

tertotheSECrequiredtosellthestockandtheprohibitinglegendcarriedonthe

stockcertificatewarningthetransferagent,andhencethepurchaser,thatthis

stockhasnotbeenregisteredwiththeSEC.Inordertosellstockinthepublic

equitymarkets,onemusteitherregisterthestockwiththeSEC,averycostlypro-

cessforanindividual,oravailoneselfofoneoftheexemptionstotherequired

registrationprocess.Rule144isonesuchexemption.

7 FromTheSecuritiesLawyer’sDeskbook,seehttp://www.law.uc.edu/CCL/33ActRls/

rule144.HTML.

8 Thereisnoknownestimateofthedisruptivequantity.

9 Assumingtheinvestorholdsadiversifiedportfolio.

10 Nineteenventurecapitalists(VCs),whichatthetimewasanotablefractionof

thewholeVCindustry,visitedOsborneinearly1977whenhewasattheSECand

complainedaboutthepercentagelimit.VCsoftenliquidatetheirpositionsunder

Rule144.


requirementsare the filingofanSECForm 144.Stocksales

requiringForm144filingarereferredtoasletterstock,legend

stock,orsimplyunregisteredstock.Mostcommonly,aseller

willhaveacquiredunregistered6stockasstockpurchasesand

stock grants made during the early years of a corporation

before the IPO. It is theelectronicavailabilityofForms3,4

and5,butprimarilyForm4andForm144atNasdaq.comon

whichtheexampledatabaseofinsidertransactionscouldbe

built.

Settingtheproperparametervaluesofthesafeharborprovi-

sionsisoneuseoftheexampledatabaseandisthesubjectof

BristowandOsborne(2002).Intheabsenceoflargeamounts

ofsuchdata,itisnotclearonwhatbasistheSECfirstsetthe

safe harbor standards. The SEC first adopted Rule 144 in

Januaryof1972.Thisactioncodifiedstaffpracticesthathad

allowedearlyinvestorsofanissuertosellrestrictedstockvia

an exemption to the ’33 Act. The staff practices where the

sellercouldnotbeanunderwriter, thesalescanoccuronly

afterareasonableholdingperiod,and‘insuchlimitedquanti-

tiesandinsuchamannerasnottodisruptthetradingmar-

kets.’7 As one might expect, there has been much debate

concerning the exact meaning of reasonable holding period

andnon-disruptivequantities.Whilehealthy,thatdebatehas

occurredlargelywithoutthebenefitofextensivedata.Despite

theimportanceofinvestorliquiditytosoundeconomicpolicy

therehasbeenscantscientificinquiry.Atpresent,duringthe

reasonableholdingperiodtheliquidityrateiszero.Investors

cannotsellasingleshare.Aftersuchperiod,thenon-disrup-

tive quantity is maximum allowable liquidity rate such that

thereisnotanydeclineinpricecoincidentwiththeentrepre-

neur’ssale.8

Thissomewhatobscureregulationnegativelyimpactsallofus

intwomanners.Firstly,itreduceseconomicgrowthbyslowing

downtheprocessofrecyclingofcapitalfromsuccessfulentre-

preneurs’ first ventures to their follow-on investments, and

secondlyitharmsanypotentialfutureinvestor9byprevent-

ingthemfrombuyingstockatthetemporarilylowerpricesat

which the entrepreneur is willing to sell. The entrepreneur

maybewillingtopayProfessorScholes’priceforimmediacy

but theSECprohibits the transaction.Whileharmingnearly

everyone in theeconomy, theregulationprotectsbuta few.

The fewareasmallminorityofprior investorswith insider-

coincidentsales.Itisimportanttonotethatthebuyandhold

ownersneitherbenefitnorsuffer.Theonlyshareholderswho

benefitarethoseextremefewwhohappentobesellingatthe

samemomentastheentrepreneur.Andthisextrememinority

ofshareholdersisalsoonlyprotectedfromsellingthatfrac-

tionoftheirportfoliowhichhappenstobesoldattheslightly

lowerprices thatmightoccurduringentrepreneur’sdesired

coincidentselling.Furthermore,thebenefittopotentialcoin-

cidentsellersis,ofcourse,exactlyequaltothedetrimentto

potentialcoincidentbuyers.Whycoincidentsellersaregiven

preferencebytheSECovercoincidentbuyers isnotknown.

Thecurrentstandardsforthenon-disruptivequantityfollow

immediately.

SinceJanuary1972,therehavebeenthreerevisionstoRule

144.Osborne(1982)ledinparttotheserevisions.Eachsubse-

quent revision has decreased restrictions on entrepreneurs

and other early investors.10 The revisions made it easier to

sellunregisteredstock.Asummaryofthecurrentfiveprovi-

sionsare:

■ Theholdingperiodforthesecuritiesisoneyearafterfully

payingforthesecurities.

■ Theissuermusthavecompliedwithallrequirementsof

the’33Act.

■ Thenumberofsharesyoumaysellinanythree-month

periodcannotexceedthegreaterof1%ofthesharesout-

standingortheaverageweeklyvolumeforthefourweeks

precedingthefilingoftheForm144.

■ Thesalesmustbehandledasroutinetransactions.Oddly,

therecanbenosolicitationofbuyordersorspecialcom-

missions.

■ AtthetimeyouplaceyourorderyoumustfileForm144,if

thesaleinvolvesmorethan500sharesorU.S.$10,000in

anythree-monthperiod.

Finally,ifyouarenotanaffiliateoftheissuerandhaveheld

13911 AccordingtotheNASDAQStockMarketGlossary,theNasdaqNationalMarket

operatesfrom9:30A.M.to4:00P.M.EST,withextendedtradinginSelectNetfrom

8:00A.M.to9:30A.M.ESTandfrom4:00P.M.and5:15P.M.EST.

12 OriginatedbyLarryWall,PERLisanopen-sourceInternet-friendlylanguagewhich

standsforthePracticalExtractionandReportLanguage,butitalsostandsforthe

PathologicallyEclecticRubbishListeraccordingtoWall’sOctober1999interview

inLinuxMagazine.


aNasdaq.comuser,whentheuservisitsMicrosoft’sReal-Time

SECFilingssummaryincludessome19,800characterswhen

viewedinMicrosoftWord.Withoutblankspaces,the20lines

of theSECFilingssummarytablerequire 189 linesofHTML

text, or nine lines of HTML are required for displaying each

lineoftabularinformation.

Ten research questions and short answers from the example databaseLet us examine four very different sets of two or three

research questions each. The following ten questions were

chosentodemonstratethebreathofquestionswhichcanbe

addressed. While overly simplified, these are novel research

questionsthatareansweredwiththeexampledatabase.They

helpdemonstratethegranularityofdatathatcanbegener-

ated.Intheinterestofbrevity,thedepthofeachareacannot

becoveredinthispaper13.

in the area of financial markets litigation research or

insider trading research, one might have the following

questions:

1.BasedonpublicinformationinMay2004,whichbroker-

agehouseduringcurrentyearproposedthemostRule

144salesforMr.WilliamH.Gates,III,theChairmanof

Microsoft?

Allen & Company handled seven Form 144 filings totaling

15,000,000sharesofMicrosoftthatwereproposedforsale.

Mr.Gatesusedatleastfouradditionalhousestoproposehis

Rule 144 transactions including Credit Suisse First Boston,

Goldman Sachs, Merrill Lynch and Morgan Stanley. And

accordingtoNasdaq.com,aslateasJune15,2004,oneofMr.

Gate’s Form 144 filings apparently did not properly list the

broker’sname.14Itislistedas‘unknown.’

2.Moreexactly,fromApril27toMay12,2004,whatwasthe

largesttransactioninvolvinganinsidersalebyMr.Gates

andwhatwastheamountofliquidityachieved?

OnMay11,2004,Mr.Gatessold3,000,000sharesforapprox-

imately U.S.$25.82 per share. This transaction yielded

U.S.$77.46 million in liquidity. The liquidity amount is an

estimate;thepricegivenonNasdaq.comisthelastpriceina

series of sales totally 3,000,000 shares. All of these sales

were most likely on that day and most likely sold by one

broker. However, this price may not have been the average

price.Oncertaindates,suchasMay12,Mr.Gateshasmore

thanoneForm4filed.Thisisperfectlycorrectastheremay

havebeenmorethanonebrokeragefirmsellingstockonthe

sameday.

in the area of entrepreneurial liquidity research or finan-

cial markets microstructure research, one might make the

inquiries:

3. Accordingtoinformationavailableshortlyafterthetech-

nologymarketcrashof2000,whatwasthetrendininves-

torliquidityviaRule144filingsinthefiveyearsleadingup

tothatcrash?

Ourresultsshowsthat leadinguptothetechnologymarket

crash of 2000, Form 144 filings grew 210% from 23,664 in

1996 to 73,242 original filings by 2000. In terms of shares

proposed,thetrendwasevenfaster,growing351%from1.082

billion shares to 4.885 billion shares over the same period.

Examiningthesesametrendsintermsofmean,median,and

modal shares proposed one can see that for the five year-

periodleadingupto2000thatthemodalForm144proposed

remainedat 10,000shares.This isalso true for the full ten

year period shown. Likewise, the median was virtually

unchanged from 10,000 shares in 1996 to 10,310 shares by

2000.However,themeangrew45.95%from45,700sharesin

1996to66,700sharesby2000.Wecanfurtherfindtheesti-

matedcashproceeds inthousandsofdollarsfromtheForm

144filingsasproposed.ThenumberofForm144filingswhich

couldbematchedtoaCRSPstockpricerangesfrom11,773in

1992toapeakof58,462in2000.Themean(median)dollar

amountproposedgrewfromU.S.$601,100(108,800) in 1996

toU.S.$2,672,800(273,500)by2000.

4.Specifically,howmanysharesofunregisteredstockwere



Thefactorsthatweremonitoredinouranalysisincluded:

■ The total number of securities downloaded for a partic-

ular month–Thesewereslightlygreaterthanthenumber

ofcompaniesastherearecasesofclassA&Bshares,two

securities,forthesamecompany—forexampleBerkshire

HathawayhastwoclassesbothofwhichtradeontheNYSE.

■ Form 4 filings–Notallcompaniesandnotallsecurities

haveForm4filings.

■ Download size–thesizeoftheentiredownloadincluding

inputfilessuchastickerlists.Theunitsonthesedataare

1,024bytes(kilobytes(KB)).Thesearetermedverylarge

downloadsasthefirstdownloadamountedto517.1mega-

bytesofinsidertradingdata.

■ Download time–theamountofruntimeindaysthatthe

downloadtooktocomplete.Wealsomonitoredperiods

whereacrashalteredthedownloadtime.OnJune30

2003,thetotalelapsedtimewas2.34days,andon

February28,2004,itwas2.39days.FortheJuly31,2002

downloadandtheOctober31,2002downloadthesnap-

shotwasacompletebust.InAugust2002,onlyMicrosoft

failedtodownload.

MicrosoftCorporationwasusedasareferencecheck.

Reflectingonthe24monthsofexperience,sometrendsare

noteworthy. Download size was reduced until February 28,

2004, from 517,144 KB to 396,274 KB, when new fields and

newformswerebeguntobecaptured.Thenumberofsecuri-

tiesfellbelow12,000inAprilandMayof2003.Nowthatfig-

ureexceedstheoriginalmaximumofjustover13,000securi-

ties.ThosesecuritieswithanyForm4sseemtobegenerally

trendinglowerhavinglostfrom7,825to7,005frominception

toMay29,2004.OnNasdaq.comboththelayoutofthedata

and theamountand typeofdatadisplayedhaschangedat

leastquarterlysincethisresearchbegan.Thechangingofthe

formatofdataisthemostcommonproblemencounteredand

accountsforthemajorfailurestodate.Mostrecently,Nasdaq.

comhaslimitedthehistoryonthosefirmswithlonghistories

ofForm4andForm 144 filingssuchasMicrosoft.Microsoft

experiencedasteadyriseinfilesizeuntilFebruary28,2004

when it was reduced by roughly 2/3rds. Now Nasdaq.com

maintainsroughlytwoyearsofForm4andForm144filings,

whereasprevioustoFebruary2004,amorecompletehistory

wasavailable.Thisisbutoneexampleofhowinformationon

theInternetisavailabletodayandisgonetomorrow.

methodology of taking snapshotsAPerl12programwaswritten toaccess the site inanauto-

matedfashionandtodownloaddata.Thesnapshotdownload

program uses Perl functions found in Basic Perl and a few

functionsfoundintheLibraryforWorld-WideWebaccessin

Perl (LWP library). As an overview, the download program

reads from an input list of stock ticker symbols found on

Nasdaq.comandontheOver-the-counterBulletinBoardsite

atOtcbb.com.OnNasdaq.com,itisnotpossibletofindinfor-

mation on any stock without first knowing (or guessing) its

tickersymbol.ThesetickersymbolsareuniquetoNasdaq.com

andsomedodifferfromthosefoundforthesamestockon,

forexample,theNYSE.ThePerlprogramthenaccessespre-

definedwebaddressesthatvaryonlybythesetickersymbols.

AlistofallknownNasdaq.comtickersymbolsisupdatedevery

monthasthebeginningstepintakingthesnapshot.

Oncethewebaddressisconstructedwithavalidtickersym-

bol, then functionswithin theLWP libraryareused to fetch

informationaboutthesecurityrepresentedbythetickersym-

bolfromNasdaq.com.Therelevantinformationisembedded

within the HTML source code of the web page and can be

viewedbyanyonewhowishestouseNASDAQ’swebsite.Even

thoughsavingtheHTMLcodeusedtodisplaycompanyinfor-

mationwithInternetbrowsermaybedesirable,duetospace

constraints,itisnotrealistic.IftheunparsedHTMLcodewas

allstored,asinglemonth’sdownloadwouldexceedtengiga-

bytes — an unmanageable quantity. A parsing program is

thereforeusedtodiscardtheHTMLcodeand leave just the

dataparticular to thecompany, the insider,and its security.

ThedataaredownloadedonceamonthandstoredonaUNIX

machine.

Forexample,theHTMLcodethatgenerateswhatisviewedby

14113 FormoreinformationpleaserefertoBristowandOsborne(2002).

14 TheauthorsarenottryingtopickonMr.Gatesbut,asasidenote,itissomehow

comfortingtoknowthatevenMr.Gates,withhissmallbutexpertarmyofaccoun-

tants,advisorsandlawyers,hastroublegettinghisgovernmentformsfilledout

correctlyandcompletely.Howwelltherestofusdo,withoutsuchanarmy,

remainsamystery.

15 NotethatU.K.forUnknownisitsownrelationshipcode.


Thispaperdescribesamethodologyforrapidlybuildingdata-

baseswhichcanbeusedtobetteraddressimportantpublic

policy questions concerning, among other things, the rapid

transformationofthefinancialmarkets.Thispaperalsopro-

vides an example database of insider transactions to show

thevolumeand thenatureof information thatcanbecon-

verted froman intractable formon the Internet toauseful

researchtool.Byansweringtenwide-rangingresearchques-

tions, the usefulness and innovativeness of the example

database is shown. The speed at which snapshots of the

Internetcannowbetakendocumentsagreaterthan1,000-

fold improvement in productivity of building research data-

basesoverpriormethods.

Therearemanypublicbenefitstothecreationofnew,large

databases not the least of which concerns improved public

policy research and improved understanding of the capital

markets. However, the process archiving of some types of

information from the Internet is fraught with public policy

and legal issues of its own.16 The builder of any Internet-

sourceddatabase isencouragedtomakecertainthatheor

shehaspermissionoranexemptingrighttousethosedata.

Asasociety,weneedtobalancethesecosts.Thecostofnot

building and using these databases is the continued poor

understandingoftheenormoustransformationoccurringin

ourfinancialmarkets,specifically,andoureconomies,gener-

ally.TherecentSarbanes-Oxleylegislationmaybeacasein

pointofpublicpolicyrapidlymovingwithoutthebenefitofa

anempiricalroadmap. Infairnesstothedrafters, twoyears

after the legislation, databases designed to better under-

standtheirdecisionshaveyettocatchupwithourmercurial

policymakers.

References• Bristow,D.andA.Osborne,2002,“Rule144twentyyearslater:Constraintson

entrepreneursandtheeconomy,”UCLAWorkingPaper,1-40

• Charlesworth,A.,2003,“LegalissuesrelatingtothearchivingofInternetresources

intheUK,EU,USAandAustralia,”AstudyundertakenfortheJISCandthe

WellcomeTrust,UniversityofBristol,CentreforITandLaw

• Kahl,M.,J.Liu,andF.Longstaff,2003,“Papermillionaires:Howvaluableisstockto

astockholderwhoisrestrictedfromsellingit?”,JournalofFinancialEconomics,

67:3,385-410

• Osborne,A.,1982,“Rule144volumelimitationsandthesaleofrestrictedstockin

theover-the-countermarket,”JournalofFinance,37:3,505-517

• Scholes,M.,2004,“Thefutureofhedgefunds,”JournalofFinancial

Transformation,TheNobelLaureateView,IssueNo.9,8-10



proposedtobesoldin2000versus1999?

In1999,2.399billionshareswereproposedforsaleonForm

144s.Thefollowingyear,4.885billionshareswereproposed

for an increase of 2.486 billion shares in 2000 versus the

2.399billionproposedin1999.

5.Whatisthemean,median,andinter-quartilerangefor

liquidityin2000?

Themean(median)isU.S.$2,672,800(U.S.$273,500).Andthe

inter-quartile range is U.S.$63,800 (25%) to U.S.$1,047,200

(75%).

in the area of corporate governance research, or financial

markets legislative and regulatory research, one might

investigate these issues:

6.AccordingtoinformationavailablejustbeforeSarbanes-

Oxleywassignedintolaw,wereChairmenandCEOsthe

recipientsofmajorityofliquidityfromunregistered

shares?

Theanswerisno.WefindthatChairmenaccountfor2.22%of

filings and 3.50% of shares proposed. CEOs, given by Code

CE, have 1.77% of filings and 2.81% of shares proposed.

Togetherthesetwopositionstotal3.99%offilingsand6.31%

ofsharesproposed.

7. Precisely,howmuchlargerisfractionofthesalesof

unregisteredsharesaccountedforbytheboardofdirec-

torsandallofficersthansuchliquiditygainedbyoutsid-

ers?

Thetotalfractionaccountedforbyallmanagementincluding

directorsandallofficersis27.74%ofallsharesproposed.Ifwe

taketheCodesNforNone15(meaningnorelationshipother

than shareholder) and SH for Shareholder (meaning only a

shareholder),onefindstogetherthesetotal46.62%.Sothe

answeristhatthepresumptionthatdirectorsandofficersare

thegreatestbeneficiariesofthesalesofunregisteredshares

is, in fact, false. Outsiders, not even including affiliates and

largebeneficialowners,have18.88percentagepointsmoreor

68.1% more unregistered shares than do directors and offi-

cers.

in the area of financial markets microstructure research,

one might investigate the following issues:

8.Isthereanyseasonalpatterntoinsiderselling?

Analyzing the quarterly sinusoidal nature of over 600,000

insider filings filed with the SEC from January 1995 to

December2001,wefindthatinsidersellingpeaksareclearly

at3month intervalsbeginning inFebruaryandrepeating in

May,August,andNovember(months:2,5,8,11).

9.Isthereanyseasonalpatterntoinsiderbuys?

Insiderbuysarecertainlylessregularthaninsidersales.Also,

thelevelofbuysisroughly1/3thatofsalesaswell

10.Areresultsofseasonalanalysisalteredbyremovingvery

highandverylowstockprices?

Filteringoutliersfrombothinsiderbuysandinsidersalesdoes

notchangetheseasonalityofthepattern.Thelowerofeach

ofthepairsoflinesisaplotofinsidertransactionswithshare

pricesaboveU.S.$100andbelowU.S.$1.00removedfromthe

sample.

These ten research questions and the short answers that

addresseachofthemdemonstratethebreathoftheexample

database.Thisdatabaseandothers like itcanbebuilt fora

smallfractionofwhatsimilardatabasescostjustashorttime

ago.More important, is therecognitionof thepowerof this

tooltobuildotherusefuldatabasestoaddressawiderangeof

questionsbeyondjustcapitalmarketsliquidity.

conclusion

14316 ForathoughtfulanalysisoftheseissuesseeCharlesworth(2003).

Enterprise

The informational role of financial analysts: interpreting public disclosures

Donal ByardAssistant Professor, Zicklin School of Business,

Baruch College, City university of New York

Kenneth w. shawAssociate Professor, Joseph A. Silvoso Faculty

Fellow, College of Business, university of Missouri

Abstract

Whileitiswidelyacknowledgedthatfinancialanalystsplayan

increasingly vital informational role in capital markets, the

precisenatureofanalysts’ informationalrole incapitalmar-

ketsisnotwellunderstood.Commentatorsfrequentlyassume

that analysts obtain their most useful (private) information

directlyfromthemanagementofthefirmstheyfollow,rather

than processing information to independently develop their

own insights. Indeed, recently enacted regulatory changes,

suchasRegulationFD(FairDisclosure) intheUnitedStates,

have been motivated in large part by this belief. This paper

reviewsarecentstreamofacademicresearchthatsuggests

analternativeperspectiveontheinformationalroleoffinan-

cialanalystsincapitalmarkets.Thisresearchsuggeststhat,at

least in the case of earnings forecasts made by active ana-

lysts, financial analysts mainly perform a role of processing

and interpreting publicly available accounting disclosures,

rather than relying primarily on privileged communications

145

with corporate management. Thus, analysts’ primary infor-

mationalcontributiontocapitalmarketscomesmainlyfrom

theirownuniqueinterpretationsoffirms’publicdisclosures.

Thisgoessomewaytoexplainingwhytherecanbeademand

forlargenumbersofanalysts(e.g.,morethentwenty)tofol-

lowsomefirms.

1 ThisisthenumberofanalystswhoissuedaforecastforS&P500firmsduring

2002thatweretrackedbytheInstitutionalBrokers’EstimationSystem(I/B/E/S).

WethankI/B/E/SInternationalInc.forprovidingearningspershareforecastdata,

availablethroughtheInstitutionalBrokers’EstimateSystem.Thesedatahave

beenprovidedaspartofabroadacademicprogramtoencourageearningsexpec-

tationresearch.

2 Becauseoftheirlargerinvestorbase,thereisagreaterdemandforinformation

forlargerfirms.Asaresult,largerfirmstendtohavegreateranalystfollowing

thansmallerfirms.Thisisparticularlythecasefortechnology-basedstocks.

Becausetechnology-basedstocksareinherentlymorecomplex,withtheirheavy

relianceonintangibleassets,theyalsotendtobefollowedbyagreaternumberof

analysts.


(s), and consensus (ρ) — the extent to which analysts share

common information. These measures, discussed in more

detailintheactualarticle,areasfollows:

Consensus (1)

Precisionofcommoninfo. (2)

Precisionofprivateinfo. (3)

Assumingeachanalysthassomeinformationthatiscommon

toallanalystsandsomeprivate(unique)information,andthat

analystsusethisinformationtoforecastaccurately,thenthe

observable characteristics of their earnings forecasts will

reflecttheunderlyingpropertiesoftheirinformation.Inpar-

ticular,thedispersionamongtheindividualforecastsisdeter-

minedbytheprecisionofanalysts’privateinformation,while

the squared error in the mean forecast mainly reflects the

precisionoftheinformationcommonacrossallanalysts.

Figure 1 presents a simple stylized example of the BKLS

approach.Theoretically,theBKLSapproachholdsinexpecta-

tion,oronaverage.Asaresult,itcanbeusedtomeasurethe

average properties of analysts’ information across a large

sampleoffirmsorfirm-years.Becauseofthelikelymeasure-

menterror,itisnotrecommendedthattheBKLSapproachbe

usedtomeasurepropertiesofanalysts’informationforindi-

vidual firms at a single point in time. Nevertheless, this

exampleofasinglefirmhelpsillustratethelinkagebetween

the properties of analysts’ underlying information and the

observablepropertiesoftheirforecasts,whichisthekeyintu-

itionoftheBKLSapproach.

Intheexample,threeanalystsareforecastingupcomingearn-

ings.TheirindividualearningsforecastsofU.S$2.75,U.S$2.95,

andU.S.$3.75persharerespectivelyyieldameanforecastof

U.S.$3.15 and dispersion of U.S.$0.28. If actual earnings are

U.S.$3.54, then the dispersion in analysts’ forecasts is 0.28

andthesquarederrorinthemeanforecastis0.1521.Plugging

these observables, along with the number of analysts fore-

casting(three),intoequations(1)through(3)yieldsempirical

estimatesoftheextenttowhichanalysts’earningsforecasts

share common beliefs (ρ = 0.174), and the precision of the

common(h=0.512)andprivate(s=2.44)informationinana-

lysts’forecasts.

Continuingwiththisexample,iftheprecisionofanalystspri-

vateinformation,s,washigher,says=5.44,thiswouldresult

intwoeffects.Firstly,onaverage,theerrorseachofthethree

analystsmakewouldbelower.Secondly,sinceeachindividual

forecast contains relatively more private information (and

consensus decreases to 0.086), the dispersion among the

forecasts(D)willincreaserelativetothesquarederrorinthe

mean forecast (SE). The BKLS approach is built upon this

intuition:thatthepropertiesofanalysts’underlyinginforma-

tion determine the observable statistical properties of the

forecaststheymake.

Recent empirical evidence using the measures of BKlsRecentstudieshaveusedthemeasures inBKLStoexamine

thechangesinanalysts’consensusaroundearningsannounce-

ments, how analysts’ information is related to accounting

qualityandthequalityofmanagements’privatecommunica-

tions with analysts, and how the level of consensus among

analystsdiffersforlow-technologyandhigh-technologyfirms.

Wesummarizetheresultsofthesestudiesnext.

changes in consensus around earnings announcementsBarron, Byard, and Kim (2002), hereafter BBK, examine

changesintheBKLSconsensusmeasure(ρ)aroundearnings

announcements. For a sample of 990 firm-years over the

146

h=SE-

DN

( )

SE- +DDN

( )

ρ =SE-

DN

( )

SE- +DDN

( )

[ ]2

s=SE- +D

DN

( )[ ]2D


147

Theinformationalroleofsell-sidefinancialanalystsincapital

markets is of interest to institutional investors, brokerage

firms, regulators, academics, and the investing public.

Commentators frequently express the view that analysts

obtainmost,ifnotall,oftheirmostuseful(private)informa-

tionfromthemanagementsofthefirmstheyfollow;thatis,

analystssimplyactasconduitsformanagementtotransmit

information to the market. Regulatory changes in recent

years, such as Regulation FD (Fair Disclosure) in the United

States,havebeenmotivatedinlargepartbythisbelief.While

certainevidencedoespointtomanagementfeedinginforma-

tiontoanalysts,inparticularthegrowingprevalenceinrecent

years of ‘the talk down’ to meetable/beatable forecasts

[Brown(2001),Matsumoto (2002)],a recentanddeveloping

streamofacademicresearchsuggeststhattheinformational

roleoffinancialanalystsissomewhatmorecomplex,andthat

muchofanalysts’private informationcomesfromtheirown

uniqueinterpretationsoffinancialdisclosures.

Thisrecentstreamofresearchalsohelpsshed lightonwhy

somefirmshavequite largeanalystfollowings.Largerfirms

tend to be followed by more analysts: in 2002 the median

numberofanalystsfollowingS&P500firmswastwenty-two,

while fifty firmswere followedby thirty-sevenormoreana-

lysts,andonefirm(NortelNetworks)hadafollowingofsixty-

four analysts1. Academic research on the determinants of

analystfollowingconfirmsthatlargerfirms,especiallythose

that are technology-based, tend to have quite large analyst

followings2. This begs an important question: what is the

informational role of all these analysts? More specifically,

whatistheinformationalcontributionof,say,thethirty-fifth

analystfollowingafirm?Inparticular,ifanalystsactsimplyas

conduits for private information from management, why is

thereaneedforsomanyanalyststofollowsomefirms?

Inthispaperwereviewsomerecentempiricalresearchthat

providesinsightsintofinancialanalysts’informationalrolein

capitalmarkets.Insum,thisresearchsuggeststhatanimpor-

tantinformationalroleoffinancialanalystsisthatofprocess-

ing publicly available accounting disclosures. In effect, ana-

lystsundertaketheirownuniqueprocessingoffirms’account-

ingdisclosuresand,intheirearningsforecasts,offertheirown

individual interpretations of the information contained in

thesedisclosures.Thisinterpretiverolesuggestsareasonwhy

therecanbeademandforthirtyormoreanalyststofollow

particular firms, especially large high-technology firms, and

why analysts’ revision activity is concentrated in periods

immediately after earnings announcements [Stickel (1989)].

This evidence also suggests that, in forming their earnings

forecasts,analystsrelymoreheavilyonhighqualityaccount-

ing disclosures and less on privileged communications with

management.

BackgroundRecent empirical research has made use of measures sug-

gested by the analytical model of Barron, Kim, Lim, and

Stevens (1998), hereafter BKLS. In essence, the model uses

observable features of the analyst forecasting environment,

includingthelevelofdispersioninanalysts’forecasts(D),the

squarederrorinthemeananalystforecast(SE),andthenum-

berofanalystsforecasting(N),todevelopempiricalmeasures

ofpropertiesofanalystsunderlyinginformation—specifically,

theprecisionofanalysts’common(h)andprivateinformation

unobservables

h=Precisionofpublicinfo

f= Precisionofprivateinfo

ρ=Consensus

unobservables

h=Precisionofpublicinfo.

f= Precisionofprivateinfo.

ρ=Consensus

Forecastspriortoannouncement

Earningsannouncement

Analyst13.75

Analyst22.75

Analyst32.95

mean forecast

F=3.15

Actual earnings

A=3.54

observables

Dispersion=0.28

SE=0.1521

N=3

Figure1:ExampleoftheBarron,Kim,Lim,andStevens(BKLS)analyticalmodel

3 Theauthorsalsoshowthatthelevelsofbothcommonandprivateinformation

containedinanalysts’forecastsincreasearoundearningsannouncements.


theprecisionofanalysts’privateinformationandthequality

of firms’ investor relations’ activities. Byard and Shaw thus

concludethatmuchofanalysts’(unique)privateinformation

derivesfromtheirprocessingofaccountingdisclosures,rath-

er than from reliance on privileged communications with

management.

consensus and high-technology firmsOnecouldalsoexpecttoseeevidenceofthisrelativelygreat-

er interpretive role for analysts who follow high-technology

firms.Undergenerallyacceptedaccountingprinciples inthe

UnitedStates,expendituresonintangibleassets(e.g.research

anddevelopmentcosts)areimmediatelyexpensed.Thisren-

ders financial statements of high technology firms less reli-

able indicators of firm value and levels of future earnings.

Thus, because of their heavy reliance on intangible assets,

high-technology firms are likely to have accounting disclo-

suresthatareharderto interpret,generatingmoredemand

for analysts’ unique interpretations of disclosures for these

firms.

Recent empirical evidence suggests this is indeed the case.

Notonlydoagreaternumberofanalysts,onaverage,follow

high-technologyfirms[Barth,Kasznik,andMcNichols(2001)],

butthelevelofconsensusamongtheseanalystsisalsolower

[Barron, Byard, Kile, and Riedl (2002)]. Barron et al. (2002)

use data for firms’ advertising expenditures, research and

development investments, and level of capitalized balance

sheet intangibles to construct measurers of the relative

degree to which firms’ are comprised of intangible assets.

Theythenmatchthisdatawithearningsforecastsforthese

firms over the period 1986 to 1998 and assess the relation

betweenfirms’relianceonintangibleassetsanddifferencesin

analysts’consensus.Theresultsindicatealowerlevelofcon-

sensusamonganalystsforecastingforhigh-technologyfirms

that rely more on intangible assets, indicating that in their

earnings forecasts for these firms there is relatively more

emphasisonanalysts’owninterpretations.

conclusionBarron,Kim,Lim,andStevens(1998)suggestasimplemodel

offinancialanalysts’informationenvironmentcanbeusedto

infer properties of analysts’ underlying information from

observable characteristics of their forecasts. Several recent

empirical studies have employed this approach to examine

howanalysts’informationchangesaroundearningsannounce-

ments,howthisinformationisaffectedbythequalityoffirms’

accountingandnon-accountingdisclosures,andhowitvaries

acrosslowandhigh-technologyfirms.

Theresultsofthesestudiesindicatethatthecommonalityof

analysts’information—thedegreetowhichallanalystsfollow-

ingafirmsharethesameinformationaboutupcomingearn-

ings—decreasesafterearningsannouncements. Inaddition,

theprecision(quality)ofindividualanalysts’privateinforma-

tionincreasesafterearningsannouncementsandishigherfor

firmswithhigherqualityaccountingdisclosures. Incontrast,

the perceived quality of analysts’ private communications

withmanagementhasnoimpactonthelevelofprivateinfor-

mationcontainedinanalysts’forecasts.Finally,whenaccount-

ing reports are relatively less informative about firm value

andfutureearnings,namelyinthecaseoffirmswithconsider-

able intangible assets, the level of consensus in analysts’

forecastsisalsolower.


Figure2:ChangesinBKLSconsensusaroundpriorearningsannouncements

1

0.9

0.8

0.7

0.6

0.5

0.4Priorannualearningsannouncements

Qtr1earningsannouncements



Annualearningsannounce-ments

BK

LS

co

nse

nsu

s


period 1986 to 1997, BBK examine changes in consensus

among analysts forecasting earnings over approximately a

fourteen-month window prior to annual earnings announce-

ments. Within this fourteen-month window BBK focus on

forecasts made immediately before and after earnings

announcements—namely,thepriorannualearningsannounce-

ment and the three interim (quarterly) earnings announce-

ments in the current fiscal year (Figure 2). For example, in

analyzing the 14-month period before a fiscal-year 1997

annualearningsannouncement,BBKanalyzeactivityaround

the 1996annualearningsannouncement,and the first, sec-

ond, and third quarterly announcements in 1997. The fore-

castsmadebeforeandafterthesepriorearningsannounce-

ments are made by the same individual analysts. BBK thus

focus on updates, occurring soon after earnings announce-

ments, of forecasts made just before earnings announce-

ments.

Figure 2, which plots the median consensus across the

approximately fourteen-month window prior to the annual

earningsannouncement,showsthemainfindinginBBK:con-

sensusactuallydropsaroundpriorearningsannouncements.

Consensusstartsoutrelativelyhigh,closetoitslimitofone,

atthebeginningofthe14-monthperiod,andthendecreases

astheupcomingannualearningsannouncementapproaches.

Significantly,relativelylargedecreasesinconsensusarecon-

centrated around prior earnings announcement dates3.

These decreases in consensus around earnings announce-

mentsaresignificantlylargerthaninnon-earningsannounce-

mentperiodsandarelargerforfirmswithlargeranalystfol-

lowing, and when revision activity is relatively high among

analysts following a firm. In sum, the evidence in Barron,

Byard, and Kim (2002), in particular the concentration of

decreases in consensus around prior earnings announce-

ments shown in Figure 2, indicates that much of analysts’

private information about annual earnings is triggered by

prior earnings announcements. This indicates that much of

analysts’privateinformationcomesfromtheirprocessingof

publicaccountingdisclosures.

Accounting disclosure quality and analysts’ com-mon and private informationByard and Shaw (2003) use survey data on the quality of

firms’accountingdisclosuresandinvestorrelationsactivities

toexaminetherelationbetweeninformationqualityandthe

precisionofanalysts’commonandprivateinformation(BKLS

measureshands).Thedataarebaseduponasetofannual

surveysofdisclosurepracticesoflargeU.S.firmsconducted

bytheAssociationforInvestmentManagementandResearch’s

(AIMR) Corporate Information Committee over the period

1985 to 1995. In these surveys, independent AIMR industry

committees, comprised of analysts that follow firms within

specific industries,separatelyassessandrankthequalityof

firms’ annual report disclosures, quarterly report and other

disclosures,andinvestorrelations’activities.

The AIMR annual and quarterly report scores proxy for the

qualityofpublic,primarilyaccounting,information,whilethe

investor relations score proxies primarily for the quality of

analysts’privatecommunications (whether individuallyor in

groups) with managements of the firms they follow.

Importantly, the AIMR data allow Byard and Shaw to distin-

guishthequalityofpubliclyavailableaccountinginformation

fromthequalityofprivatecommunicationsbetweenmanage-

ment and analysts, and thus provide evidence on analysts’

roleaseitherprocessorsofinformationorconduitsofinfor-

mationfrommanagement.

ByardandShawregresstheBKLSmeasuresoftheprecision

ofanalysts’commonandprivateinformation(hands)onthe

threeAIMRmeasuresoffirms’informationquality,withcon-

trol variables for firm size and the size of the most recent

earningssurprise.Theyfindthatfirstly,theprecisionofana-

lysts’privateinformationincreasesasfirms’overalldisclosure

quality increases. Secondly, this increased precision of ana-

lysts’privateinformationemantesprimarilyfromthequality

of firms’accountingdisclosures—thequalityof information

contained in their annual and quarterly reports and other

disclosures. Finally, they find no direct relationship between

149

Together,thisevidencepointstoanalystsplayingakeyroleas

interpretersofpubliclyavailableinformation,ratherthansim-

plyactingasconduitsofinformationfrommanagement.This

evidence indicates that much of individual analysts’ private

information is derived from their processing of accounting

disclosures.Thus,akey roleofanalystsappears tobe their

processingof firms’accountingdisclosures toproduce their

ownuniqueinterpretations.Thisroleofanalysts,inturn,helps

explainwhy there isdemand for somany (i.e., greater then

twenty)analyststofollowsomefirmsandwhymuchofana-

lysts’ revision activity is concentrated soon after earnings

announcements(Stickel1989).

References• Barron,O.,D.Byard,C.Kile,andE.Riedl,2002,“High-technologyIntangiblesand

Analysts’Forecasts,”JournalofAccountingResearch,40:2,289-312

• Barron,O.,D.Byard,andO.Kim,2002,“ChangesinAnalysts’Informationaround

EarningsAnnouncements,”TheAccountingReview,77:4,821-846

• Barron,O.,O.Kim,S.Lim,andD.Stevens,1998,“Usinganalysts’forecaststo

measurepropertiesofanalysts’informationenvironment,”TheAccountingReview,

73:4,421-433

• Barth,M.,R.KasznikandM.McNichols,2001,“AnalystCoverageandIntangible

Assets,”JournalofAccountingResearch,39:1,1-34

• Brown,L.,2001,“ATemporalAnalysisofEarningsSurprises:ProfitsversusLosses,”

JournalofAccountingResearch,39:2,221-242

• Byard,D.,andK.Shaw,2003,“CorporateDisclosureQualityandPropertiesof

Analysts’InformationEnvironment,”JournalofAccounting,Auditing,andFinance,

19:3,355-378

• Matsumoto,D.,2002,“Management’sIncentivestoAvoidNegativeEarnings

Surprises,”TheAccountingReview,77:2,483-514

• Stickel,S.,1989,“Thetimingofandincentivesforannualearningsforecastsnear

interimearningsannouncements,”JournalofAccountingandEconomics,11:2,275-

292

151

Guidelines for authors

Inordertoaidourreadership,wehaveestablishedsomeguidelinestoensurethatpublishedpapersmeetthehigheststandardsofthoughtleadershipandpracticality.Thearticlesshould,therefore,meetthefollowingcriteria:

1. Doesthisarticlemakeasignificantcontributiontothisfieldofresearch?

2.Cantheideaspresentedinthearticlebeappliedtocurrentbusinessmodels?Ifnot,istherearoadmaponhowtogetthere.

3.Canyourassertionsbesupportedbyempiricaldata?4. Ismyarticlepurelyabstract?Ifso,doesitpictureaworldthatcan

existinthefuture?5. Canyourpropositionsbebackedbyasourceofauthority,preferably

yours?6.Wouldseniorexecutivesfindthispaperinteresting?

subjects of interestAllarticlesmustberelevantandinterestingtoseniorexecutivesoftheleadingfinancialservicesorganizations.Theyshouldassistinstrategyformulations.Thetopicsthatareofinteresttoourreadershipinclude:

• Impactofe-financeonfinancialmarkets&institutions• Marketing&branding• Organizationalbehavior&structure• Competitivelandscape• Operational&strategicissues• Capitalacquisition&allocation• Structuralreadjustment• Innovation&newsourcesofliquidity• Leadership• Financialregulations• Financialtechnology

manuscript submissions should be sent toShahinShojai,[email protected]

CapcoClementsHouse14-18GreshamStreetLondonEC2V7JETel:+44-20-73671321Fax:+44-20-73671001

manuscript guidelines

AllmanuscriptsubmissionsmustbeinEnglish.

Manuscriptsshouldnotbelongerthan5000wordseach.ThemaximumnumberofA4pagesallowedis10,includingallfootnotes,references,chartsandtables.

Allmanuscriptsshouldbesubmittedbye-maildirectlytotheeditor@capco.cominthePCversionofMicrosoftWord.TheyshouldalluseTimesNewRomanfont,andfontsize10.

Wheretablesorgraphsareusedinthemanuscript,therespectivedatashouldalsobeprovidedwithinaMicrosoftexcelspreadsheetformat.

Thefirstpagemustprovidethefullname(s),title(s),organizationalaffili-ationoftheauthor(s),andcontactdetailsoftheauthor(s).Contactdetailsshouldincludeaddress,phonenumber,faxnumber,ande-mailaddress.

Footnotesshouldbedouble-spacedandbekepttoaminimum.TheyshouldbenumberedconsecutivelythroughoutthetextwithsuperscriptArabicnumerals.

For monographsJensen,M.,CorporateControlandthePoliticsofFinance.JournalofAppliedCorporateFinance(1991),pp.13-33.

For booksCopeland,T.,T.Koller,andJ.Murrin.Valuation:MeasuringandManagingtheValueofCompanies.JohnWiley&Sons,NewYork,NewYork(1994).

For contributions to collective worksRitter,J.R.,1997,InitialPublicOfferings,inLogue,D.andJ.Seward,eds.,WarrenGorham&LamontHandbookofModernFinance,South-WesternCollegePublishing,Ohio.

For periodicalsGriffiths,W.,Judge,G.,1992,‘Testingandestimatinglocationvectorswhentheerrorcovariancematrixisunknown,’JournalofEconometrics54,121-138.

For unpublished materialGillan,S.,andL.Starks.RelationshipInvestingandShareholderActivismbyInstitutionalInvestors.WorkingPaper,UniversityofTexas(1995).

Guidelines for manuscript submissions


Theworldoffinancehasundergonetremendouschangeinrecentyears.Physicalbarriershavecomedownandorganizationsarefindingithardertomaintaincompetitiveadvantagewithintoday’strulyglobalmarket-place.Thisparadigmshifthasforcedmanagerstoidentifynewwaystomanagetheiroperationsandfinances.Themanagersoftomorrowwill,therefore,needcompletelydifferentskillsetstosucceed.

ItisinresponsetothisgrowingneedthatCapcoispleasedtopublishthe‘journaloffinancialtransformation.’Ajournaldedicatedtotheadvance-mentofleadingthinkinginthefieldofappliedfinance.

The journal,whichprovidesauniquelinkagebetweenscholarlyresearchandbusinessexperience,aimstobethemainsourceofthoughtleadershipinthisdisciplineforseniorexecutives,managementconsultants,academics,researchers,andstudents.Thisobjectivecanonlybeachievedthroughrelentlesspursuitofscholarlyintegrityandadvancement.Itisforthisreasonthatwehaveinvitedsomeoftheworld’smostrenownedexpertsfromacademiaandbusinesstojoinoureditorialboard.Itistheirresponsibilitytoensurethatwesucceedinestablishingatrulyindependentforumforleadingthinkinginthisnewdiscipline.

Youcanalsocontributetotheadvancementofthisfieldbysubmittingyourthoughtleadershiptothejournal.

Wehopethatyouwilljoinusonourjourneyofdiscoveryandhelpshapethefutureoffinance.

[email protected]

Request for papers — Deadline october 1st, 2004

Formoreinfo,seepage150

©2004TheCapitalMarketsCompany.VU:ShahinShojai,

PrinsBoudewijnlaan43,B-2650Antwerp

Allrightsreserved.Allproductnames,companynamesandregisteredtrademarksin

thisdocumentremainthepropertyoftheirrespectiveowners.

153

Design,production,andcoordination:Cypres—DanielBrandtandPieterVereertbrugghen

©2004TheCapitalMarketsCompany,N.V.

All rights reserved. This journal may not be duplicated in any way without the express

writtenconsentof thepublisherexcept in the formofbriefexcerptsorquotations for review

purposes.Makingcopiesofthisjournaloranyportionthereofforanypurposeotherthanyour

ownisaviolationofcopyrightlaw.

SEI.New Ways. New Answers.

1

2

3

4

Rising costs, heavier regulation, tougher clients in a tougher market – how can a Private Banking business cope?

Can we respond quickly enough to all these changes?

Should we be redefining our private client value proposition?

How do we decide what is best done in house, and what needs to be outsourced?

SEI knows about the competitive benefits of outsourcing in private banking institutions.

To get a copy of SEI’s special report ‘Outsourcing and the European Wealth Management Market’*, call Francis Jackson on + 44 (0)207 297 6308, or Email [email protected]

Hear what the industry is saying.

Four Key Questions for Wealth Managers

* Research amongst CEOs, CFOs and Senior Executives of wealth management institutions in 10 European Countries

SEI Investments (Europe) Ltd, 4th Floor, The Economist Building, 25 St James’s Street, London SW1A 1HAis regulated and authorised by the Financial Services Authority

www.capco.com

Antwerp T+3237401000

Bangalore T+91805270353

Frankfurt T+496997609000

London T+442073671000

NewYork T+12122848600

Paris T+33147553090

SanFrancisco T+14154450968

Singapore T+6563956999

Sunnyvale T+14085229900

Toronto T+4169234570

the journal of financial transformation #11

Documents

world of economic data

deutschebankgroupdavid

research data

hp servers

ubsprivatebankingbruno

capcojohn owen

capcobill irving

planetfinancerudi bogni