Volume 17, Issue 1 Winter 2006

Newsletter StaffUniversity of Notre Dame

Editors

Michael Coppedgecoppedge.1@nd.edu

Anthony M. Messinamessina.3@nd.edu

Assistant Editor

Lucas Gonzálezgonzalez.90@nd.edu

Book Review Editor

Naunihal Singhsingh.16@nd.edu

Editorial Board

Robert Dowd, C.S.C.

Andrew Gould

Frances Hagopian

Debra Javeline

Donald Kommers

Scott Mainwaring

A. James McAdams

Guillermo O’Donnell

Contact: Decio Hall, Box “D”University of Notre Dame, Notre Dame,Indiana 46556. Tel. 574-631-5681http://www.nd.edu/~apsacp

Letter from the PresidentSpace and Comparative Politics

Sidney TarrowCornell Universitysgt2@cornell.edu

what I mean by “abstract space.” Butwhen we ask why mountains shouldbe correlated with civil unrest, wehave to turn to “concrete space.” It isconcrete mechanisms – like the iso-lation of mountain residents from thecenters of power, their knowledge ofthe terrain, and the interpersonaltrust that grows up in small communi-ties – that enhance “the likelihoodthat mountain dwellers will be able toengage successfully in illegal or sub-versive activities like smuggling orguerrilla warfare” (Sewell, p. 55; alsosee Tilly 2000, 2003).

Consider the long survival of theWaldensian sect in the French Alpsabove the town of Embrun with whichCharles Tilly begins his Trust andRule (2005: pp. 1ff.). Their occupa-tion of “abstract space” high abovethe valley predicted the likelihoodthat they could escape the intrusionof the monarchy. And they did. Butonly by specifying the properties ofthe “concrete space” they inhabitedcan we understand how they sur-vived for as long as they did.

Adjacent disciplines have done a bet-ter job than we have in integratingspatial variables within their modelsand methods. Geographers like JohnAgnew (1987), Sallee Marston(2000), Doreen Massey (1994), andByron Miller (1994) have specifiedways in which space structures poli-tics. Sociologists like Herbert Gans(2002), Tom Gieryn (2000), Edward

A few years ago, a group of col-leagues and I put together a collec-tive book called Silence and Voice inthe Study of Contentious Politics(Aminzade et al. 2001).1 One of ournumber, Bill Sewell, found an emptysilence in the study of contentiouspolitics – the concept of space itself(Sewell 2001)! Oddly, the same lacu-na exists in comparative politics moregenerally.

Of course, our subdiscipline dealsimplicitly with space but for the mostpart, we have specified the spatialdimensions of politics as what Sewellcalled abstract space – “continuous,unbounded, unlimited extension inevery direction, regarded as void ofmatter or without reference to this” (p.53). But space also means concretespace – “a definite location of a par-ticular size and shape.” Used in thisway, Sewell continues, “space isdefined not by objective quantifiablecharacteristics,” but is “defined inrelation to human occupation, use, orgaze” (ibid.).

Here is an example: Jim Fearon andDavid Laitin map countries in whichcivil wars are likely to break out; suchcountries are more likely to be moun-tainous than more pacific places(Fearon and Laitin 2003:97). That is (Continued on page 2)

Volume 17, Issue 1 Winter 2006 Newsletter APSA - CP

The Organized Section in Comparative Politics of the American Political Science Association

LetterAPSA-CP Vol 17, No. 12

Section OfficersPresident

Sidney TarrowCornell Universitysgt2@cornell.edu

Vice President, President-Elect

Peter A. GourevitchUniversity of California, San Diegopgourevitch@ucsd.edu

Secretary-Treasurer

Karen RemmerDuke Universityremmer@duke.edu

2006 Program Chair

Deborah J. YasharPrinceton Universitydyashar@princeton.edu

Executive Council

Elizabeth PerryHarvard Universityeperry@gov.harvard.edu

Mathew ShugartUniversity of California, San Diegomshugart@ucsd.edu

Susan StokesYale University susan.stokes@yale.edu

Anna Grzymala-BusseUniversity of Michiganabusse@umich.edu

Soja (1989), and Dingxin Zhao (1999;2001) have invested space and placewith concrete meanings. Anthropo-logists and historians with an anthro-pological vision have taught us thatspace must be constructed into“place” (Boone 2002; Edelman 2001;Ragusa and Torre 2003; Wood 2001).

1976). It was not the size of thesetowns but the relations betweennewly-industrialized workers andpeasants that produced the higher-than-average vote for the Left. LikeFearon’s and Laitin’s mountains, sizewas the abstract spatial category thatcould only be understood in terms ofthe relations within concrete space.

Concrete space also helped DingxinZhao understand the dynamics theTienanmen Square rebellion. As iswell known, Chinese communist lead-ers had established 67 universities inthe Haidan district, isolated from thesurrounding neighborhoods (Zhao1998: 1502; I quote Sewell’s summa-ry from p. 72 of his article). A goodpart of the explosion of contention in1989 can be explained by traditionalsocial network theory (McAdam 1988;Osa 2003; Gould 1995). But it wasthe co-presence of students frommany universities in public space thatproduced the massing of studentgroups marching on TienanmenSquare (Zhao 1998: 1508-09). Forthat, Zhao turns to the brokerage roleof the “liaison men” who linked stu-dents from different campuses to oneanother (Zhao 1998: 1514-15; Sewell:74). It was these relations within “con-crete space” that made “abstractspace” work to produce a near-revo-lution.

Spatial Diffusion

A second process – diffusion – is animportant spatial factor in contentiouspolitics. By diffusion, I mean thespread of a form of contention, anissue, or a way of framing it from onesite to another.2 The diffusion ofsocial movements, revolutions, andother forms of contentious politics issimilar in many ways to the diffusionof innovations (Soule and Zylan 1997;Strang and Soule 1998). For exam-ple, in their examination of the Swingrebellion in southern England,Hobsbawm and Rudé (1975) found

(Continued from page 1)

“Adjacent disciplines have

done a better job than we

have in integrating spatial

variables within their models

and methods.”

When we look inside “concretespace,” we can identify numerousprocesses in which proximity mattersthat can enrich our subdiscipline’scontributions. From among many pos-sible ones, I choose to illustrate three:space and co-presence, spatial diffu-sion, and scale shift.

Spatial Co-Presence

As a Berkeley graduate student in the1960s, I went off to Italy trying tounderstand how the different structur-al conditions of North and Southimpacted the organizational, ideologi-cal, and the electoral behavior of theItalian Communist Party (PCI) and itsrelations with the southern peasantry.Serendipitously, I discovered that inSouthern Italy, both the big cities andthe villages voted for the Right whilecommunes of 30,000 to 100,000voted for the Left. Why? The proper-ties of concrete space inspired ahypothesis: the medium-sized citieswere large agrarian borghi with littleinfrastructure, in which small artisanalworkers existed alongside peasantworkers (Schneider and Schneider

3APSA-CP Vol 17, No. 1

that the incidents of Swing violencefollowed the main lines of road com-munication between London and thecoast. Rudé’s work on rural riots in18th century France similarly showedthat contentious episodes followedthe river valleys (Rudé 1981:24).

ing sessions for Georgian reformers.The Georgians, in turn, trained acadre of grassroots activists. Diffusionand brokerage combined to producethe spread of anti-election fraud cam-paigns in former state socialistregimes in Eastern Europe.

Scale Shift

The Internet’s importance is critical toa special form of diffusion – scaleshift (Tarrow and McAdam 2005).3Scale shift is an essential element ofall contentious politics, without whichcontention that arises locally wouldremain at that level. We can define itas a change in the number and levelof coordinated contentious actions toa different focal point, involving a newrange of actors, different objects, andbroadened claims (McAdam, Tarrow,and Tilly 2001: 331). It can also gen-erate a change in the meaning andscope of the object of a claim as afunction of its transfer.

Students of American politics arefamiliar with the process of scale shiftin the areas of welfare and regulatorypolicy. Until the 1930s, most policyinnovations began at the state level;only after long experimentation at thatlevel did they percolate up to the fed-eral realm. The same is true of con-tentious politics. For example, the19th century temperance movementshifted back and forth from the stateand local to the federal level severaltimes in response to opening andclosing political opportunities, beforesettling on the federal strategy of aconstitutional amendment (Szymanski2003: ch.3).

In his article, Sewell shows how anew national scale for politics wasestablished in France during thatcountry’s great revolution. This devel-opment began soon after the taking ofthe Bastille, when local uprisings putnew “patriot” administrations in powerall over the country (Sewell, 2003: 84-

Letter

5). But the nationalization of politicswas not a natural or an automaticphenomenon; it was the Jacobin club,with its network of provincial affiliates,that nationalized that revolution.When they seized power in 1792, theJacobins’ provincial societies becameinstruments of the Paris-organizedemergency government. As Sewellconcludes, “the country-wide networkof concerted opinion and civicactivism constituted by the Jacobinsocieties was probably as central tothe creation of a national scale of pol-itics as was the organization ofnational elections” (p. 85).

Scale shift can operate in two direc-tions: upward – in which case localaction spreads outward from its ori-gins; or downward, when coordinationat a higher level generates coordina-tion among lower-level units.4 Today’sinternational system offers a specialopportunity for activists because itopens conduits for upward shift ofcontention and brings new “global”practices back to the local level(Tarrow 2005). The “world” socialforums in Brazil and India were theresult of upward scale shift fromEuropean counter-summits in the1990s; many who attended thesesummits returned to their own coun-tries to establish versions of thesocial forum that soon becameengaged with local alignments(Tarrow 2005: ch. 7).

Some observers see in the global jus-tice protest events of the last decadesigns of the formation of a “global civilsociety” (Clark 2003; Fiorini 2003).There is much anecdotal evidence tosupport their view: the diffusion of aninternational support network for theZapatista rebels in Mexico (Olesen2005); the formation of groups likePeople’s Social Action following thatepisode (Wood 2005); and the dramaof contested international summitslike the WTO summit in Seattle, theG-8 in Genoa, and the more recent

Brokerage and diffusion combine inmany episodes of contention, produc-ing new coordination. Brokerage rein-forces diffusion, allowing individualsand groups that have had no previouscontact to coordinate their actionsthrough third parties with contactswith both. When Slobodan Milosevictried to steal local elections in Serbiain 1996, Otpor militants used a varietyof performances to oppose theregime (Tarrow 2005: ch. 6). Between2000 and 2004, the Otpor model dif-fused from Serbia to Georgia. Threeweeks of peaceful street protests cul-minated in a “March of the AngryVoters,” led by Mikhail Saakashvili,leader of the opposition coalition andthe country’s president afterShevardnadze was forced to stepdown.

Where had the opposition learnedsuch tactics? In the months beforethe election, Saakashvili had traveledto Serbia to contact the former organ-izers of the anti-Milosevic movement.He returned with a plan for nonviolentaction modeled closely on the suc-cess of Otpor. Ex-Otpor activists trav-eled to Georgia, where they led train-

“The diffusion of social

movements, revolutions, and

other forms of contentious

politics is similar in many

ways to the diffusion of inno-

vations.”

APSA-CP Vol 17, No. 14

Cancun and Hong Kong WTO sum-mits (Andretta and Mosca 2004). Dothese campaigns demolish nationalspace – at least as far as contentiouspolitics is concerned?

The death of the nation-state hasbeen announced before, but its pass-ing has always been found to be pre-mature. Not even the activists whoengage in “global” social movementsescape the spatial constraints of theirnational regimes, if only becausethey need to appeal to constituencieswhose vistas are far less global whenthey return home.

Consider the massive number ofEuropean activists who joined local“social forums” in Europe after takingpart in the Porto Alegre and MumbaiWorld Social Forums. WhenDonatella della Porta and her collab-orators interviewed many of them,they identified strongly with the strug-gle against globalization, but themajority were deeply embedded indomestic forms of activism (dellaPorta, et al. 2006). Rather thandemolishing space constraints, theywere taking the lessons of transna-tional activism back to the contestedspaces of domestic politics.

Conclusions

In this article, I have made somescattered observations about spaceand the need to enter into concretespace to understand the processesand mechanisms in which proximityand distance matters. Had I hadmore space (and more time!), I couldhave raised a number of researchquestions, answers to which wouldneed to be built on a better opera-tionalized notion of space. Abouteach of the dimensions I havesketched, I might have asked:

o About spatial co-presence: whatwas it about the co-presence ofworkers and peasants in the small

cities of Southern Italy that produceddisproportionately high votes for theCommunist party? And were thedynamics of interaction there anythinglike other class interactions in our field– for example, the interaction amongworkers and intellectuals that RobertFishman has found in Spain (2004)?

o About spatial and non-spatial diffu-sion: are the patterns of participationin the Tienanmen Square rebelliontypical of local forms of diffusion,while diffusion to higher levelsdepends more on impersonal chan-nels like the Internet? Or do the threeforms of diffusion – through interper-sonal networks, third-party brokerage,and impersonal media channels –complement one another, as theyseemed to do in the formation of theZapatista solidarity network (Olesen2005)?

o About scale shift: are globalizationand internationalization producingincreasingly transnational patterns ofcontentious politics? Or do theinducements and constraints ofdomestic politics continue to structurecontention?

As yet, I see little systematic evidencethat domestic social movements are“spacing out” into the uncharted ter-rain of global social movements. Buteven if they were, I would bet that thespatial boundaries of national politicsconstrain their activities – much as theboundary of the Mason-Dixon lineconstrained civil rights activism in theUS in the 1960s.

And that is because boundaries – andthe spaces they enclose – are notobjective factors that can be erasedby technology, travel, or mass com-munication. Boundaries are the rela-tional markers of identities, and identi-ties are the spaces within whichactivists and other people mobilize.But that is another story.

Notes

1 The contentious politics project wassponsored by the Mellon Foundationand sited at the Center for AdvancedStudy in the Behavioral Sciences, andowed much to the support of PhilConverse and Bob Scott at theCenter and to Harriet Zuckerman atthe Foundation. Members of thegroup to whom I owe thanks for theideas in this letter are Ron Aminzade,Jack Goldstone, Doug McAdam, LizPerry, Chuck Tilly and especially BillSewell, on whose work 2001 articlethis letter is heavily dependent. Otherproducts of the CASBS/Mellon con-tentious politics gang will be found inGoldstone, et al. 2003, in McAdam,Tarrow and Tilly 2001, and in the PhDdissertations of the fourteen graduatefellows associated with it.

2 Social control of contention also dif-fuses across space. For examples,see della Porta and Reiter 1998. Foran account of how the two streams ofdiffusion can intersect, see McAdam1983.

3 We prefer this more neutral term toSewell’s “scale-jumping,” whichimplies that scale shift is only anupward movement. Downward scaleshift seems to us an important part ofcontentious politics as well.

4 As often happens to me, it wasChuck Tilly who found a more felici-tous definition for a concept I wasusing loosely — “downward scaleshift.”

Letter

Note:

Complete citations for this issue areonline at http://www.nd.edu/~apsacp/backissues.html.

5APSA-CP Vol 17, No. 1 Symposium

After more than a decade of intensedebate about methodological princi-ples, some new ideas about how topractice comparative politics arebeginning to emerge. This sympo-sium provides short descriptions ofseveral of the most recent innovativemethods for evaluating theoreticalpropositions in comparative politics.Methodological innovation is not acompletely new phenomenon in ourdiscipline, especially in statisticalmethods, where it is a constant.However, it seems to us that the paceof innovation has accelerated recent-ly. Whereas over the past fewdecades comparativists could do acase study, write a comparative histo-ry, or run a regression, we now havealternative options such as designingcomputational simulations, embed-ding experiments in surveys, or sub-jecting fuzzy sets to Boolean logic.Although few scholars currentlyemploy these methods, all have thepotential to transform the way westudy politics. As a result, we thoughtit would be helpful to introduce you,our readers, to some of them.

Toward this end this symposiumbrings together concise descriptionsof three non-traditional methods byscholars who have pioneered theiruse. Each of these brief piecesdescribes the method and the prob-lem it addresses, defends its superi-ority over more traditional methods,specifies the phenomena that it canand cannot address, offers an exam-ple from published research, andadvises us on how we can becomeproficient in using it.

Kenneth Benoit and Michael Laverlead off with an introduction to auto-mated content analysis using a soft-ware package called Wordscores,

Few methods are more traditional incomparative politics than writing nar-ratives that reconstruct causalprocesses. In recent years, however,many scholars have suggested waysto improve upon this method, usuallyby putting greater emphasis on theo-ry. In our second contribution, TuliaFalleti surveys the family of methodsthat fall under the umbrella of “theory-guided process-tracing.” They includemethods proposed by AlexanderGeorge and Timothy McKeown,Alexander George and AndrewBennett, Ronald Aminzade, TimBüthe, Peter Hall, Robert Bates,James Mahoney, David and RuthCollier, and others, which have been

which they developed in collaborationwith Will Lowe. Wordscores, theyclaim, enables researchers to minetexts for useful information without thetaxing burden of actually readingthem. Although their claim is a boldone, the authors offer examplesshowing that this method has reliablylocated parliamentary speeches, partymanifestos, and amicus briefs on rel-evant ideological dimensions. In afield that relies as heavily on the con-tent of communications as ours, thepotential of such an approach seemsboundless … so long as the authors’claims withstand careful scrutiny.Benoit and Laver helpfully provide ref-erences for further reading for thosewho wish to give automated contentanalysis more careful consideration.

Innovative Methods in Comparative Politics

Introduction labeled “process-tracing,” “analyticnarratives,” “historical narratives,” and“systematic process analysis,” amongother terms. In her contribution Falletidefines and contrasts the most promi-nent examples of these related meth-ods, clarifying their common ele-ments, how they differ, and what theydo uniquely well.

The final contribution to the sympo-sium is that of Simon Hug, whoexplains selected innovations in (orimports into) quantitative comparativeanalysis. Comparativists are increas-ingly aware that many of our commonexplanatory variables are endoge-nous, i.e., caused by other explanato-ry variables. This is a problem whenthe unspecified determinants of theexplanatory variable are correlatedwith the unspecified determinants ofthe outcome. Hug shows that anotheroft-cited problem, selection bias, canbe understood as a special case ofendogeneity: it is a situation in whichour ability to observe a variable, notmerely its value, is a function of someother variables. Hug argues thatthese problems are just as relevantfor qualitative analysis as for quantita-tive methods, but that quantitativemethods provide various ways ofsolving them. For the benefit of read-ers who wish to learn to these meth-ods, he cites a variety of sources thatexplain more fully how to addressendogeneity and selection bias.

Automated content analysis.

Theory-guided process-trac-

ing. Statistical solutions for

endogeneity and selection

bias.

6 APSA-CP Vol 17, No. 1Symposium

tial as a source of information aboutpolitical actors.

Text Analysis as StatisticalAnalysis

The analysis of the text written orspoken by humans has traditionallytaken one particular form: being“read” for its “meaning,” in the sameway that an APSA-CP reader wouldread an article about Wordscores in asymposium. This form of essentiallyqualitative reading allows humans tocommunicate with each other usingwords that users believe to havesome mutually understood meaning.Various attempts by researchers to bemore systematic about this haveevolved into the measurement tech-nique of “content analysis,” the vastbulk of which still involves the searchfor systematic and replicable methodsof reading texts for meaning. Contentanalysis may be performed by humanreaders, who code either whole textsor units of text using a predefinedcoding scheme; or, conversely, it maybe performed with the assistance ofcomputers that use “dictionaries,”designed by human analysts with theparticular purpose of assigning indi-vidual words, or groups of words, intopredefined and meaningful codingcategories. Text coding schemes thataim to uncover the meaning of texts,however, are typically difficult toimplement, invariably rely upon sub-jective judgment calls by researchers,and often tend to be very costly interms of both time and intellectualeffort. And, because nothing comesclose to a functioning human brainwhen it comes to interpreting mean-ing, automated text coding schemesthat attempt to determine the mean-ing of texts are inherently unreliable.

The Wordscores method, by contrast,takes a fundamentally differentapproach to extracting informationfrom a corpus of text. Rather thanbeing concerned with the meaning of

Michael LaverNew York Universitymichael.laver@nyu.edu.

The Richness of Text as PoliticalData

Of the many sources of informationwe can collect to describe humanactivity, the words that humans writeand speak offer perhaps the mostimmense, yet also most underused,data resource. In the course of almostany form of political activity, politicalactors tend to generate text in theform of spoken or written activity, andthis text typically contains valuableinformation about the political orienta-tions and motivations of the actors.While not every type of political activi-ty gives rise to easily observabletexts, it is certainly the case that polit-ical texts such as campaign speech-es, party manifestos, legislative delib-erations, judicial decisions and sub-missions, executive addresses, gov-ernment statements, and treaty nego-tiations are increasing in number andavailability. The problem with politicaltext as a data source, indeed, stemsnot from its lack of easy availability,but rather from precisely the oppositeproblem: there is so very much textthat it becomes almost impossible to

analyze systematically. To separateand analyze the handful of needlesfrom the truckloads of hay, we needefficient and proven methodologiesfor sorting through huge volumes oftext.

Textual analysis offers numerousadvantages as a source of data inpolitical analysis, provided propermethodologies can be developed toanalyze it. Once texts have been writ-ten or speeches recorded, theybecome part of the historical record.Quite unlike techniques such as sur-vey research or other forms of inter-viewing, for example, the analysis oftexts does not require that a meas-urement instrument be constructed inadvance with some particular purposein mind, and does not by the very actof measurement disturb the environ-ment under investigation.

“...the words that humans

write and speak offer per-

haps the most immense, yet

also most underused, data

resource.”

The systematic analysis of text canbe deeply retrospective and perfectlynon-intrusive, while the same corpusof text can be analyzed for many dif-ferent reasons that were quite unan-ticipated at the time the text was gen-erated. These are prodigious advan-tages, especially in comparative poli-tics where other methods of gatheringinformation about political actors –such as interviews, surveys, or direct-ly observing non-textual behavior –are costly, difficult, or even danger-ous. While it would be silly to claimthat text analysis is a panacea for allresearch problems, there should beno doubt that it has immense poten-

AutomatedContent Analysisof Political TextsUsing Wordscores

Kenneth BenoitTrinity College, Irelandkbenoit@tcd.ie

7APSA-CP Vol 17, No. 1Symposium

reference texts in terms of relativeword frequencies both within andacross texts, Wordscores gives eachword in the set of reference texts ascore, representing the most likelypolicy position of a user of the word inquestion. The technique is now“primed” for an analysis of “virgin”texts, about which the analyst has noa priori information, but would like tolocate on the dimension under investi-gation. This list of scored words,derived from analyzing the referencetexts, is used to produce estimatedpositions for each of the virgin textsbased on their patterns of wordusage. The core of the method is verysimple and can be performed easilyusing only a spreadsheet. (Fordetails, see Laver, Benoit, and Garry2003).

As a fundamentally statistical method,Wordscores has three great advan-tages over more traditional methodsof text analysis. It requires no sub-stantive judgment call on the part ofthe investigator during the process ofdata analysis and is thus perfectlyreplicable; it operates in any lan-guage, since it does not involve thecreation or maintenance of a codingdictionary; and because it is a statisti-cal procedure, it generates an esti-mate of the uncertainty associated

with any estimated policy position.The last feature in particular distin-guishes Wordscores from nearlyevery previous form of quantitativecontent analysis, redressing a criticalomission with preexisting quantitativemeasures.

“Rather than being con-

cerned with the meaning of

text, we view a text as con-

taining behavioral informa-

tion about its author that can

be analyzed statistically,

based on systematic pat-

terns of word usage.”

text, we view a text as containingbehavioral information about itsauthor that can be analyzed statisti-cally, based on systematic patterns ofword usage. More specifically, the rel-ative frequencies of the use of partic-ular words by political actors provideobservable manifestations of underly-ing political orientations, beliefs, andattitudes. The words contained intexts, therefore, may be analyzed sta-tistically, without requiring the analystto interpret, synthesize, or even readthe texts being examined, in muchthe same way that a researcher con-cerned with public opinion would notattempt to read each respondent’scoded survey responses prior to ana-lyzing a survey dataset. Indeed,because Wordscores works in anylanguage by analyzing patterns ofword usage, it is not necessary thatthe analyst be capable of reading thetexts being analyzed.

Wordscores Explained

Wordscores is a form of uni-dimen-sional scaling technique that allowsresearchers to extract single-dimen-sional information from texts aboutwhich nothing is known a priori. Forexample, it can extract informationabout left-right positions from electionmanifestos, about attitudes towards anew “constitution” for Europe fromtreaty negotiations, or attitudestowards the government expressed inimpeachment or no-confidencedebates. No method – short of divinerevelation – can transform total igno-rance into valid new knowledge, how-ever, and Wordscores certainlymakes no such claim. WithWordscores, what happens is that theresearcher starts with a policy dimen-sion of substantive interest and a setof “reference” texts for which theinvestigator has valid independentinformation, or is confident of beingable to make assumptions, abouttheir positions on the dimensionunder investigation. By analyzing the

“The words contained in

texts [...] may be analyzed

statistically, without requiring

the analyst to interpret, syn-

thesize, or even read the

texts being examined...”

Because Wordscores maps a prioriknowledge of reference texts andscores onto the virgin texts underinvestigation, the analyst’s role inidentifying and scoring appropriatereference texts is crucial. If the refer-ence texts are inappropriate for thedimension under investigation, or iftheir estimated or assumed policypositions are misleading, then this willproduce misleading estimates of thepositions of the virgin texts. The key,therefore, is to build on a solid foun-dation of well-chosen reference texts,with solid estimates or assumptionsabout their policy positions.

Applications to ComparativeResearch

Wordscores assumes that texts con-tain information about the positions oftheir authors on separable dimen-sions on which they may be individu-ally scaled (although nothing con-strains a text to contain informationon only a single dimension). The pro-totypical case, and the one on whichLaver, Benoit, and Garry (2003) suc-cessfully demonstrated the use ofWordscores to extract party position-

8 APSA-CP Vol 17, No. 1Symposium

Wordscores has also been shown towork well when two opposing extremepositions are known, and the object isthen to place a number of unknowntexts along a “pro- versus anti-”dimension. Laver and Benoit (2002),for example, successfully demonstrat-ed the ability of Wordscores to place58 different speakers in a 1991 Irishconfidence debate, using the speech-es of two opposition party leaders(with a reference scores of -1.0 each)and the incumbent Irish prime minis-ter’s speech (reference score 1.0). Asimilar research design was used byBenoit et. al. (2004) to place 2,493written submissions from participantsin the Convention on the Future ofEurope along a dimension of beingfor or against the proposed “constitu-tion” of Europe. In another example,

ing information, was in the measure-ment of left-right policy positions ofpolitical parties using their electionmanifestos. In that paper, British partymanifestos in 1992 were used as ref-erence texts for an analysis of thepolicy positions of British party mani-festos in 1997, and Irish party mani-festos in 1992 as reference texts foran analysis of Irish party manifestosin 1997. The results indicated thatWordscores was able almost perfectlyto match independent expert place-ments of the 1997 virgin texts, includ-ing British Labour’s move to the cen-ter of the economic policy spaceunder Tony Blair.

other data, concluding that theimpeachment vote was predominantlyexplained by spatial positions.

Evans et. al. (2005) show that, of 104amicus curiae briefs submitted toaffirmative action cases before theSupreme Court, Wordscores accu-rately classified 89% of the amici forpetitioners and 75% for respondents,using no additional knowledge of anykind besides reference text based onfour principal briefs in one of thecases. As a whimsical yet also suc-cessful example of document classifi-cation and as part of the third andfinal round of revision of the originalWordscores paper for the AmericanPolitical Science Review, we used thetechnique to show the movementtowards acceptance of the three ref-erees between their first and secondreports (a brief report on this is avail-able from the Wordscores web site,below).

The second tantalizing application forautomated content analysis toolssuch as Wordscores relates to intra-party politics. Spatial models of partycompetition that go beyond the uni-tary actor assumption must modeldecision-making within parties whenthey select actions that serve as inputto inter-party competition. Empiricalimplementations will involve plottingmaps of, for example, the positions ofall party legislators. Particularly inparliamentary democracies, wherehighly disciplined parties mean thatlegislative roll calls are uninformativeabout the policy positions of individuallegislators, Wordscores analysis oftexts generated by members of parlia-ment offers the prospect of significantprogress. We have already noted aversion of this in the work of Bertelliand Grose (1996). A recent paper byGiannetti and Laver (2005b) used aWordscores analysis of delegatespeeches at party congresses to mapthe internal factional structure of theItalian DS party and to predict party

“Wordscores is thus a useful

tool for extracting positional

information from the texts as

estimates of the unobserv-

able positions of their

authors.”

speeches made during the Clintonimpeachment trial were used to showthe different positions of 73 senators,taking the speeches of Jesse Helms(+1.0) and Edward Kennedy (-1.0) asreference texts, correctly predictingtwo thirds of the impeachment voteson the perjury charge (Bertelli andGrose 2006). In a first application tojudicial decisions, McGuire andVanberg (2005) have applied themethod to extract ideological posi-tions from U.S. Supreme Court deci-sions.

The method also has been reliablydemonstrated to work in non-Englishlanguages, correctly classifyingGerman parties based on their mani-festos (Laver, Benoit, and Garry2003), French presidential speeches(Laver, Benoit, and Sauger forthcom-ing), and Italian ministerial positions(Giannetti and Laver 2005a).Unpublished trials using EuropeanUnion political documents have alsodemonstrated Wordscores to work ina statistically indistinguishable identi-cal manner on both French andEnglish versions of the same texts.

Thus far Wordscores has mainly beenused to estimate the positions onsome known dimension of politicalactors whose texts, but not positions,are observable directly. Wordscores isthus a useful tool for extracting posi-tional information from the texts asestimates of the unobservable posi-tions of their authors. We see a hugepotential for Wordscores in politicalanalysis, however, in two fields thatremain as yet largely untapped.

The first of these fields lies in thearea of document classification, andthe closely associated classification ofactors who generate the documents.Bertelli and Grose (2006), forinstance, use Senatorial speeches toclassify votes based on legal-constitu-tional grounds, contrasting this to aspatial-ideological prediction from

9APSA-CP Vol 17, No. 1Symposium

The method of process-tracing wasoriginally proposed to incorporate his-torical narratives within highlyabstract theories and explanations inthe social sciences. As such, themethod is not new. Although it hasbeen unearthed and polished recent-ly, the debates on the relationshipbetween historical narratives and the-oretical explanations, and betweenthe specificity of historical events andthe generalizations of law-like propo-sitions span several decades and var-ious disciplines.

In political science, one of the earliestexplicit definitions of process-tracingwas provided by Alexander George

Tulia G. FalletiUniversity of

Pennsylvaniafalleti@sas.upenn.edu

Theory-GuidedProcess-Tracing inComparativePolitics:Something Old,Something New

splits on key foreign policy roll calls.The next empirical phase of our ownwork will involve mapping the faction-al structure of each of the main Britishparties, using every word spoken inthe House of Commons between1992 and 2001.

Although Wordscores was originallydesigned for the analysis of politicaltexts, it has the potential to work wellin other fields where text analysis isperformed. This includes disciplinesrelated to political science, such ashistory, law, sociology, and anthropol-ogy, but also more applied fields suchas media studies, business and mar-keting, and even comparative litera-ture. Indeed, much emerging workusing Wordscores since the publica-tion of our 2003 original paper hastaken place in fields not directly relat-ed to political analysis. In any fieldwhere researchers need to extractinformation from the texts generatedby the actors they study, but who pre-viously have lacked a feasible meansto retrieve such information,Wordscores may be a useful researchtool.

How to Use Wordscores

Since the publication of Laver, Benoit,and Garry (2003), a suite of programsknown (unsurprisingly) asWordscores is available as a libraryfor the Stata statistical package, andis available fromwww.wordscores.com

That website also contains a beta ver-sion of a standalone Java implemen-tation of the software. Over the nextyear and a half we plan to makemany improvements in both themethodology as well as the softwareavailable for implementing it, includ-ing a book-length project onWordscores and related methods ofcomputerized content analysis.

References

Benoit, Kenneth, Michael Laver,Christine Arnold, Madeleine O. Hosli,and Paul Pennings. 2005 "MeasuringNational Delegate Positions at theConvention on the Future of EuropeUsing Computerized Wordscoring."European Union Politics 6(3,September): 291-313.

Bertelli, Anthony M. and Christian R.Grose. Forthcoming. "The SpatialModel and the Senate Trial ofPresident Clinton." American PoliticsResearch 34.

Evans, Michael, Wayne McIntosh,Cynthia L. Cates, and Jimmy Lin.2005. "Recounting the Courts?Toward A Text-Centered Compu-tational Approach to Understandingthe Dynamics of the Judicial System."Paper prepared for delivery at the2005 annual meeting of the MidwestPolitical Science Association,Chicago, Illinois, April 7-10.

Giannetti, Daniela and Michael Laver.2005a. "Policy Positions and Jobs inthe Government." European Journalof Political Research. 44(1): 91-120.

Giannetti, Daniela and Michael Laver.2005b. "Party Cohesion, PartyFactions and Legislative PartyDiscipline in Italy." Joint WorkshopSessions of the European Consortiumfor Political Research. Granada,Spain.

Laver, Michael and Kenneth Benoit.2002. Locating TDs in Policy SpacesUsing Computer Word-Scoring. IrishPolitical Studies 17:1, 59-72

Laver, Michael, Kenneth Benoit, andJohn Garry. 2003. "Extracting PolicyPositions from Political Texts UsingWords as Data." American PoliticalScience Review 97(2, May): 311-331.

Laver, Michael, Kenneth Benoit, and

Nicholas Sauger. (Forthcoming.)"Policy Competition in the 2002French Legislative and PresidentialElections." European Journal ofPolitical Research.

McGuire, Kevin T. and GeorgVanberg. 2005. "Mapping the Policiesof the U.S. Supreme Court: Data,Opinions, and Constitutional Law."Prepared for delivery at the AnnualMeeting of the American PoliticalScience Association Washington,D.C., September 1-5.

APSA-CP Vol 17, No. 110Symposium

only one independent variablechanges), the latter identifies thecausal mechanisms that connectcauses and effects. They definecausal mechanisms as “ultimatelyunobservable physical, social, or psy-chological processes through whichagents with causal capacities operate,but only in specific contexts or condi-tions, to transfer energy, information,or matter to other entities” (2005:137). To the extent that these agentsdo not need to be individuals, thismore recent definition of process-trac-ing allows for the identification ofcausal mechanisms that do not haveto be rooted at the individual level.Moreover, following Mahoney’s (2001)conceptualization of causal mecha-nisms, George and Bennett grant thatthese may ultimately beunobservable.

Recent Approaches

Other approaches have been appliedto what we can label the “theory-guid-ed process-tracing” (TGPT) method.There are important commonalitiesamong them but also significant dif-ferences, particularly regarding theutility of the TGPT method to gener-ate and test theories.

Sociologist Ronald Aminzade (1993)provides one of the most succinct andilluminating definitions of TGPT.According to Aminzade (1993: 108),the researcher has to provide “theo-retically explicit narratives that care-fully trace and compare thesequences of events constituting theprocess” of interest. Narratives, hewrites (1993: 108), “allow us to cap-ture the unfolding of social actionover time in a manner sensitive to theorder in which events occur. By mak-ing the theories that underpin our nar-ratives more explicit, we avoid thedanger of burying our explanatoryprinciples in engaging stories. Bycomparing sequences, we can deter-mine whether there are typical

sequences across [cases] … and canexplore the causes and conse-quences of different sequence pat-terns.”

The celebrated book by Ruth BerinsCollier and David Collier (1991) onthe different patterns of working classincorporation in eight Latin Americancountries and their effects on thepolitical systems and regimes of theregion constitutes an excellent exam-ple within the comparative politics lit-erature of the type of research designand method described by Aminzade.In their book, Collier and Collier ana-lyze the political developments ofeight countries along four paired com-parisons. Countries with similar typesof incorporation of institutionalizedlabor are paired together. At the sametime, major socioeconomic differ-ences between the countries of eachpair allow the authors to control forone of the alternative explanations(the impact of socioeconomicchange). The case narratives andstructured comparisons are organizedaround the conceptualization ofreform, incorporation, aftermath, andheritage periods, spanning most ofthe twentieth century. TGPT showsthat varying patterns of institutional-ization of the conflicts among domi-nant and middle classes, organizedlabor, and the state in the earlystages of the incorporation period hadsignificant effects on the features ofthe political parties, their links tounions, and the political regimes oflater periods.

Other methodological approaches toTGPT include those advanced byRobert Bates et al. (1998), Tim Büthe(2002), and Peter Hall (2003). The“analytic narrative” approach pro-posed by Bates and his coauthorscombines the theoretical tools ofrational choice and game theory withthe narrative method. They “seek toaccount for outcomes by identifyingand exploring the mechanisms that

and Timothy McKeown (1985), whodefined it as a method of within-caseanalysis to evaluate causal process-es. According to George andMcKeown (1985: 35), this methoddoes not solely rely on the compari-son of variations across variables ineach case, but also “investigate[s]and explain[s] the decision processby which various initial conditions aretranslated into outcomes” (emphasisadded). More concretely, the authors(1985: 35) argue that “[t]he process-tracing approach attempts to uncoverwhat stimuli the actors attend to; thedecision process that makes use ofthese stimuli to arrive at decisions;the actual behavior that then occurs;the effect of various institutionalarrangements on attention, process-ing, and behavior; and the effect ofother variables of interest on atten-tion, processing, and behavior.” Thusdefined, the method of process-trac-ing is deeply rooted in the tradition ofmethodological individualism. Itattempts to uncover the microfounda-tions of individual behavior that con-nect hypothesized causes and out-comes and to reduce the difficultiesassociated with unobserved contextu-al variables. Moreover, the authorsconceptualize process-tracing as amethod that not only permits the test-ing of hypotheses but also, unlike thestatistical method, the generation ofthem.

More recently, Alexander George andAndrew Bennett (2005: 206) definedprocess-tracing as the “method [that]attempts to identify the interveningcausal process - the causal chain andcausal mechanism - between an inde-pendent variable (or variables) andthe outcome of the dependent vari-able.” According to these authors, themain ontological difference betweenthe statistical method and the methodof process-tracing is that while theformer attempts to define causaleffects (i.e. the expected value of thechange in outcome when - in theory -

11APSA-CP Vol 17, No. 1

generate them” (Bates et al. 1998:12). Similar to George andMcKeown’s uncovering of the deci-sion-making process, the five collabo-rators (Bates et al. 1998: 11) in thisbook “seek to understand the actors’preferences, their perceptions, theirevaluation of alternatives, the infor-mation they possess, the expecta-tions they form, the strategies theyadopt, and the constraints that limittheir actions.” Analytic narrativesserve to illustrate the workings of apreviously specified and deductivelyderived model. The ultimate episte-mological goal of analytic narratives isto test models that yield unique equi-libria. This is, however, the most limit-ing aspect of this approach. Whennarratives seek to incorporate rela-tively long periods of time, changes invariables that are exogenous (orendogenous) to the model are likelyto occur and can significantly changethe actors’ preferences, options, andstrategies. The idea of stable equilib-ria is severely confining when appliedto historical narratives. Hence, analyt-ic narratives are not well suited toincorporating changes that reshapethe preferences of actors or theeffects of events that may render pre-viously available options no longerviable.

In an excellent methodological article,Tim Büthe (2002) proposes the con-cept of “historical narratives.” Thesenarratives seek to provide support fora theoretical argument and can beviewed as simplifications of realitythat depict a deductively sound, sys-tematic, regular relationship betweenvariables (2002: 482). Unlike theauthors of Analytic Narratives, Bütheallows for an analysis of endogeneityin his proposed approach. He recog-nizes the importance of feedbackloops in social phenomena. Theseare cases in which changes in thedependent variable at one point intime may lead to changes in the inde-pendent variables at a later point in

Symposium

time, which in turn will lead to furtherchanges in the dependent variable.

As other scholars have also noted(Pierson 2004), “[t]ime itself thusbecomes an element of the causalexplanation, a factor in the model”(Büthe 2002: 486). Historical narra-tives are useful for contextualizing thedifferent steps of the process ratherthan leaving them fragmented intoanalytical stages. Büthe (2002: 489),however, remains skeptical about theextent to which historical narrativescan assess alternative explanations.He argues that historical narrativesare, at best, “plausibility probes” ofmodels.

Peter Hall (2003), instead, gives ahigher epistemological status to hisproposed method of “systematicprocess analysis.” He observes thattheories of strategic interaction andpath dependence have recentlybrought to light complex ontologicalaspects of social phenomena. Issuesof timing, sequencing, complex inter-action effects, and multiple causalitiesare rendering traditional methods incomparative politics inadequate andwanting. The method he proposesbegins by specifying the set of theo-ries that identify the relevant causalfactors and how they operate. Fromeach theory, the investigator thenderives predictions about the patternsthat should be observed if the causaltheory is valid and if it is false. Otherrelevant observations, as many anddiverse as possible, are then made.The patterns present in these obser-vations are then inspected for consis-tency with the predictions of each ofthe relevant theories to determinewhich causal theory is superior to theothers (Hall 2003: 391-2).

Hall argues that in order to unfold theprocess that connects causes andoutcomes, small-N comparisons pro-vide much more explanatory leveragethan has conventionally been recog-

nized by the comparative method.Small-N comparisons allow scholarsto assess more complex causalprocesses in a much richer set ofobservations. According to Hall, sys-tematic process analysis is the mostpromising and fruitful venue toachieve an understanding of causalcomplexity.

The main difference among theserecent approaches to TGPT is theirconceptualization of the relationshipbetween theory and method. In theAnalytic Narratives project, TGPT pro-vides an edge to understandingactors’ preferences, expectations, andstrategies, which could not be eluci-dated through the application of othermethodologies. Yet, the function ofthe method is to illustrate how thetheory or formal model works in thereal world rather than to generate the-ories. According to Büthe (2002),TGPT is more appropriate than othermethods in the study of phenomenacharacterized by complex causality.However, he argues that, as amethod, TGPT can only present uswith plausible explanations, and can-not rule out alternative theories. Inother words, it can generate but can-not test theories. In contrast, Hall(2003) maintains that TGPT is anepistemologically superior method inthat it can map the ontological com-plexity of the social world and alsorule out competing theories. It cangenerate and test theories.

Causal Mechanisms, Periodization,and Proficiency

What do these different approachesto TGPT have in common? In its dif-ferent variants, the method of TGPTexplains the outcomes of interest bygoing back in time and identifying thekey events, processes, or decisionsthat link the hypothesized cause orcauses with the outcomes. Note thatthis procedure entails that theresearcher start with a set of hypothe-

APSA-CP Vol 17, No. 112

“competition” (Pierson 2004: 40-1,124-9), “institutional conversion,” and“institutional layering” (Thelen 2004:35-7). These mechanisms are notsimply intervening variables; instead,they refer to processes that can beapplied to empirical situations differ-ent from those in which they were firstconceived.

The utilization of the TGPT methodposes two difficult questions to theresearcher: when the narrative shouldstart and when it should end. How farback the researcher goes in tracingthe process is totally dependent onher theoretical framework and thecompeting theories under considera-tion, which define the theoretically rel-evant cause or causes to be studied.This, however, is not always as con-clusive a criterion as we might like.The issue of starting points may behighly contentious. Collier and Collier(1991) start their TGPT in moments ofcritical junctures, whereas Mahoney(2000) argues that only contingentevents (and not all critical juncturesare contingent events) can triggerpath-dependent processes. Whetheror not we adopt a more stringent defi-nition of starting point, we shouldalways justify the choice of periodunder study. Whereas the justificationof the starting point may be contest-ed, the end point is easier to estab-lish; there is less disagreement onthis issue since the end point is large-ly determined by the presence of theoutcome of interest.

In linking the start and the end of thesequence, the researcher whoapplies the TGPT method has toreconstruct an explicit chronology ofthe sequence of events that constitutethe process of interest. This is not asimple task; it requires a precise con-ceptualization of the types of eventsthat are constitutive of the process aswell as those that are not - even ifthey pertain to related, albeit different,processes. Therefore, an explicit the-

“How far back the researcher

goes in tracing the process

is totally dependent on her

theoretical framework and

the competing theories under

consideration, which define

the theoretically relevant

cause or causes to be stud-

ied.”

Lots of ink has been spilled fuelingthe debate between the advocates offieldwork and archival research, onthe one hand, and the advocates ofthe sufficiency of secondary sources,on the other. Exemplary books andarticles have been written using bothresearch strategies, thus making it

Symposium

ory underpinning our process of inter-est as well as a profound familiaritywith our cases are both necessary.

Successful use of the TGPT methodrequires the researcher to tap into hersociological or political imagination inorder to identify the theories relevantto the problems and puzzles sheseeks to explain, and to be able toderive feasible causal mechanisms.Moreover, familiarity with the case orcases under study is crucial.Familiarity with the history, historiog-raphy, and politics of the cases ofstudy makes it possible to avoid prob-lems of selection bias (Lustick 1996)and to improve the validity and relia-bility of our proposed narratives(Vitalis 2006). This requirement, inturn, poses two other practical ques-tions: Is fieldwork always necessary?How many cases can a single authorstudy using the TGPT method?

ses – preferably competing hypothe-ses – about the relevant causes andhow they connect to the final out-come. The method of TGPT seeks tospecify the mechanisms linking caus-es and effects.

“The method of [theory-guid-

ed process-tracing] explains

the outcomes of interest by

going back in time and iden-

tifying the key events,

processes, or decisions that

link the hypothesized cause

or causes with the out-

comes.”

Unfortunately, there is astonishinglylittle consensus about what the“causal mechanisms” are. JamesMahoney (2001: 579-80) identifiedtwenty-four definitions of causalmechanisms in the literature. In manycases, causal mechanisms becomesynonymous with “intervening vari-ables.” However, if causal mecha-nisms were to be reduced to interven-ing variables, which could be opera-tionalized and measured using othermethods, then TPGT as a methodwould lose most of its comparativeadvantage. Ideally, causal mecha-nisms should be portable concepts.They should be of a higher level ofaggregation than intervening vari-ables. Causal mechanisms shouldidentify relationships between condi-tions and outcomes that can beapplied to other contexts (whichshould pertain to the theoreticaldomain of the theory proposed; seeGeddes 2003: 152). Examples ofthese mechanisms are “learning,”

APSA-CP Vol 17, No. 1 13

difficult to establish a priori thesupremacy of either approach.

However, in the early stages of ouracademic careers as comparativists,it would be hard to compensate forthe many insights we get through in-depth interviews, the analysis of pri-mary documents and archives, andthe overall experience of living in thesociety whose politics we seek toexplain. Independent of the researchstrategy we choose, the TGPTmethod requires in-depth knowledgeof our cases. For this reason, worksthat apply TGPT will be closer to thesmall-N end than to the large-N endof the sampling spectrum. If, as Hall(2003) says, this is the price we payto understand complex causality, thetrade-off is worth it.

Symposium

The increasing salience of TGPT as aqualitative method can be appreciat-ed in the sessions offered by theConsortium on Qualitative ResearchMethods (CQRM). In January of thisyear, the CQRM (for the fifth consec-utive year) offered a ten day-courseon Qualitative Research Methods atthe University of Arizona. At least fourof the twenty-two general sessionsoffered dealt explicitly with differentaspects of TGPT, such as the“process tracing” method, “compara-tive-historical methodology,” “histori-ography, archival research, and tem-

“...the TGPT method requires

in-depth knowledge of our

cases. For this reason,

works that apply TGPT will

be closer to the small-N end

than to the large-N end of

the sampling spectrum.”

porality,” and “temporality, criticaljunctures, and path dependence.”

The TGPT method can provide a cog-nitive map for the type of ontologicalworld described by Hall (2003). In thisregard, the TGPT method has severaladvantages compared with traditionalmethods. First, as emphasized byGeorge and Bennett (2005) and Hall(2003), it permits the study of com-plex causal relationships such asthose characterized by multiplecausality, feedback loops, pathdependencies, tipping points, andcomplex interaction effects. Second, itcan lead to the formulation of newtheories or hypotheses on the causalmechanisms that connect correlatedphenomena. Third, TGPT in struc-tured, focused comparisons (i.e.,measuring the same causal mecha-nisms and outcomes in the same wayacross each case) permits the testingof hypotheses and theories. Finally,the TGPT method can reveal howendogenous changes affect the evo-lution of our variables of interest. Forall these reasons, the careful applica-tion of TGPT to generate and testhypotheses will continue to advanceour knowledge about complex causal-ity phenomena in comparative poli-tics.

References

Aminzade, Ronald. 1993. "ClassAnalysis, Politics, and French LaborHistory." In Rethinking Labor History,edited by L. Berlanstein. Urbana andChicago: University of Illinois Press,90-113.

Bates, Robert H., Avner Greif,Margaret Levi, Jean-LaurentRosenthal, and Barry R Weingast.1998. Analytic Narratives. Princeton,NJ: Princeton University Press.

Büthe, Tim. 2002. "Taking TemporalitySeriously: Modeling History and theUse of Narratives as Evidence."

American Political Science Review 96(3): 481-493.

Collier, Ruth Berins, and DavidCollier. 1991. Shaping the PoliticalArena. Critical Junctures, the LaborMovement, and Regime Dynamics inLatin America. Princeton, NJ:Princeton University Press.

Geddes, Barbara. 2003. Paradigmsand Sand Castles: Theory Buildingand Research Design in ComparativePolitics. Ann Arbor: The University ofMichigan Press.

George, Alexander L. and AndrewBennett. 2005. Case Studies andTheory Development in the SocialSciences. Cambridge, MA andLondon, England: MIT Press.

George, Alexander L. and Timothy J.McKeown. 1985. "Case Studies andTheories of Organizational DecisionMaking." Advances in InformationProcessing in Organizations 2: 21-58.

Hall, Peter A. 2003. "AligningOntology and Methodology inComparative Politics." In ComparativeHistorical Analysis in the SocialSciences, edited by J. Mahoney andD. Rueschemeyer. New York:Cambridge UP, 373-404.

Lustick, Ian. 1996. "History, Historia-graphy, and Political Science: MultipleHistorical Records and the Problem ofSelection Bias." American PoliticalScience Review 90 (3): 605-618.

Mahoney, James. 2000. "Pathdependence in historical sociology."Theory and Society 29 (4): 507-548.

Mahoney, James. 2001. "BeyondCorrelational Analysis: RecentInnovations in Theory and Method."Sociological Forum 16 (3): 575-593.

Pierson, Paul. 2004. Politics in Time.History, Institutions, and Social

APSA-CP Vol 17, No. 114 Symposium

1 Introduction

Scholars in comparative politicsalmost by definition can only resort toquasi-experimental studies to testtheir theories. Few are the situationswhere a comparativist finds her/him-self in the lucky situation of havingcontrol over an explanatory variable.This becomes most obvious when weturn to the question of whether institu-tions matter (e.g., Przeworski 2004).While comparativists have becomemore self-conscious in making theo-retically derived causal claims, theyhave also been alerted to two relatedproblems which make testing theircausal claims difficult, or, perhapseven impossible. These two inter-twined problems relate to the quasi-

experimental nature of most empiricalresearch in comparative politics andare described as selection biases andthe problem of endogeneity.

In this short note, I wish to demon-strate that these two problems are oftantamount importance, independentof the research strategy, i.e., quantita-tive or qualitative, employed. Giventhe more solidly developed tools toaddress these problems in quantita-tive research, I will discuss theseproblems in this context and highlightways in which they can beaddressed, albeit sometimes onlyimperfectly. I start by presenting theproblem of endogeneity bias, showingalso how it relates closely to selectionbiases.1 I examine this second prob-lem in section three, before conclud-ing in section four.

2 Models of endogeneity

A frequent approach in comparativepolitics is the attempt to assess whyparticular outcomes differ amongcountries. An important researchstrand attributes considerable weightto the role of institutions:

(1)

Underlying this simple equation is theassumption that if in a particularcountry different institutions, e.g.,those of a democracy instead ofthose of an autocracy, were present,the outcome would be different. To tiethings down more precisely, let’sassume that indicates thepresence of a particular institutionand its absence. Then ourstatement would involve a compari-son of the outcome under twoscenarios. Let’s assume that weobserve in our dataset the outcomefor a particular observation forwhich . To assess the effect of

we would need to know theoutcome for .Unfortunately, we never observe

. To circumvent thisproblem we normally assume that theeffect

can be assessed by comparing caseswith 0 and 1 as values of . Moreprecisely we resort (in a linear regres-sion framework) to the followingmodel:

(2)

Estimating this model under the well-known assumptions of the classicallinear regression (CLR) would yieldan estimate of our quantity of interest,

. Obviously, this model is veryunlikely to satisfy the assumptions ofthe CLR model. Most glaringly, it israther unlikely that no omitted vari-ables influencing our outcome areunrelated to our included variable.

There are at least two reasons whythis might occur. First, several vari-ables might affect our outcome vari-able and be related to , forinstance, because the chosen institu-tion affects this other variable.Controlling for these variables byincluding them as a set of variableswould address this issue. Second,variables affecting might also berelated to the presence of , whichresults in the problem of endogeneity(of to be precise). Again, includingthese additional variables as controlsmay help address this issue. In orderto assess the effect of precisely,these variables included in whichexplain the presence of must eitherexplain the presence of perfectly,or whatever is not explained is unre-lated to our outcome .2 Both ofthese eventualities are unfortunatelyquite unlikely, especially in compara-tive politics. Our models which try toexplain, for instance, the presence ofa particular institution, are oftenentailed with a considerable error:3

(3)

Endogeneity andSelection Bias inComparativeResearchSimon HugUniversität Zürichsimon.hug@access.unizh.ch

)(= nsinstitutiofoutcome

1=1x

0=1x

iyi

1=1,ix

)( iyE

1x

1=1,ix

0)=|( 1,ii xyE

0=1,ix

0)=|(1)=|( 1,1, iiii xyExyE −

εββ ++ ixi xy 0=

xβ̂

y

x

z

yx

x

xz

xx

y

ijij

l

ji zx θδδ ++ ∑

1=0=

Analysis. Princeton and Oxford:Princeton University Press.

Thelen, Kathleen. 2004. HowInstitutions Evolve. The PoliticalEconomy of Skills in Germany,Britain, the United States, and Japan.New York: Cambridge UP.

Vitalis, Robert. 2006. "The Past IsAnother Country." In A Handbook forSocial Science Field Research.Essays & Bibliographic Sources onResearch Design and Methods,edited by Ellen Perecman and SaraR. Curran. New York: SagePublications.

APSA-CP Vol 17, No. 1 15

to be good instruments, since theyare correlated with another variable,namely human capital, which alsoexplains economic growth.

Another approach to the problem ofendogeneity is matching. Intuitivelywe need to find for observationswith cases which are (almost)identical in all other variables toobservation , except that .

These matching methods, as nicelyillustrated in Persson and Tabellini’s(2003) work on the economic effectsof political institutions, involves esti-mating equation 2.6 The predictedprobability gives us an indication ofhow close a particular observation isto another one with a different valueon . We can compare, for instance,the value of for pairs of observa-tions that are similar in their estimat-ed probabilities of having the institu-tion but different with respect towhether the institution is actuallypresent. The average of the differ-ences in the values of for all thepairs is our estimate of the effect of. .7 As Persson and Tabellini’s(2003) work nicely illustrates, thisapproach also allows researchers toaddress the problem of endogeneity.

While I discussed the problem ofendogenous regressors in the contextof a classical linear regression model,similar solutions exist for limiteddependent variables (Rivers andVuong, 1988; Bollen, Guilkey, andMroz, 1995). Interesting studiesemploying this approach to deal withendogeneity bias range fromElbadawi and Sambanis’s (2002)study of the incidence of civil wars toPrzeworski, Alvarez, Cheibub, andLimongi’s (2000)8 work on the effectof democracy on human well-being.The former authors, surprisingly hard-ly find any evidence for endogeneityin their empirical model explaining theoutbreak of civil war. The latterauthors, however, find quite dramatic

Symposium

differences in their study on democra-cy and human well-being. Mostimportantly, probably, is their findingthat economic development can hard-ly explain democratic transitions, butthat it is clearly associated with dem-ocratic survival.

This latter study also highlights theclose connection between addressingproblems of endogeneity and selec-tion biases. The easiest way to graspthis is to observe that in the notationused at the beginning of this note wefail to observe forobservations with . Thus, byrestricting our analysis to the caseswith we have a subset of oursample of observation, and this sam-ple is selected. Ignoring this selectionmay lead to biases in our estimates.9To derive these biases, I will detail inthe next section the basic elements ofselection bias models.

3 Models of selection bias

At least since the publications ofAchen (1986) and Geddes (1991),scholars in comparative politics havebeen aware of the problem of selec-tion biases. Increasingly interestingapplications of models addressingthis problem appear in journals andbooks.

The problem arises when (at least)our dependent variable is observedfor only a subset of the observationsof sample we wish to study. Thus, inthe model

(4)

we have

(5)

Under the assumption that is relat-ed to some independent variableswe have

1=ix

0=ix

0)=|( 1xyE1=1x

1=1x

εβββ +++ 22110= xxy

0= if . 1= if

=tty

y obs

i

j

xy

x

y

tz

y

In this equation correspond toadditional explanatory variables whichwe think contribute to the presence ofinstitution . Equation 2 clearlyshows under what circumstancesadding variables in equation 3 isan adequate strategy. As long asin equation 2 is uncorrelated with the

in equation 1, adding these con-trols solves our problem of endogene-ity. In most cases the assumption that

is unwarranted, whichleads to biased estimates of the effectof a particular institution as deter-mined by .

Equation 2 also shows a way toaddress the problem of endogeneity.By construction

obtained by estimating equation 2and using the estimated coefficientsto obtain predicted values for , willbe uncorrelated with provided thatthe s are uncorrelated with anycontrol variables we might wish toinclude in equation 2.4 Insertinginstead of will solve our problem.This solution, based on instrumentalvariables (IVs), has become commonpractice.5 It is not, however, withoutits pitfalls. First, as Bartels (1991) andBound, Jaeger, and Baker (1995)have authoritatively shown, if theinstruments for are weak, i.e.,the fit of equation 2 is low, our esti-mated coefficient for may bevery imprecise. Second, findinginstruments uncorrelated with otherincluded variables is often difficult. Anillustrative example comes from theongoing debate about whether goodinstitutions enhance economicgrowth. Aware of the endogeneity ofgood institutions Acemoglu, Johnson,and Robinson (2001) employ asinstrumental variables settler mor-tality and indigenous population den-sity in 1500. Glaeser, Porta, López deSilanes, and Shleifer (2004) argue,however, that these two variables fail

zj...zl

0=εθσ

xβ̂

jijl

ji zx δδ ˆˆ=ˆ1=0 ∑+

ε

ε

θ

jz

x̂

X̂β̂

x

x

z

x

xz

z

APSA-CP Vol 17, No. 116Symposium

(6)

If , then estimating equa-tion 1 on the subset of cases forwhich is observed yields unbiasedestimates. If , however,then biases are to be expected. Thisis all the more likely if the set of vari-ables contains elements appearingin equation 3. If these are imperfectmeasures, omitted variables are likelyto be similar across the equations 3and 5, inducing covariation betweenthe two error terms. Solutions to thisproblem exist but differ, however,according to the availability of data. Ingeneral, the setup consists of equa-tions 3 and 5.

The question now is what remainsunobserved if . The simplestcase is if only is unobserved (as inequation 4), while all the remainingvariables are observed. Heckman(1976) derived a simple estimator dis-cussed in detail in Achen (1986). Inessence, it consists of estimating in afirst step the equation as a probitmodel and then use the inverse of theMills ratio as additional regressor inthe outcome equation.10

In the more complicated case, neither nor are observed when . In

that case a truncated regressionresults. For obvious reasons, a firststage selection equation cannot beseparately estimated, since for allobserved cases , hence no vari-ation appears in the probit model.Outcome and selection equation can,however, be estimated jointly, sincethe two equations together induce alog-likelihood function for theobserved cases (e.g., Maddala 1983,176ff). This function reflects the likeli-hood that, conditional on beingobserved, observations take on par-

ticular values for the dependent vari-able.

Hug (2003), building up on Muthenand Jöreskog (1983), provides MonteCarlo simulations for this estimator.He demonstrates that with sufficientcorrelation between the error terms inthe equations 3 and 5, a sufficientlylarge sample, and a minimum ofselectivity, the estimator clearly out-performs a simple OLS estimation ofthe outcome equation. He also showsin three empirical examples dealingwith the success of new political par-ties, the degree of violence in demon-strations, and the rebellious behaviorof “minorities at risk,” that selectioneffects are quite real.

For the simpler models, Achen (1986)based on Heckman’s (1976) work,offers a very accessible introduction,as well as Breen (1996). While mostof the issues discussed here are pre-sented in a concise fashion inMaddala (1983), Greene (2003, 780-790) discusses these issues in greatdetail. Dubin and Rivers (1990) offera general treatment of selection mod-els when dealing with limited depend-ent variables.

More recently, scholars have becomeinterested in addressing selectionbiases in more complex situations.Both Brehm (2000) and Boehmke(2003) suggest ways to use externalinformation to arrive at an estimate ofthe Mills ratio necessary in the sec-ond stage of a Heckman model, evenif a first-stage selection equation can-not be estimated separately.

Sartori (2003) derived, under theassumption of identical errors acrossthe two equations, an estimator for abinary selection model without exclu-sion restrictions. Finally, Boehmke,Morey, and Shannon (forthcoming,2006) present an elegant way toaddress selection bias in durationmodels.

Nice illustrations of the relevance ofmodels of selection bias appear inPrzeworski and Vreeland (2000), whohighlight the effect of selection biasesin assessing the role of IMF loans.While other authors find very diverg-ing effects of IMF loans on economicgrowth, Przeworski and Vreeland(2000) report very subtle differences.During the loan program, countriessee lower economic growth than theywould have if they had not requesteda loan. After the end of the program,these same countries grow, however,faster than comparable countries thatrefrained from entering into it. Thedecision to join a loan programappears to be influenced by the gov-ernments’ willingness to adopt politi-cally costly adjustment policies.

Similarly, Nooruddin (2002) showsselection effects in the results of eco-nomic sanctions. When one does notconsider why the US imposed sanc-tions against a country, the successof the sanctions does not appear tobe related to the country being alliedto the US; but when the selectionmechanism is taken into account, USties contribute to the explanation.Specifically, the success of sanctionsdepends on whether the target wasinvolved in a military dispute andwhether there was cooperation withthe US.

4 Conclusion

Comparative research, whether quali-tative or quantitative, is almost by def-inition confronted with problems ofendogeneity and selection bias. Totest our theories, we would ideallyneed experimental control, but inpractice we can only rely on quasi-experimental data. This should makeus all more sensitive to the problems Idiscussed in this note. While someauthors (e.g., Przeworski, 2004)come close to suggesting that study-ing the effect of institutions is impos-sible because of these problems, I

≥

++ ∑

0< if 00 if 1

= and

=

*

*

1=0

*

i

ii

iji

l

ji

tt

t

zt θδδ

0=,θεσ

0=, /θεσ

ti<0

ti

ti<0

ti≥0

y

z

y

y x

APSA-CP Vol 17, No. 1 17

Hence, even if our explanatory vari-ables are endogenous or our sampleis only a subset of the cases we wishto study, ways exist not only toassess whether this results in biasesbut also to correct them. Required forthis, however, are theoretically well-grounded empirical models explainingthe presence of institutions and theireffects. Given that the problems ofendogeneity and selection biases arealmost a mainstay of comparative pol-itics, comparativists should always beaware of these potential problems.

Notes

1 This close link is discussed in detailby Przeworski, Alvarez, Cheibub, andLimongi (2000, 279-289) and Perssonand Tabellini (2003, 113-153).

2 In technical terms this is oftenreferred to as “selection on observ-ables.”

3 Here I gloss over the problem that alinear model is normally hardly appro-priate if is dichotomous, even

would argue that adequate empiricalmodels may alleviate some of themost egregious biases.

though Angrist and Krueger (2001)suggest using a linear probabilitymodel in such situations.

4 If control variables are likely to becorrelated with the instruments,Acemoglu (2006) pleads for the inclu-sion of the control variables in the firststage regression as additional regres-sors.

5 Accessible introductions and sur-veys of the relevant literature on IVestimation appear in Angrist andKrueger (1999) and Greene (2003),while Hausman (1978) discusses thepossible specification tests.

6 Persson and Tabellini (2003, 113-153) provide an accessible survey ofthe problems discussed in this note,and present the basic logic of match-ing methods (138-142). Acemoglu(2006 forthcoming) criticizes, howev-er, their implementation of the IV-esti-mator and the usefulness of some ofthe instruments used to predict thepresence of particular institutions.

7 This form of matching is only one toderive estimates for the quantity ofinterest to us (see Persson andTabellini 2003, 138-142).

8 These authors rely both on instru-mental variables and selection mod-els to address the endogeneity ofpolitical institutions.

9 Given that is a constant in thissubset of observations, the effect ofthis variable is absorbed into the con-stant.

10 A constraint has to be respected,namely that in the set of variablesat least one variable does not appearin the set of variables . A full-infor-mation likelihood estimator exists aswell for this model.

References for this article are on-lineat www.nd.edu/~apsacp

“Comparative research,

whether qualitative or quanti-

tative, is almost by definition

confronted with problems of

endogeneity and selection

bias [...] [A]dequate empirical

models may alleviate some

of the most egregious bia-

ses.”

x

x

x

z

In 2005, the World Bank made publicthe latest update of its GovernanceIndicators, a comprehensive paneldataset that covers six dimensions ofgovernance for practically all coun-tries in the world, plus several associ-ated territories.1 Data are available foralternating years during the period1996-2004. This update not only addsthe 2004 estimates to the series, butit also revises estimates for previousyears on the basis of newly availabledata. Clear and complete method-ological and substantive informationabout this data source is found inKaufmann, Kraay, and Mastruzzi(2004) and, for the last update, inKaufmann, Kraay, and Mastruzzi(2005). The World Bank does notplan to extend the database back intothe past (because many of the datasources are not available before1996), but will continue updating it inthe future. The next wave of indica-tors for 2006 will be available in early2007.

The six dimensions of governanceidentified and measured by the WorldBank are: 1) Voice and Accountability(political, civil, and human rights); 2)Political Instability and Violence (thelikelihood of violent threats to govern-ments, such as terrorism); 3)Government Effectiveness (the quality

Data Set Review:The World Bank'sGovernanceIndicators (1996-2004)

Carlos GervasoniUniversity of

Notre Damecgervaso@nd.edu

Symposium

18 APSA-CP Vol 17, No. 1

the governance situation within acountry; and 2) assessments by spe-cialized agencies such as risk ratingcompanies, universities, NGOs, andmultilateral aid organizations whosestaffs include knowledgeable countryanalysts or in-country correspon-dents. The data available for eachcountry from this diverse array ofsources is then combined through anunobserved components model,3which allows calculating the precision(i.e., the standard error) of thescores.4

Each of the six indicators is a stan-dard normal variable (by definitionmean=0 and standard deviation=1),so that almost all scores lie in the -2.5to 2.5 range (higher scores corre-spond to better performance). Thefact that each wave includes moresources implies that the estimates aremore precise in the most recentyears: the average standard error forall countries in each of the six dimen-sions ranges from 0.26 to 0.36 in the1996 wave, while it ranges from 0.18to 0.27 in the last series. However,the accompanying methodologicalpaper emphasizes that, even in 2004,the magnitude of standard errors isnot trivial, and therefore small differ-ences among countries are typicallynot significant.

Although relying exclusively on sub-jective measures may be seen as aweakness, especially by researchersaccustomed to working with allegedlymore accurate ones, the authorsmake a strong case for the former.They not only argue that key gover-nance dimensions, such as corrup-tion, are basically impossible tomeasure objectively, but also pointout that perceptions-based measuresoften capture information missed byobjective ones, especially when thelatter assess de jure situations thatare often poor descriptions of de factorealities (Kaufmann, Kraay, andMastruzzi 2005, 27-31).

Datasets

The database, downloadable as anExcel spreadsheet (fromhttp://www.worldbank.org/wbi/gover-nance/govdata/), does not have thetypical “long” format but the “wide”format for panel (or longitudinal) data.Each of the 209 rows is a country andeach of the 90 substantive columnscorresponds to the dimension score,its standard error, and the number ofsources on which the estimate isbased (that is, 3 columns*6 dimen-sions*5 years=90).

Despite the obvious usefulness of thesix indicators on their own, they are infact the product of the operationaliza-tion of the concept of governance,understood as having three dimen-sions: 1) “the process by which gov-ernments are selected, monitored andreplaced”; 2) “the capacity of the gov-ernment to effectively formulate andimplement sound policies”; and 3)“the respect of citizens and the statefor the institutions that govern eco-nomic and social interactions amongthem” (Kaufmann, Kraay, andMastruzzi 2005, 130). Voice andAccountability and Political Instabilityand Violence measure the first dimen-sion, Government Effectiveness andRegulatory Quality the second, andRule of Law and Control of Corruptionthe third one.

Given this conceptualization, it isimportant to assess whether theseindicators are measuring a singleunderlying “governance” variable, thethree independent dimensions men-tioned above, or maybe other dimen-sions not foreseen by the authors ofthe study. My factor analysis of thesix indicators in 2004 provides strongevidence in favor of the single dimen-sion hypothesis: the first factorexplains 87.3% of the total varianceand correlates very highly with allindicators (from a maximum of .984for Rule of Law to a minimum of .856for Voice and Accountability). Thisfinding suggests that an aggregate

of the bureaucracy and public servic-es); 4) Regulatory Quality (the inci-dence of market-unfriendly policies);5) Rule of Law (the quality of contractenforcement, the police and the judi-ciary, and the likelihood of crime); and6) Control of Corruption (the exerciseof public power for private gain).Political scientists in general, andcomparativists in particular, will recog-nize in these labels some of theirmost interesting dependent variablesand theoretically crucial independentvariables.

Each of the five waves from 1996 to2004 has incorporated more countriesand data sources. The average num-ber of countries covered in each ofthe six dimensions was 173 in 1996(with a minimum of 151 in the“Control of Corruption” dimension),monotonically increasing since thento the current 207 (with a minimum of204 in the “Regulatory Quality” and“Control of Corruption” dimensions).Likewise, the number of sources percountry used in the construction ofthe measurement for each dimensiongrew from a median of 4 in 1996 to amedian of 9 in the 2004 wave. Thepercentage of countries with only onedata source declined from 15% to 7%in the same period.

The array of data sources used toconstruct each indicator is impres-sive. For the 2004 wave, the WorldBank used 352 variables drawn from37 data sources (12 of them used forthe first time in 2004), produced by 31different organizations, such as theEconomist Intelligence Unit, FreedomHouse, Gallup International, theHeritage Foundation, Political RiskServices, State Department/AmnestyInternational, and the World Bank.2

All the data sources are subjective (orperceptions-based) measures. Theycan be broadly divided into two types:1) surveys of individuals and domes-tic firms with first-hand knowledge of

19APSA-CP Vol 17, No. 1 Datasets

“Index of Governance” can andshould be constructed. Although eachdimension taps a conceptually (andfor some countries empirically) differ-ent trait, it is apparent that the highinter-dimension correlations points tothe existence of a latent variable thatstrongly influences all of them.

Governance is generally assumed tobe a relatively enduring feature ofpolitical systems. The World Bankindicators confirm this idea: changesfrom wave to wave are not very large.The correlations between the scoresin 1996 and the scores in the subse-quent waves decline over time butremain always very high. For Voiceand Accountability, the Pearson corre-lation coefficient between 1996 and2004 scores is .927, for GovernmentEffectiveness .925, for Rule of Law

.920, for Control of Corruption .913,for Regulatory Quality .861, and forPolitical Stability .810. In sum, thestability of scores is similar and veryhigh for the first four indicators, some-what lower for Regulatory Quality,and still lower for Political Stability.But even in this case, a 1996 countryvalue is a rather good predictor of its2004 score: the adjusted R2 is .655and the mean residual is .58 (a rela-tively low figure for a variable that hasa range of approximately 5).

The website for the database has abuilt-in utility for obtaining data andgraphs for specific countries andregions, which permits an easy andquick access to snapshots of thedata. More complex graphs and sta-tistics require downloading the fullExcel database and making minor

Figure 1. Average Governance Scores by World Region (1996-2004)

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

1996 1998 2000 2002 2004

Ave

rag

e S

core

East. Europe&Baltics

East Asia

Latin America

Middle East & N. Africa

South Asia

Subsaharan Africa

OECD

changes to it in order to make it read-able by statistical, graphing, or fileconversion packages.

Figure 1 summarizes the data byregion and year. In order to avoidcluttering the graph, I collapsed thesix series of data into a single line perregion that represents the unweightedaverage of the six governance indica-tors. Because of the relative nature ofthe indicators, the aggregate statisticsare zero-sum: if a country or regiongoes up another one has to go down.Therefore, the trends in the graphhave to be interpreted in relativerather than absolute terms.

For example, the slight decline of theOECD countries in the period 1998-2004 is probably more a function ofabsolute improvement in Eastern

APSA-CP Vol 17, No. 120

and its intention to keep producingand improving the database in thefuture, the uses, analyses, and criti-cisms that political scientists make ofthese data will likely lead to a virtuousfeedback circle. This will, a few yearsfrom now, allow the profession tocount on a temporally longer andqualitatively better GovernanceIndicators database.

References

Kaufmann, Daniel, Aart Kraay, andMassimo Mastruzzi. 2004.“Governance Matters III: GovernanceIndicators for 1996, 1998, 2000, and2002.” World Bank Economic Review18: 253-287.

Kaufmann, Daniel, Aart Kraay, andMassimo Mastruzzi. 2005.“Governance Matters IV: GovernanceIndicators for 1996-2004.” The WorldBank. Retrieved December 8, 2005,from: www.worldbank.org/wbi/gover-nance/pubs/govmatters4.html

Kaufmann, Daniel, Aart Kraay, andPablo Zoido-Lobaton. 1999.“Aggregating Governance Indicators.”World Bank Policy Research WorkingPaper No. 2195, Washington, D.C.Retrieved December 8, 2005, from:http://www.worldbank.org/wbi/gover-nance/pubs/aggindicators.html

Notes

1 Several of the polities included inthe database, such as the CookIslands, the Virgin Islands, Reunion,and the West Bank, are not strictlysovereign states. In the rest of thereview I use the word “countries” torefer to all the polities included in thedatabase.

2 This list of sources may raise ques-tions about possible ideological bias-es, in the sense that some of themmay assess the governance situationmore favorably in countries with, for

Datasets

Europe and East Asia than of its owndecline in absolute terms.

The authors of the accompanyingpaper are modest about the accuracyand usefulness of this important andconstantly improving database: “Whilethese aggregate governance indica-tors have been useful in providing ageneral snapshot of the countries ofthe world for various broad compo-nents of governance, now for 8 years,and while the margins of error havedeclined over time, they remain arather blunt instrument for specificpolicy advice at the country level. Aswe have argued in the past, theseaggregate indicators need to be com-plemented with in-depth in-countrygovernance diagnostics, based onmicro-surveys of households, firmsand public officials within the coun-tries” (Kaufmann, 2005, 41). Noticethat this caveat does not necessarilyapply to aggregate analysis: even ifthey carry relatively large standarderrors, the governance scores canstill yield unbiased (if inefficient) sta-tistical estimates when they are usedas dependent variables in regressionanalysis. As independent variables,however, these not-too high levels ofprecision will result in important atten-uation biases.

The World Bank’s GovernanceIndicators are a comprehensive,source-rich, dynamic, and method-ologically sophisticated source of datafor some of the most important vari-ables that comparativists and otherpolitical scientists study. The project’son-line information and data accessfunctions are clear, useful, and easilyaccessible, as are the accompanyingmethodological and substantivepapers. The methods and sourcesare transparent, and the authors arecautious in highlighting the limitationsof the data.

Given the human, organizational, andfinancial resources of the World Bank,

example, conservative administra-tions. The authors conducted a test todetect such bias, and concluded thatthere is not evidence of it for anysource except the HeritageFoundation, which is the “only sourcethat appears to have a consistent ide-ological bias,” although “fairly modestin magnitude” (Kaufmann, Kraay, andMastruzzi 2005, 22-3).

3 Details on the statistical procedureare provided in the appendix ofKaufmann, Kraay, and Mastruzzi(2005) and in Kaufmann, Kraay, andZoido-Lobaton (1999). TheUnobserved Components Model isbasically a weighted average of theobserved scores from each source foreach country (re-scaled to commonunits) in which weights are proportion-al to the precision of each datasource. Precision, in turn, depends onthe strength of the correlation amongthe sources.

4 It is important to note that the impre-cision of the estimates does not arisefrom their being based on subjectivemeasures but from the fact that alter-native measures provide differentestimates of the underlying dimen-sion. The more sources an estimateis based on and the more precisethese sources are, the better the esti-mate’s precision.

21APSA-CP Vol 17, No. 1

Recent writings concerning the meas-urement of political democracy offersophisticated discussions of the prob-lems of conceptualization, opera-tionalization, and aggregation. Yetthey have less to say about the errorthat derives from the use of inaccu-rate, partial, or misleading datasources. Drawing on evidence fromfive Central American countries, weshow that this data-induced measure-ment error compromises the validityof the principal, long-term cross-national scales of democracy(Comparative Political Studies,October 2005).

In response, we call for an approachto index construction that relies oncase expertise and the use of a widerange of data sources. We employthis approach in developing a newindex of political democracy for theCentral American countries during the20th century. The index draws on acomprehensive set of secondary andprimary sources even as it rigorouslypursues standards of conceptualiza-tion, operationalization, and aggrega-tion. The raw data for each country,the BLM democracy index, and thescores for each of 5 dimensions ofdemocracy are available for downloadat http://blmdemocracy.gatech.edu/.

Our index is dynamic in the sensethat new data or sources may refineour coding of individual dimensionsfor individual years, particularly in theearly part of the 20th century.

If you believe that our current rawdataset has errors or shortcomings,please contact the authors.

Kirk BowmanGeorgia Institute of TechnologyKirk.bowman@inta.gatech.edu

Fabrice LehoucqCentro de Investigación y DocenciasEconómicas (CIDE)Fabrice.Lehoucq@cide.edu

James MahoneyNorthwestern Universityjames-mahoney@northwestern.edu

Other Datasets Editors’ Notes

The editors welcome suggestions ofother relatively new and potentiallyuseful datasets that should beannounced or reviewed in APSA-CP.Anyone interested in reviewing adataset for the newsletter, along thelines of Carlos Gervasoni’s review ofThe World Bank's GovernanceIndicators dataset, should contactMichael Coppedge atcoppedge.1@nd.edu.

We invite our readers to request hardcopies of back issues (beginning withthe winter 2003 newsletter issue) atthe cost of $1.50 per issue. Theyshould send their request(s) by emailto gonzalez.90@nd.edu.

BLM DemocracyScale for CentralAmerica: 1900-1999

Datasets

APSA-CP Vol 17, No. 122News & Notes

The International Political ScienceAssociation has bestowed the firstFoundation Mattei Dogan Prize forLifetime Achievement in PoliticalScience on Guillermo O'Donnell ofthe University of Notre Dame.O'Donnell will deliver a prize lectureat the IPSA World Congress inFukuoka, Japan, in July. The text ofhis lecture will be published in thesummer 2006 issue of APSA-CP.

Thanks to Program Chair DeborahYashar, it looks as if the ComparativePolitics section will make a large andinteresting contribution to the 2006APSA meeting in Philadelphia. TheProgram Chairs for the 2007 meeting(Deborah Gerber and David Lake) arealready at work planning for thatmeeting and have asked me to get towork soon to name a chair or chairsfor our section. Here is a copy of theirmessage to the Section heads:

We write at this time to ask for yournominations for your section’s panelorganizers. These individuals will beresponsible for organizing your sec-tion’s panels and poster sessions,and collectively will make up themeeting program committee. Dutiesinclude attending a program commit-tee luncheon at the 2006 AnnualMeeting, writing the section’s call forpapers, reviewing and selecting elec-tronically submitted paper and panelproposals, collaborating via phoneand email with other program commit-tee members to create co-sponsoredpanels, securing discussants andpanels chairs, keeping track of accep-tances and rejections, and assistingAPSA staff in finalizing the programlistings. Each organized section isasked to nominate one or two individ-uals to organize their section’s pan-els. Nominees should be people withexcellent professional judgment whoare well connected in their subfield,and who are reliable and well organ-ized. We strongly encourage nomi-nees who are members of under-rep-resented groups or are from under-represented institutions. I would bemost happy to get your suggestionsfor this important (and challenging!)assignment. Sid Tarrow.

InternationalPolitical ScienceAssociation Prize

Sidney Tarrow onthe 2006-2007APSA programs

MichaelWallerstein, 1951-2006Political Science and ComparativePolitics lost one of its most distin-guished and beloved theorists andpractitioners with the passing away ofMichael Wallerstein on January 7,2006 from brain cancer. Wallersteinserved as president of theComparative Politics section from1999 to 2001.

A political economist with enormousempirical range and methodologicalsophistication, Michael Wallersteinfocused his attention on understand-ing relationships among political-eco-nomic institutions, electoral politics,and market outcomes. Some of hisbest-known work explores the socialand economic impact of wage bar-gaining institutions, with an eyetoward understanding the contribu-tions that centralized wage bargainingand social democratic politics canmake both to economic efficiency andsocial equality.

Among other things, Wallerstein’swork showed that powerful laborunions can reduce wage inequalitywithout increasing unemployment,and that centralized unions play animportant role in reducing inequality.Wallerstein also investigated howsocial democracy, such as that foundin the Scandinavian nations, cancounteract the inefficiencies andinequities of a market society. Inaddition to influential single-authoredworks, Wallerstein was also knownfor fruitful long-term collaborations,among others with Adam Przeworskiand Karl Ove Moene. He recentlycompleted two articles, one withMiriam Golden on the impact of glob-alization and occupational change onwage inequality, and a second with

David Austen-Smith, on the impact ofracial divisions in society on politicalsupport for welfare spending.

Wallerstein earned his undergraduatedegree in political science fromStanford University in 1974, and hisPh.D. in political science from theUniversity of Chicago in 1985. Hebegan his teaching career in 1984 atthe University of California at LosAngeles and spent ten years therebefore moving to NorthwesternUniversity in 1994. Wallerstein spentthe next decade at Northwestern,where he served as chair from 1997to 2000. In 2004, he moved to YaleUniversity, where he assumed theCharlotte Marion SadenProfessorship of Political Science. In2005, Wallerstein was elected to theAmerican Academy of Arts andSciences. Other honors that MichaelWallerstein earned over the course ofhis career include the Franklin L.Burdette/Pi Sigma Alpha Award forthe best paper presented at the annu-al political science meetings (1985)and runner-up for the GregoryLuebbert Award in comparative poli-tics (1999).

Over the course of his career,Wallerstein’s work received supportfrom the National ScienceFoundation, the Norwegian ResearchCouncil and the MacArthurFoundation. He was a scientific advi-sor for the Foundation for Researchin Economic and BusinessAdministration in Norway, where hehad been a visiting scholar at boththe University of Oslo and at theInstitute for Social Research duringthe academic year 1989-1990. Since2001 he served as a member of theScientific Committee of the Center forthe Advanced Study in the SocialSciences at the Juan March Institutein Madrid, Spain, where he visitedand taught regularly. In the AmericanPolitical Science Association, he hadserved on the Executive Council and

was also a member of the Task Forceon Graduate Education.

Wallerstein was also an accomplishedcompetitive sailor. While atNorthwestern, he spent many hourson Lake Michigan pursuing excel-lence in sailing as in everything else.A guitar player, he had an abidinglove of music, especially jazz, andespecially John Coltrane.

He was a loving husband and father,survived by his wife, Elizabeth Atlas,an artist, whom he met as a studenton an archaeological dig in Mexico;his son Jonah, a recently graduatedsoftware engineer; and his daughterHannah, a student at WesleyanCollege. He leaves behind a richintellectual legacy in the many gradu-ate students whose scholarship hehelped to shape. Many colleagueswere, in a sense, also his grateful stu-dents. On a personal level, he will beremembered across the country andabroad as an extremely generousscholar who gave freely of his time.He will also be remembered as ascholar whose work was imbued witha deep passion for politics and anabiding desire to use his scholarshipto advance the cause of justice andequality.

Kathleen Thelen, Miriam Golden, andPeter Swenson

The Department of Political Scienceand School of Public Policy atUniversity College London havelaunched a master’s program inDemocracy and Democratization, cur-rently in its first year. The programfocuses on the design and operationof democratic institutions in old andnew democracies. More informationabout the program is available athttp://www.ucl.ac.uk/spp/teaching/msc-democracy-democratisation/ or bycontacting Dr. Sherrill Stroschein ats.stroschein@ucl.ac.uk.

APSA-CP Vol 17, No. 1 23News & Notes

New Master's inDemocracy andDemocratization,University CollegeLondon

APSA-CP NewsletterUniversity of Notre DameDecio Hall, Box “D”Notre Dame, IN 46556USA

Non Profit OrganizationU.S. Postage

PaidNotre Dame Indiana

Permit No.10

In this Newsletter:

Letter from the President 1Space and Comparative Politics, Sidney Tarrow

Symposium: Innovative Methods in Comparative Politics 5Automated Content Analysis of Political Texts using Wordscores, Kenneth Benoit and Michael Laver 6Theory-Guided Process-Tracing in Comparative Politics: Something Old,Something New, Tulia G. Falleti 9Endogeneity and Selection Bias in Comparative Research,Simon Hug 14

Datasets 17Data Set Review: The World Bank’s Governance Indicators (1996-2004),Carlos Gervasoni 17Other datasets 21

News & Notes 22Awards 22Announcements 22

Copyright 2006 American Political Science Association. Published with financial assistancefrom the members of the Organized Section in Comparative Politics of the AmericanPolitical Association and the College of Arts and Letters, the Helen Kellogg Institute forInternational Studies, the Nanovic Institute for European Studies, and the Graduate Schoolof Arts and Sciences of the University of Notre Dame du Lac.

Volume 17, Issue 1 Winter 2006

How to Subscribe

Subscription to the APSA-CP Newsletter isa benefit to members of the OrganizedSection in Comparative Politics of theAmerican Political Science Association. Tojoin the section, check the appropriate boxwhen joining the APSA or renewing yourAssociation membership. Section dues arecurrently $8 annually, with a $2 surchargefor foreign addresses. The printing andmailing of the Newsletter is paid for out ofmember dues. To join APSA, contact:

American Political Science Association1527 New Hampshire Ave., NWWashington, DC 20036, USA

Telephone: (202) 483-2512Facsimile: (202) 483-2657e.mail: membership@apsanet.org

Changes of address for the Newslettertake place automatically when memberschange their address with the APSA.Please do not send change-of-addressinformation to the Newsletter.