in tandem or out of sync? academic economics research and...

IN TANDEM OR OUT OF SYNC? ACADEMIC ECONOMICS RESEARCHAND PUBLIC POLICY MEASURES

LEA-RACHEL KOSNIK∗

This paper investigates whether academic research attention to certain policy-related measures (including gross domestic product, unemployment, and inflation) iscorrelated with empirical measurements of the measures themselves. In other words,when unemployment rises, does research attention to the matter increase? Or doeconomists pursue research (in the short run) relatively uninfluenced by policy shockson the ground? Text analysis implies that economic attention to key policy terms doescorrelate with empirical movements of the terms in most instances; however, the strongerand more consistent correlation is between use of policy terms in the literature anddiscussion of them by the broader public. (JEL A11, H00)

I. INTRODUCTION

Is economics research, as represented by pub-lications in the top general interest journals in thefield, correlated in the short run with public policyconcerns? When the 2007 recession and associ-ated economic crises hit, was there any significantchange in research attention to “gross domesticproduct” (“GDP”), “unemployment,” “inflation,”or “bankruptcy”? Similarly, as societal concernsrelated to broad policy issues such as acid rain,social security, or health care reform waxed andwaned, did the attention of economics researchersto these topics follow similar paths?

This paper employs methodological tech-niques related to text analysis in order toinvestigate the relationship between academicresearch attention and measures of public policyconcerns as they change over time. It assumesthat at least one goal of economics researchis indeed to address public policy. Other goalshave of course been offered, including thedevelopment of sound theoretical models, thesatisfaction of pure intellectual curiosity, andthe approval of one’s peers and prestige at one’sjob. Not every researcher cares about each ofthese goals equally, or even all of them at any

∗The author acknowledges the insight and assistance fromHarald Uhlig, Dan Hamermesh, David Card, David Laband,Robert Tollison, and various seminar participants. All errorscan be attributed to the author.Kosnik: Associate Professor, Department of Economics, Uni-

versity of Missouri-St. Louis, St. Louis, MO 63121.Phone 1-314-516-5564, Fax 1-314-516-5352, [email protected]

particular time, but most would agree that at leastone broad goal of economics research is to speakto public policy concerns. This paper exploresthe short-run relationship between contemporarypolicy issues and economics research attention,at least in the top general interest journals inthe field.

A demand and supply-based theoretical modelis assumed, where events on the ground affectthe demand for policy-based academic research.The question is whether or not the supply ofacademic research is elastic enough to respondin any significant way in the short run whensuch demand shifts. The supply of academicresearch is assumed to be relatively inelastic dueto lags involved in funding opportunities, pref-erence immobilities of researchers themselves,slow professional turnover at most departments,and a host of other factors. But given the factthat economists profess to care about the rele-vance of their research, one could assume that

ABBREVIATIONS

AER: American Economic ReviewE: EconometricaGDP: Gross Domestic ProductGNV: Google’s Ngram Vieweridf: Inverse-Document FrequencyJEL: Journal of Economic LiteratureJEP: Journal of Economic PerspectivesJPE: Journal of Political EconomyQJE: Quarterly Journal of EconomicsRES: Review of Economic StudiesVSM: Vector-Space Model

190Contemporary Economic Policy (ISSN 1465-7287)Vol. 34, No. 1, January 2016, 190–202Online Early publication July 17, 2015

doi:10.1111/coep.12117© 2015 Western Economic Association International

KOSNIK: IN TANDEM OR OUT OF SYNC? 191

there would be attempts to respond to publicpolicy concerns, at least in a limited way. Per-haps increased research attention to contempo-rary policy issues can only make its way intothe conclusions sections or welfare analyses ofbroader research papers, but it would be surpris-ing if the context of the real world failed to filterinto the research attention of academic papers,at least somewhat. Most people are behaviorallyresponsive to the environment around them, andeconomists are no exception. Does their atten-tion to public policy concerns correlate in anyway with short-run changes in the measuresthemselves? It is important to emphasize thatthis paper is not about the impact of economicsresearch, per se, than it is much more simplyabout the attention economists devote to nonsta-tionary public policy concerns.

The data for the analysis involve all full-length monographs published in seven top jour-nals in the field of economics from 1960 to 2010.The corpus of these texts are investigated fortrends in terms related to key policy measure-ments such as “GDP,” “unemployment,” “infla-tion,” and “bankruptcy.” Frequency analyses ofusage of these terms are compared with empiricalmeasurements of the terms themselves. In addi-tion, frequency analyses of these same terms aregathered from other English language-based textsources (in particular, Google’s Ngram Viewer,GNV) to determine if discussion of them in thewider public is also correlated in any way withuse of the terms in the academic literature. Themost striking finding is that academic discussionof many of these terms appears correlated morestrongly with their discussion in the wider pub-lic, than it is with their movements in real time.In other words, academic economists are not iso-lated in their proverbial ivory towers without anywindows onto the wider world. There are win-dows; it is just that the view from these windowsappears to overlook the public, more so than itdoes the marketplace.

II. LITERATURE REVIEW

Investigations into the impact and relevanceof academic research to public policy concernshave a long history. Milton Friedman (1986),Robert Solow (1997), Maureen Cropper (2000),Rebecca Blank (2002), and Victor Fuchs (2002)are just a few of the noteworthy individuals whohave penned short (and sometimes long) missiveson the topic. Conferences and symposiums havebeen convened at respected institutions over the

years to try and determine the impact and policyrelevance of economics and other social sciences.A main conclusion of much of this effort is thateconomists (as well as other social scientists) doseem to care if their research has an impact; therejust seems to be no answers, and no sure ways tomeasure, whether or not it does.1

The goal of this paper is less ambitious thanthis main line of research investigating the pos-sible policy effects of academic research. Noattempt is made here at determining the opti-mality, efficiency, relevance, or even the maincausative factors of academic research; instead,we investigate whether the attention of aca-demic economists is correlated with events asthey change in the world around them. As such,this paper can be considered to be behaviorallymotivated, as it assumes that economists, likemost people, are affected and inspired by theworld around them. Economists may be boxedinto research agendas and funding opportuni-ties that focus on yesterday’s (or even tomor-row’s) problems, but they can still always devoteattention to today’s issues, to try and make theirpapers contemporarily relevant in at least a tan-gential way. The question is, do they? An ear-lier paper (Laband, Shughart, and Tollison 1990)based on data from 1950 to 1988 found thatthey do. Laband, Shughart, and Tollison (1990)investigated two policy measures—inflation andunemployment—and found empirical evidencethat economists’ research attention to these twotopics is Granger-caused by changes in the pol-icy environments of these terms themselves. Thispaper investigates more topics than just inflationand unemployment, and it uses a more completeand updated dataset, but the ultimate conclusioncorroborates the results from Laband, Shughart,and Tollison (1990): The research attention ofeconomists—as measured by publications in aselect group of journals— is affected by the short-run contemporary policy environment.

III. THEORY

The initial assumption from which this paperproceeds is that one goal, qi, of academic eco-nomics research is to address and possiblyimpact public policy concerns of the day. Most

1. There is also a literature on research impact as mea-sured by citation analysis, publication counts, and h-indexscores (Hamermesh 2013; etc.), but this is about the perceivedinfluence of research on other researches—not necessarily onpublic policy or policy concerns.

192 CONTEMPORARY ECONOMIC POLICY

researchers simultaneously pursue other goalsas well (qi ; i= 1,… , n), including developingsound theoretical models that help advance thefield, satisfying pure intellectual curiosity abouttopics that interest them, and working to achievetenure, prestige, and the approval of one’s peers.All of these objectives strain the limited attentionbudget of any individual researcher.

When demand for research attention to cer-tain public policy factors changes—perhaps itincreases due to a shock to the system such asa sudden banking crisis, or decreases due to awithering of societal concern (acid rain?)—howresponsive is the output of research attention tothese changes; in other words, how elastic areresearchers’ supply curves to the goal of publicpolicy attention? Ultimately, it depends on theindividual researcher and a host of factors suchas preferences for such output, available institu-tional resources to devote to such output, and tal-ent and ability in addressing such output, but as anaggregate in the economics profession as a whole,how significant a response in research attentiondo we see as public policy concerns change?Is there an obvious response in altered researchattention, or is the response muted to nonexis-tent as the result of a nearly inelastic researchsupply curve?

Figure 1 focuses on these demand and supplytensions around the goal of addressing public pol-icy concerns, qi. As demand changes, the quantitysupplied will also change, but whether or not bya significant amount depends on the elasticity ofresearchers to (what are assumed exogenous) pol-icy shocks.

IV. DATA

The data for this project constitute 20,321 arti-cles published in the following seven top-tieracademic journals from 1960 to 2010: Ameri-can Economic Review (AER), Econometrica (E),Journal of Economic Literature (JEL), Journal ofEconomic Perspectives (JEP), Journal of Politi-cal Economy (JPE), Quarterly Journal of Eco-nomics (QJE), and Review of Economic Studies(RES). An effort was made to use observationsfrom the top general interest journals in the field,and this list was chosen after considering a num-ber of different rankings, including Engemannand Wall (2009), Kalaitzidakis, Mamuneas, andStengos (2003), and a variety of online listings.Not all journals that publish academic researchare included in this list, obviously, nor are thereany specific field journals. This should not be a

FIGURE 1Theoretical Model

handicap, as it is common in the literature to focuson a select few top journals in the field and makebroader conclusions about a discipline from there(Card and DellaVigna 2013; Hamermesh 2013;Laband and Tollison 2000; Laband, Tollison, andKarahan 2002). Often this is assumed appropri-ate because it is the top journals that set the trendsand research priorities of the lower-ranked jour-nals. The journals utilized in this work are inclu-sive of the journals most often used in work of thekind that attempts to determine trends, patterns,and influence of academic economics researchmore broadly (Figure 1).2

It may be worth mentioning that the listincludes both journals that publish refereed(AER, E, JPE, QJE, RES) and nonrefereed(JEP, JEL) articles. The distinction does notseem important in this context when the goal isinvestigating broad research attention. What ismore important is reputation as a top, well-readjournal in the field, and the chosen journals are allcertainly that. At the end of the empirical section,disaggregated results by journal type (refereedand nonrefereed) are also presented; so if thereis a concern about a distinction between journaltypes, the reader can glean any differences thatmight appear in the results which follow.

All of the articles published in these journalsfor the years 1960–2010 are in the database.

2. It should be noted that the supply curve in Figure 1,therefore, more accurately measures the supply of academicresearch from these particular outlets, and not all of eco-nomics research.


TABLE 1Article Counts per Decade

Journal 1960s 1970s 1980s 1990s 2000s Totals

American Economic Review 693 1,184 1,194 888 1,092 5,051Econometrica 896 1,105 841 570 671 4,083Journal of Economic Literature 14 180 142 201 216 753Journal of Economic Perspectives 0 0 144 615 620 1,379Journal of Political Economy 1,271 1,181 814 563 460 4,289Quarterly Journal of Economics 496 539 564 462 457 2,518Review of Economic Studies 315 534 525 394 480 2,248Totals 3,685 4,723 4,224 3,693 3,996 20,321

The corpus includes everything research orientedthat has been published in English,3 includ-ing full-length monographs, full-length bookreviews,4 and comments and replies. Entriesnot included in the dataset include editor’snotes, conference announcements and programs,auditor’s reports, indexes, and other similarnonresearch-focused entries. Special symposiumarticles are included.5 Given these criteria thecorpus includes 20,321 articles, some descriptiveinformation for which can be found in Table 1.

V. METHODOLOGY

This paper utilizes textual analysis for itsprimary results. Textual analysis (sometimesalso called “content analysis” or “computationallinguistics”) involves the accumulation of largeamounts of text (in this case, academic researcharticles), cleaning and parsing the text withunique algorithms, and then turning the textinto a database where the words themselves arestatistically analyzed for trends and correlativepatterns. Textual analysis as a methodologicaltool has taken off in the last decade in manysocial science disciplines (most notably politicalscience and psychology), and it has begun tobe utilized in the economics literature as well(Antweiler and Frank 2004; Baker et al. 2014;Gentzkow and Shapiro 2010; Kosnik 2014, 2015;Tetlock 2007).

The unstructured text from each research arti-cle is organized within a vector-space model

3. Some of these journals, especially in earlier years,included the occasional article in French or German.

4. It is worth noting that short book reviews and indexes,that appear primarily in the Journal of Economic Literature(JEL), are not included.

5. It is worth noting, however, that the American Eco-nomic Review’s annual Papers and Proceedings issue is notincluded.

(VSM). In the VSM, each element of the vec-tor indicates the occurrence of a word within thedocument. A collection of documents results ina collection of vectors; 20,321 to be exact in thisstudy.

There is some debate as to whether the ele-ments of the vectors should be transformed inany way, perhaps turned into logs of frequencyof use in order to tamp down the raw frequen-cies. Another option is to weight the elements insome way, such as through an inverse-documentfrequency (idf) transformation.6 In this paper,we have chosen to leave the elements as raw,unweighted counts of frequency of use. This isbecause we want single occurrences of terms tocount, and we want multiple occurrences of termsto count for relatively more, as a representationof greater attention and focus. All of the fol-lowing results, therefore, are based on raw termfrequency analysis.

In the following section we compare fre-quency analyses of specific terms in the researcharticles corpus with various empirical measuresof the terms themselves, but also with compa-rable frequency analyses of the terms in GNV.7

The GNV is based on the unstructured textof millions of books that Google has scannedand digitized as part of their Google BooksLibrary Project. This project includes books fromthe year 1500 to 2008, in multiple languagesfrom around the world. Keyword and phrase fre-quency searches can be done on cuts of the avail-able text, for example over specific years, bycertain languages, and with or without smoothing

6. An inverse-document frequency (idf) transformationreflects the frequency of a term within a document, but alsoacross all the documents within a corpus. It often works tolower the frequency weight of a word if it is common acrossthe entire corpus, under the assumption that it is thus not avery unique or informationally important word, such as “the.”

7. Google’s Ngram Viewer can be found here: https://books.google.com/ngrams/info.

https://books.google.com/ngrams/info

https://books.google.com/ngrams/info


techniques. Searching English-language basedtext allows even more choice, including focus-ing on texts published in Great Britain, or theUnited States, or on those that are fiction ornonfiction.

For this research we focused on Google’sAmerican English 2012 corpus. This is the mostup-to-date corpus currently available, and itincludes books written in English and predom-inantly published in the United States. Textsearches were also conducted on the more inclu-sive English 2012 corpus (which includes bookswritten in English but published anywhere); inall cases the results were nearly identical.

VI. RESULTS

A. Gross Domestic Product

We begin with an analysis of GDP. A compari-son is made among frequency rates of the 1-gram“GDP” in the research articles corpus over time(a), the measurement of real GDP per capita overtime (b), and the percentage changes in real GDPover time (c). A visually informative graphicalrepresentation comparing these three rates can befound in the Appendix, Figure A1.

The Pearson’s correlation coefficient of seriesa and b is high, ra,b = 0.899, indicating thesimultaneous overall upward trend in both mea-surements. At the same time, research attentionto GDP is certainly uncorrelated with percentchanges in its movement, as ra,c =−0.219.

These correlations were also tested with lagson the GDP measurements of 1–5 years (i.e.ra,b−1, ra,c−1, ra,b−2, ra,c−2, … , ra,b−5, ra,c−5),in order to test the possibility that publishedresearch may respond to policy measures, butwith a lag due to the publication process. Noneof these lagged rs improves upon the nonlaggedresults reported above.8

Next we compare the frequency rate of “GDP”in the articles corpus (a) to the frequency rateof the 1-gram “GDP” in the larger English-language literature over time, as represented bythe GNV (Michel et al. 2011) (d). Figure A2 inthe Appendix displays these results side by side,and overlaid one on top of the other, which illus-trates quite starkly the strong correlation betweenthe two series. The Pearson’s correlation coeffi-cient between them, ra,d, is 0.856. The frequencyof academic economists’ use of the term “GDP”

8. Numerical results available from the author.

is remarkably similar to its use by the widerEnglish-language speaking public.9,10

B. Unemployment

Next we investigate “unemployment.” Series(a) is again a measurement of frequency rates,this time of the 1-gram “unemployment” in theresearch articles corpus over time. Series (b)is of the U.S. civilian unemployment rate overtime, and series (c) is the yearly change in theannual U.S. civilian unemployment rate overtime. Graphical comparisons of the rates can befound in the Appendix, Figure A3.

This time, measurements of the public pol-icy term appear unrelated to usage of the termin the economics academic literature. The Pear-son’s correlation coefficients are ra,b = 0.206 andra,c =−0.115. Yearly movements and fluctua-tions in the unemployment rate do not seem to beaffecting the amount of academic research atten-tion to the topic of unemployment, at least not inthe short run.

When lagged values are compared, however,we see an improvement in the correlation num-bers, with ra,b−3 in particular jumping to 0.509;ra,c−3, however, remains low at 0.190. In thisinstance, therefore, there is an improvement in thecorrelations if we allow for a lag, which can bejustified due to potentially similar lags in journalpublication times.

In comparing attention to “unemployment”in the academic economics literature to use of“unemployment” in the wider English-languageliterature, we find that they follow similar paths(Figure A4). The Pearson’s correlation coeffi-cient is now 0.612, and whatever seems to bemotivating use of the term in the academics lit-erature, appears to be affecting use of the term inthe wider English-language literature as well.11

C. Inflation

There was a notable increase in researchattention to inflation in the late 1970s and early1980s, and this corresponds quite well withactual jumps in the U.S. inflation rate aroundthis time (Figure A5). The academic researchliterature does appear to be responding in the

9. Again, comparing lagged values of up to 5 years doesnot significantly improve the correlation results.

10. Similar results were found for “GNP,” as well asfor other GDP-related terms such as “economic growth,”“output,” and “economic performance.”

11. Lagging the Ngram Viewer results does not signifi-cantly improve the correlation results.


near term to an important event happening inthe policy world, namely spiking inflation rates.ra,b = 0.469, confirming the correlation, andthe lagged correlation, ra,b−3, is even strongerat 0.540. The correlation between attention toinflation in the academic literature and yearlychanges in its empirical measurement, however,is minimal (ra,c =−0.143 and ra,c−3 = 0.182).

While a correlation exists between academicattention to inflation in the literature, and itsannual empirical measurement, the relationshipbetween academic attention to the topic and itsuse in the wider English-language literature, asbefore, is also strong. Figure A6 displays use ofthe term “inflation” in both the academic cor-pus and the wider English-language corpus, andra,d = 0.521 (which does not improve with anylagged values). In this instance as well, academicattention to certain policy-related terms mirrorsquite strongly attention to those terms in thewider English-language literature.

D. Bankruptcy

Finally, we investigate “bankruptcy.” The firstthing to note is that data on bankruptcy ratesin the United States before 1980 were unavail-able, so our comparison is limited to a three-decade time span, instead of a five-decade one.The second thing to note is that both of theempirical measurements of bankruptcy rates inthe United States (series (b) and (c)) are relativelyuncorrelated with attention to the term in theacademic literature, ra,b = 0.119 and ra,c = 0.383(ra,b−3 = 0.140 and ra,c−3 =−0.081) (Figure A7in the Appendix). However, as has happenedwith all of the previous terms investigated, thecorrelation between use of the term in the aca-demic literature with use of the term in the widerEnglish-language literature is strong, in this casewith a ra,d = 0.530 (Figure A8 in the Appendix).

E. Acid Rain, Social Security, Climate Change,and Health Care Reform

The results so far indicate that academiceconomists’ attention to certain public policy-related measures is often correlated withempirical measurements of the measures them-selves, particularly if we include measurementslagged by 3 years. It is likely significant that it isa lag of 3 years (and not 1, 2, 4, or 5 years) thatoften produces the optimal correlation, as thiscorresponds reasonably well to the lag involvedin journal publication times. Other terms werealso tested (including “interest rate,” “recession,”

“net exports,” and “budget deficit”), though notpresented here for brevity’s sake, and similarresults held.

In addition, a high correlation betweenthe terms and their use in the wider English-language literature always held, and often evenmore strongly. This close relationship betweenacademics’ use of key terms and their use bythe wider public (unlagged) is intriguing. It isinvestigated further with tests of the correlationbetween other topical public policy issues thatdo not necessarily have precise comparableempirical measurements of their terms, and theiruse in the two corpora. Figure A9 comparesfrequency rates of “acid rain,” “social security,”“climate change,” and “health care reform”from the research articles corpus to their fre-quencies in the GNV. The correlations are allhigh, with ra = 0.800, rb = 0.616, rc = 0.538, andrd = 0.666.12,13

It is also noteworthy that the peaks in the fre-quency rates do appear to correspond to the datesof major legislative or regulatory initiatives in theUnited States: the 1990 Clean Air Act for acidrain, President George W. Bush’s social secu-rity reform efforts in 2005, the American CleanEnergy and Security Act (the federal-level cap-and-trade bill) of 2009 for climate change, andthe Health Security Act of 1993 for health carereform. This implies that the usage frequencyrates are likely related to specific policy events onthe ground. Further analysis needs to be done todetermine any sort of causal relationship betweenpolicy and legislative or regulatory change andattention to the topics by academic economists, orfor that matter between discussion of these termsin the wider public (as represented by the GNV)and the attention paid to them in the academic lit-erature, but it appears that there is a relationshipof some form.

F. Refereed versus Nonrefereed Journals

In this section, we investigate if the resultsobtained above differ when analyzing solely ref-ereed or nonrefereed journals. Figures A10 andA11 display the results for the two sets of policy

12. Note that more specific terms related to these topics(e.g., “ozone layer” with respect to acid rain, and “greenhousegas emissions” with respect to climate change) show similarlyhigh correlations between the two corpora; in other words, thespecific term we use does not seem to be driving the results.The results are broad-based across term use and term topic.

13. For most of the terms studied, lags did not improvethe correlation results; only with climate change did thecorrelation improve slightly (to rc-3 = 0.561).


measures studied. The main result is that there isnot much difference. Visually, the levels of atten-tion follow similar paths with, if anything, thenonrefereed journals (JEL and JEP) slightly lag-ging the refereed journals (particularly in FigureA11). The correlations between the two sets ofresults are always positive and, with lagging,range from 0.249 to 0.970. Overall, the levelof research attention to these policy measuresfollows similar paths in both refereed and non-refereed journals. This may seem surprising assome have suggested that nonrefereed journalsare deliberately edited with an eye to currentevents. This may be true; however, the textualresults presented here seem to imply that refereedjournals are as well.

VII. CONCLUSIONS

From this limited keyword analysis a fewoverall impressions are drawn. First, that eco-nomic discussion of key public policy terms, inthe short run, does move in a correlative waywith (often lagged) empirical measurements ofthe terms themselves.14 This is great news forthe economics profession, as it can be pointed toas evidence that economists are discussing realpolicy concerns in their research in real time.This result also confirms that found in Laband,Shughart, and Tollison (1990).

In addition, the consistency of the lag (3years), when it is relevant, is consistent with aver-age publication lags. Economists are thus activelyaware of policy events around them and seekto publish high quality research articles aboutimportant policy-related topics; it is just that theirapparent attention to them is likely often laggeddue to the publication process. Speedier publi-cation times would seem to imply a greater per-ceived relevance of economics research to policyevents of the day.15

At the same time, economists’ attention topolicy terms correlates with use of the terms inthe wider English-language literature even morestrongly and consistently than it does to corre-lation with measures of the terms themselves.In other words, economists seem to talk aboutimportant policy events as they happen, but even

14. Although in no instances investigated were the per-cent, or yearly, changes correlative.

15. This result also argues for the importance of ResearchPapers in Economics, Social Science Research Network, andother online websites that allow access to scholarly workingpapers—and their results—before final journal publicationtimes.

more so, as everyone else is talking about themas well. The profession seems to be slightly moreattuned to the public square, than it does tothe marketplace.

Considering future research, there is still anopen question as to what motivates researchattention overall. The correlative results herewere always less than one, leaving room for othermotivations to additionally drive research atten-tion, including ideology, passion, funding levelsand so on. It would be of interest to construct amore complete picture as to what drives researchattention, when, and to what degree.

Similarly, it would be of interest to determinewhy certain policy terms receive more attentionthan others, both in the academic research sphereand also in the public sphere. What, ultimately,is motivating all of this attention overall? Is itthat the public (as well as economists) discuss“acid rain,” “social security,” “climate change,”and “health care reform” when federal legislationrelated to those topics becomes imminent, ordoes discussion surrounding those topics jumpstart from media events and newspaper reporting?Which way does the causation run and whatdrives attention overall?

There is also the possibility that the resultspresented here are exclusive to the seven particu-lar journals investigated and not to the wider aca-demic literature. Evidence against such a possi-bility comes most recently from Kelly and Brues-tle (2010), which investigates research trendsacross economics journals and finds that mosttrends are the same whether they are investigatedfrom a select top group of journals (as is donehere), or across all journals publishing economicsresearch. But of course one can never be certainthat our results hold across the entire universe ofarticles published in economics, so future worth-while research should include exhaustive datasets(including corpora from field journals), as well asother sorts of policy topics investigated, to try andunderstand what grabs the attention of academicresearchers across the field. We do know that aca-demic research attention is itself valuable. Judg-ing by salary levels and rates of output, it can beroughly estimated that a single academic researchpaper is worth thousands of dollars in opportunitycosts, both to the researchers themselves and tothe funding university or organization. In order todetermine if academic researchers are allocatingtheir scarce and valuable time and resources well,it is first necessary to discover where they areputting their research attention at all. This paperaids in the discussion.


APPENDIX

FIGURE A1

Measurements of GDP, 1960–2010

0.00E+00

1.00E-04

2.00E-04

3.00E-04

4.00E-04

5.00E-04

6.00E-04(a)

(b) (c)

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“GDP” Frequency Analysis from Articles Database*

0

10000

20000

30000

40000

50000

60000

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Real GDP per capita, Bureau of Labor Statistics, 2011 dollars

-4

-2

0

2

4

6

8

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

% Change in GDP from Preceding Period, Bureau of Economic Analysis, Real dollars

*The vertical axis in graph (a) measures the percent frequency of the 1-gram “GDP” in a 1-gram count of the corpus.

FIGURE A2

Measurements of GDP, 1960–2010

0.00E+00

1.00E-04

2.00E-04

3.00E-04

4.00E-04

5.00E-04

6.00E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“GDP” Frequency Analysis from Articles Database

0.00E+00

2.00E-06

4.00E-06

6.00E-06

8.00E-06

1.00E-05

1.20E-05

1.40E-05

1.60E-05

1.80E-05

1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“GDP” Frequency Analysis from Ngram Viewer

0.00E+00

2.00E-06

4.00E-06

6.00E-06

8.00E-06

1.00E-05

1.20E-05

1.40E-05

1.60E-05

1.80E-05

0.00E+00

1.00E-04

2.00E-04

3.00E-04

4.00E-04

5.00E-04

6.00E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“GDP” Frequency Analysis fromArticles Database & Ngram Viewer

(a) (d)

(e)

*The vertical axes in the graphs measure percent frequency in the relative corpora.


FIGURE A3

Measurements of Unemployment, 1960–2010

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Unemployment” Frequency Analysis from Articles Database*

0

2

4

6

8

10

12

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Annual U.S. Civilian Unemployment Rate U.S. Bureau of Labor Statistics

-3.0

-2.0

-1.0

0.0

1.0

2.0

3.0

4.0

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Yearly Change in Annual U.S. Civilian Unemployment RateU.S. Bureau of Labor Statistics

(a)

(b) (c)

*The vertical axis in graph (a) measures the percent frequency of the 1-gram “unemployment” in a 1-gram count of thecorpus.

FIGURE A4

Measurements of Unemployment, 1960–2010

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Unemployment” Frequency Analysis from Articles Database

0.00E+00

5.00E-06

1.00E-05

1.50E-05

2.00E-05

2.50E-05

3.00E-05

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Unemployment” Frequency Analysis from Ngram Viewer

0.00E+00

5.00E-06

1.00E-05

1.50E-05

2.00E-05

2.50E-05

3.00E-05

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Unemployment” Frequency Analysis from Articles Database & Ngram Viewer

(a) (d)

(e)



FIGURE A5

Measurements of Inflation, 1960–2010

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

2.00E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Inflation” Frequency Analysis from Articles Database*

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Annual U.S. Inflation Rate U.S. Bureau of Labor Statistics

-6.00%

-4.00%

-2.00%

0.00%

2.00%

4.00%

6.00%

8.00%

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Yearly Change in U.S. Inflation Rate U.S. Bureau of Labor Statistics

(a)

(b) (c)

*The vertical axis in graph (a) measures the percent frequency of the 1-gram “inflation” in a 1-gram count of the corpus.

FIGURE A6

Measurements of Inflation, 1960–2010

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

2.00E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Inflation” Frequency Analysis from Articles Database

0.00E+00

5.00E-06

1.00E-05

1.50E-05

2.00E-05

2.50E-05

3.00E-05

3.50E-05

4.00E-05

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Inflation” Frequency Analysis from Ngram Viewer

0.00E+00

5.00E-06

1.00E-05

1.50E-05

2.00E-05

2.50E-05

3.00E-05

3.50E-05

4.00E-05

0.00E+00

2.00E-04

4.00E-04

6.00E-04

8.00E-04

1.00E-03

1.20E-03

1.40E-03

1.60E-03

1.80E-03

2.00E-03

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Inflation” Frequency Analysis from Articles Database & Ngram Viewer

(a) (d)

(e)



FIGURE A7

Measurements of Bankruptcy, 1960–2010

0.00E+00

5.00E-05

1.00E-04

1.50E-04

2.00E-04

2.50E-04

3.00E-04

3.50E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Bankruptcy” Frequency Analysis from Articles Database*

0

500

1000

1500

2000

2500

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Total Business & Non -Business U.S. Bankruptcies (‘000s) American Bankruptcy Institute (ABI)

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

Annual % Change in Business & Non -Business U.S. Bankruptcies American Bankruptcy Institute (ABI)

(a)

(b) (c)

*The vertical axis in graph (a) measures the percent frequency of the 1-gram “bankruptcy” in a 1-gram count of the corpus.

FIGURE A8

Measurements of Bankruptcy, 1960–2010

0.00E+00

5.00E-05

1.00E-04

1.50E-04

2.00E-04

2.50E-04

3.00E-04

3.50E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Bankruptcy” Frequency Analysis from Articles Database

0.00E+00

2.00E-06

4.00E-06

6.00E-06

8.00E-06

1.00E-05

1.20E-05

1.40E-05

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Bankruptcy” Frequency Analysis from Ngram Viewer

0.00E+00

2.00E-06

4.00E-06

6.00E-06

8.00E-06

1.00E-05

1.20E-05

1.40E-05

0.00E+00

5.00E-05

1.00E-04

1.50E-04

2.00E-04

2.50E-04

3.00E-04

3.50E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Bankruptcy” Frequency Analysis from Articles Database & Ngram Viewer

(a) (d)

(e)



FIGURE A9

Other Policy-Related Term Measurements, 1960–2010

0.00E+00

2.00E-07

4.00E-07

6.00E-07

8.00E-07

1.00E-06

1.20E-06

1.40E-06

1.60E-06

1.80E-06

0.00E+00

5.00E-05

1.00E-04

1.50E-04

2.00E-04

2.50E-04

3.00E-04

3.50E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Acid Rain” Frequency Analysis from Articles Database & Ngram Viewer

0

0.000001

0.000002

0.000003

0.000004

0.000005

0.000006

0.000007

0.000008

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

0.0035

0.004

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Social Security” Frequency Analysis from Articles Database & Ngram Viewer

0.00E+00

5.00E-07

1.00E-06

1.50E-06

2.00E-06

2.50E-06

3.00E-06

3.50E-06

0.00E+00

1.00E-04

2.00E-04

3.00E-04

4.00E-04

5.00E-04

6.00E-04

7.00E-04

8.00E-04

9.00E-04

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Climate Change” Frequency Analysis from Articles Database & Ngram Viewer

Articles

Ngram Viewer

0

0.0000002

0.0000004

0.0000006

0.0000008

0.000001

0.0000012

0

0.0001

0.0002

0.0003

0.0004

0.0005

0.0006

0.0007

1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010

“Health Care Reform” Frequency Analysis from Articles Database & Ngram Viewer

ArticlesNgram Viewer

Articles

Ngram Viewer

Articles

Ngram Viewer

(a) (b)

(c) (d)


FIGURE A10

Measurements of GDP, Unemployment, Inflation, Bankruptcy 1960–2010

0

0.001

0.002

0.003

0.004

0.005

0.006

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Inflation

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

0.0016

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Bankruptcy

0

0.0005

0.001

0.0015

0.002

0.0025

0.003

0.0035

0

0.0005

0.001

0.0015

0.002

0.0025

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

00.0010.0020.0030.0040.0050.0060.0070.0080.009

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Unemployment

Refereed

Non-Refereed

Refereed

Non-Refereed

Non-Refereed

Refereed

Non-Refereed

Refereed

GDP(a) (b)

(c) (d)



FIGURE A11

Measurements of Acid Rain, Social Security, Climate Change, Health Care Reform 1960–2010

00.0010.0020.0030.0040.0050.0060.0070.0080.009

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Climate Change

0

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Health Care Reform

0

0.0002

0.0004

0.0006

0.0008

0.001

0.0012

0.0014

0.0016

0

0.0001

0.0002

0.0003

0.0004

0.0005

0.0006

0.0007

0.000819

6019

6319

6619

6919

7219

7519

7819

8119

8419

8719

9019

9319

9619

9920

0220

0520

08

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

1960

1963

1966

1969

1972

1975

1978

1981

1984

1987

1990

1993

1996

1999

2002

2005

2008

Social Security

Refereed

Non-Refereed

Refereed

Non-Refereed

Refereed

Non-Refereed

Non-Refereed

Refereed

Acid Rain(a) (b)

(c) (d)


REFERENCES

Antweiler, W., and M. Z. Frank. “Is All That Talk JustNoise? The Information Content of Internet Stock Mes-sage Boards.” The Journal of Finance, 59(3), 2004,1259–94.

Baker, S. R., N. Bloom, B. Canes-Wrone, S. J. Davis, andJ. Rodden. “Why Has US Policy Uncertainty RisenSince 1960?” American Economic Review: Papers &Proceedings, 104(5), 2014, 56–60.

Blank, R. M. “What Do Economists Have to Contribute toPolicy Decision-Making?” Quarterly Review of Eco-nomics and Finance, 42, 2002, 817–24.

Card, D., and S. DellaVigna. “Nine Facts About Top Journalsin Economics.” Journal of Economic Literature, 51(1),2013, 144–61.

Cropper, M. L. “Has Economic Research Answered theNeeds of Environmental Policy?” Journal of Envi-ronmental Economics and Management, 39, 2000,328–50.

Engemann, K. M., and H. J. Wall. “A Journal Ranking forthe Ambitious Economist.” Federal Reserve Bank of St.Louis Review, 91(3), 2009, 127–39.

Friedman, M. “Economists and Economic Policy.” EconomicInquiry, 24, 1986, 1–10.

Fuchs, V. “Reflections on Economic Research and PublicPolicy John R. Commons Award Lecture, 2002.” TheAmerican Economist, 46(1), 2002, 3–9.

Gentzkow, M., and J. M. Shapiro. “What Drives Media Slant?Evidence from U.S. Daily Newspapers.” Econometrica,78(1), 2010, 35–71.

Hamermesh, D. S. “Six Decades of Top Economics Publish-ing: Who and How?” Journal of Economic Literature,51(1), 2013, 162–72.

Kalaitzidakis, P., T. P. Mamuneas, and T. Stengos. “Rankingsof Academic Journals and Institutions in Economics.”

Journal of the European Economics Association, 1(6),2003, 1346–66.

Kelly, M. A., and S. Bruestle. “Trend of Subjects Publishedin Economics Journals 1969–2007.” Economic Inquiry,49(3), 2010, 658–73.

Kosnik, L.-R. “Determinants of Contract Completeness:An Environmental Regulatory Application.” Inter-national Review of Law and Economics, 37, 2014,198–208.

. “What Have Economists Been Doing for the Last50 Years? A Text Analysis of Published AcademicResearch from 1960–2010.” Economics, 9, 2015, 1–38.

Laband, D. N., and R. D. Tollison. “Intellectual Collabo-ration.” Journal of Political Economy, 108(3), 2000,632–62.

Laband, D. N., W. F. Shughart II, and R. D. Tollison.“Economists and the Economy.” Review of Economicsand Statistics, 72(4), 1990, 707–11.

Laband, D. N., R. D. Tollison, and G. Karahan. “Qual-ity Control in Economics.” Kyklos, 55, 2002,315–34.

Michel, J.-B., Y. K. Shen, A. P. Aiden, A. Veres, M. K. Gray,The Google Books Team, J. P. Pickett, D. Hoiberg, D.Clancy, P. Norvig, J. Orwant, S. Pinker, M. A. Nowak,and E. L. Aiden. “Quantitative Analysis of CultureUsing Millions of Digitized Books.” Science, 331, 2011,176–82.

Solow, R. M. “It Ain’t the Things You Don’t Know ThatHurt You, It’s the Things You Know That Ain’t So.”American Economic Review, 87(2), 1997, 107–8.

Tetlock, P. C. “Giving Content to Investor Sentiment: TheRole of Media in the Stock Market.” Journal of Finance,62(3), 2007, 1139–68.

in tandem or out of sync? academic economics research and...

Documents