a bibliometric analysis of the journal...

A BIBLIOMETRIC ANALYSIS OF THE JOURNAL

ECONOMETRICA

A study submitted in partial fulfillment

of the requirements for the degree of

MSc Information Systems

at

THE UNIVERSITY OF SHEFFIELD

by

OLHA RODZEN

September 2013

2

Abstract

Background: Bibliometric analysis is used for assessment of contribution to the

particular journal by the scientists. It shows impact of each country or research

institution on the journal. In addition, bibliometric studies can explain publishing and

citing trends that is presented in the journal. Citation analysis provides information

about connection of the different disciplines with each other.

Aims: The study aimed to carry out bibliometric analysis of the journal Econometrica

and to investigate knowledge diffusion from this journal to others fields of sciences

and social sciences.

Methods: A data for the research was collected from Web of Science Core Collection

database. Based on that data bibliometric analysis was carried out and analysis for

main figures was provided. In addition, data about citation for specific publication

was retrieved from WoS and connection with other research areas was provided.

Results: The results of the study can be divided into two parts. First, bibliometric and

citation analysis of the journal Econometrica is provided. Bibliometric analysis

includes year wise distribution of publication with explanation of some critical values,

geographical distribution of contributors with outlining of recent trends and

organization distribution of publication. In addition, citation analysis of the journal

presents citation distribution. Second part of the study is focused on citation analysis

of specific highly cited publications in the journal Econometrica.

Conclusions: It is concluded that the journal Econometrica is international journal

with high impact on other research areas. Number of publications in the journal in

more than 20 years is stable, however some fluctuations were presented in 1965-

1974. The biggest contribution to the journal is made by authors from the USA and

UK. The biggest contributors to the journal is organizations which is situated in the

USA. Journal Econometrica is highly cited journal. It is cited by the journals from the

same research areas as well as from the different research areas.

3

Acknowledgements

I would like to thank my supervisor, Peter Willett, for his help and support throughout

the completion of this project.

I would like to thank my family for their encouragement and support.

4

Contents

Abstract ........................................................................................................................ 2

Acknowledgements ...................................................................................................... 3

Contents ....................................................................................................................... 4

List of Tables ................................................................................................................. 6

List of Figures ............................................................................................................... 7

1 Introduction .......................................................................................................... 9

1.1 Background context....................................................................................... 9

1.2 Journal description ...................................................................................... 10

1.3 Aim and objectives ...................................................................................... 11

1.4 Project overview .......................................................................................... 12

2 Literature review ................................................................................................ 13

2.1. Bibliometric studies of single journal .............................................................. 13

2.2 Citation analysis ................................................................................................ 15

2.3 Knowledge diffusion ......................................................................................... 18

3 Methodology ........................................................................................................... 20

3.1 Approach .......................................................................................................... 20

3.2 Justifying the use of WoS citation index .......................................................... 21

3.3 Data collection process .................................................................................... 22

3.4 Limitations of available data ............................................................................ 22

4 Results and Discussion: Publication and Citation analysis ................................. 24

4.1 Introduction ...................................................................................................... 24

4.2 Year wise distribution of publications in journal Econometrica ...................... 24

4.3 Geographical distribution of contributors in journal Econometrica ................ 27

4.4 Organisational distribution of contributors in journal Econometrica .............. 31

4.5 Citations distribution of publication in journal Econometrica ......................... 32

Contents

5

4.6 Citation distribution of source titles and research areas of journal

Econometrica .......................................................................................................... 36

5 Results and Discussion: Citation patterns of publications ...................................... 40

5.1 Introduction ...................................................................................................... 40

5.2 Citation pattern of most cited publication in journal Econometrica ............... 40

5.3 Citation pattern of oldest most cited publication in journal Econometrica .... 43

5.4 Citation pattern of newest most cited publication in journal Econometrica ... 46

5.4 Citation pattern of most cited publication in journal Econometrica that was

published in XXI century ......................................................................................... 48

5.5 Comparison of citation patterns ...................................................................... 50

6 Conclusions ............................................................................................................. 52

6.1 Key findings ...................................................................................................... 52

6.2 Meeting the aim and objectives ....................................................................... 54

6.3 Recommendation for further research ............................................................ 55

7 Bibliography ............................................................................................................ 57

6

List of Tables

Table 1: Types of published items in journal Econometrica

in 1964-1974

26

Table 2: Geographical distribution of top 10 main contributors

in journal Econometrica in 1933-2014

29

Table 3: Geographical distribution of contributors in journal

Econometrica in 1999-2013

30

Table 4: Most cited articles in journal Econometrica in 1933-

2014

35

Table 5: Journals most frequently citing journal Econometrica 37

Table 6: Research areas of the journals most frequently citing

journal Econometrica

38

7

List of Figures

Figure 1: Distribution of publications in journal Econometrica

in 1933-2014

25

Figure 2: Types of published items in journal Econometrica in

1964-1974

27

Figure 3: Geographical distribution of contributors in journal


28

Figure 4: Zipf distribution of contributors in journal


28

Figure 5: Organisational distribution of publications in journal


32

Figure 6: Citations distribution of publications in journal


33

Figure 7: Zipf distribution of citations of publications in journal


34

Figure 8: Citations distribution of publications in journal


36

Figure 9: Importers of knowledge from journal Econometrica 39

Figure 10: Citation over time to the article by Kahneman and

Tversky

41

Figure 11: Research areas most frequently cited Kahneman and

Tversky Prospect theory - analysis of decision under

risk

42

Figure 12: Citation over time to the article by Nash 44

Figure 13: Research areas most frequently cited Nash The

bargaining problem

45

Contents

8

Figure 14: Citation over time to the article by Johansen 47

Figure 15: Research areas most frequently cited Johansen

Estimation and hypothesis testing of cointegration

vectors in Gaussian vector autoregressive models

47

Figure 16: Citation over time to the article by Melitz 48

Figure 17: Research areas most frequently cited Melitz The

impact of trade on intra-industry reallocations and

aggregate industry productivity

49

Figure 18: Citation over time to the article by Kahneman and

Tversky, Nash, Johansen, Melitz

51

9

1 Introduction

1.1 Background context

Scholarly communication has a considerable impact on science and social science

development in the world. Exchange of ideas is mainly introduced by citation of

previous publication. This can be considered as connections of different opinions to

produce additional value for the scientific researches. Garfield as cited in Noruzi

(2005) identifies citations or “association-of-ideas” as “formal, explicit linkages

between papers that have particular points in common.”

Scientists use citation analysis, one of bibliometric techniques, for evaluation

productivity and importance of paper (Narin, 1976, p. 1). Therefore, it has a wider

area of implication, for example identification scientist of which country cited

particular journal (Borkenhagen, Decker, Brahler, Strauss, 2002), journals in which

particular paper is cited (Borkenhagen, Decker, Brahler, Strauss, 2002; Torricella-

Morales, Van Hooydonk, Araujo-Ruiz, 2000) and who is the author of cited

publication (Torricella-Morales, Van Hooydonk, Araujo-Ruiz, 2000).

Citation analysis is used by scientists to determine the impact of discipline to the

other research areas of science and social science (Lowry, Karuga, Richardson, 2007).

Scientists cites publication of other authors when it is important to their work and

has influence on it (Loebbecke, Leidner, 2012). In this study, an attempt to determine

influence of econometric studies on other research areas is made. This shows that

econometric studies is used by other fields of science and social science.

In addition, there are others methods, which helps to evaluate importance of the

research. Bibliometrics is used for evaluation of contribution to the particular field of

research by the authors of different nations (Damrosch, 2006; McElwee, Atherton,

1: Introduction

10

2005; Elster, Chen, Jenkins, 2004) and main topics which is relevant for this field

(McElwee, Atherton, 2005).

However, current bibliometric researches do not cover all field of science and social

science. Econometrics is one of the fields where few bibliometric researches were

carried out. Therefore, in this research will be analysed main trends in econometric

research and their influence on science and social science in general. This study will

be carried out based on journal Econometrica.

As was mentioned earlier, any bibliometric researches connected with econometrics

was not find. However, there are variety of bibliometric researches, which connected

with other disciplines. Methods from this scientific works can be implied on

econometric journal. Data for the research will be retrieved from Web of Science

databases.

1.2 Journal description

The journal Econometrica was created in 1933. Information about editors of the

journal Econometrica is displayed on the official web site. There have been twelve

editors of the journal since 1933.

The first editor of the paper was Norwegian economist Ragnar Frisch and he took

this post from the 1933 to 1954. He and Jan Tinbergen were first winners of Nobel

Prize in economics “for having developed and applied dynamic models for the

analysis of economic processes” (The Concise Encyclopedia of Economics, 2008). In

1953-1954, co-editor with Ragnar Frisch was American economist Robert H. Strotz.

Eventually, he become an editor of Econometrica in 1955-1968. Next editor of the

journal Econometrica became American economist Franklin M. Fisher. He took this

post in 1968-1977 and was replaced by another American economist Hugo

Sonnenschein who was an editor for 1977-1984. British economist Angus Deaton was

editor of journal Econometrica in 1984-1988. In 1988-1992 post of Econometrica

editor belongs to Spanish economist Andreu Mas-Colell. French economist Guy

Laroque was an editor in 1992-1996. He was replaced by American economist Drew

Fudenberg in 1996 and he took this position to 2000. Econometrica editor in 2000-

1: Introduction

11

2003 was Glenn Ellison and in 2003-2007 was Eddie Dekel. He was replaced by

Stephen Morris and he took this post to 2011. Turkish-American economist Daron

Acemoglu is the Econometrica editor since 2011.

Econometrica publishes theoretical and empirical articles in all economics fields. The

goal of the series is defined on the web site of journal: “to promote the publication

of high-quality research works in the fields of Economic Theory, Econometrics, and

Quantitative Economics more generally”. Econometrica is published bi-monthly and

there are more than 81 available volumes of journal nowadays. Econometrica is

published in English language, however first issues contains material on French

language.

Econometrica is international journal published in English language. Most of the

Econometrica editors were representatives of USA, however first editor was

Norwegian economist and nowadays it has international editor.

1.3 Aim and objectives

Aim

To make bibliometric analysis of journal Econometrica and investigate export

relationship between econometrics and other research areas based on this journal.

Objectives

• To carry out literature review presenting previous bibliometric analysis of

different journals and their impact on different scientific areas.

• To outline year wise distribution of publications in the journal Econometrica.

• To identify geographical distribution of contributors to the journal

Econometrica.

• To identify main organisations contributors to the journal Econometrica.

• To describe citation trends of the journal Econometrica.

1: Introduction

12

• To explore which disciplines and journals export knowledge from the journal

Econometrica.

• To identify most cited articles in the journal Econometrica.

• To explore export relationship of the journal Econometrica by the most cited

publications.

1.4 Project overview

The rest of dissertation is divided into five main sections. The next section presents a

literature review. This section includes presentation of researches that is connected

to bibliometric studies of single journal, citation analysis and knowledge diffusion

between disciplines.

Next part describes methodology that was used for conducting this study. It includes

description of the approaches for the research, justification of usage Web of Science

citation index, the process of data collection and main limitations of the research.

Than results of bibliometric analysis of the journal Econometrica is displayed. This

results include year wise distribution of publication, geographical and organization

distribution. In addition, citation distribution for the journal Econometrica is

presented. Next chapter includes results and discussion for the citation patterns of

most cited articles in the journal Econometrica. All presented data in tables and

graphs is discussed and analysed.

The final chapter presents main findings of the research. Meeting the aim and

objectives is discussed and recommendation for further research is presented.

13

2 Literature review

2.1. Bibliometric studies of single journal

A number of bibliometric studies was carried out on single journal. Scientist use this

technique for a different proposes. Anyi, Anuar and Zainab (2009) reviewed 82

bibliometric studies to reflect different approaches to bibliometric analysis, however

in general all bibliometric studies on single journal “creates a portrait of the journal”

and “indicate the quality, maturity and productivity of the journal”. Bibliometric

studies are carried out on different journals. These includes library and information

science journals as well as science and technology, medical and health sciences, arts,

humanities and social sciences journals (Anyi, Anuar and Zainab, 2009).

Anyi, Anuar and Zainab (2009) concluded that the major input into bibliometric

studies is done by Asian and African countries. However, the contribution of scientists

from the United States remain quite high. In addition, researchers who study journals

use following bibliometric measures article productivity, author characteristics,

author’s productivity, co-authorship pattern, content analysis, citation analysis and

characteristic of the editorial board.

For example, Jena (2006) demonstrated distribution of numbers of articles published

in Indian Journal of Fibre and Textile Research and their length from 1996 to 2004.

Author concluded that number of articles shows increasing trend from the year 1996

and average length of article is 6.45 pages. Similar study was produced based on

journal Economic botany by Biswas, Roy, Sen (2007). However, authors did not find

any trend in number of articles published in journal from 1994 to 2003 average length

of article is varying from 9 to 13.

In addition, study introduced by Jena (2006) include geographical distributions of

contributors of articles. Even though journal is published in English language it has

2: Literature Review

14

more than 86% contributors among Indian scientists because journal is from India.

Elster, Chen, Jenkins (2004) in the American Journal of Roentgenology, spots similar

trend in geographical distribution. Most articles, which is published in journal,

belongs to American scientist – more than 65% in 2000-2002. Input of American

scientist in Journal of Molecular Graphics and Modelling is lower than in previous

journal, but still significant – “no less than 45.8% of the articles” (Willett, 2007).

Similar trends reported by Jishi and Willett (2010) in Journal of Chemical Information

and Computer Science – significant part of the articles are contribution of United

States scientist.

However, trend of domination authors from one country is not present in all journals.

Borkenhagen, Decker, Brahler, Strauss (2002) shows that number of authors of the

journal Psychotherapy Research from North America and Europe are almost equal.

Same conclusion can be made based on Biswas, Roy, Sen (2007) bibliometric study of

the Economic botany journal.

Different trends in authorship pattern can be determined in journals. Ravichandra

Rao and Bhusan (2008) observed multiple authorship in the Journal of American

Society for Information Science & Technology (JASIST) and Scientometrics from 1992

to 2004. They show average number of authors in JASIST per paper increased from

1.45 in 1992 to 2.29 in 2004. Similar trend is present in Scientometrics average

number of authors rose from 1.43 in 1992 to 2.35 in 2004. Biswas, Roy, Sen (2007)

outlined that in the journal Economic Botany “the contributions by single author and

small teams comprising two or three authors are dominating the field”. However, in

study carried out by Jena (2006) three authored papers is dominant (more than 30%)

followed by two authored articles (28.6%) and multi-authored (22.88%). Single

authored papers were 14.4%.

In addition, Jishi and Willett (2010) studied most used subject area for indexing

Journal of Chemical Information and Computer Science articles. Researchers outlined

ten main subject areas and find out that most articles are indexed as computer

science and chemistry studies.


15

Literature review of single journal bibliometric studies shows that there are not one

trend for all journals. There are significant difference between journals from different

science and social science areas. Therefore, generalization cannot be made. This

shows that results of bibliometric analysis of journal Econometrica cannot be

extrapolated from previous researches.

2.2 Citation analysis

Science was compared by Bonzi, Snyder (1991) with social enterprise and citing other

papers “fills both a professional and a connectional role” in that enterprise to

maximize output of highly valued papers. MacRoberts and MacRoberts (1989)

suggest that usage of another work is an evidence that a different scientist has an

influence on particular paper. In addition, Urata (1990) suggest that citation is an

award and represents high prestige of a researcher. Citation analysis can be used for

“understanding of the ‘collective’ knowledge production within a field” (Hammarfelt,

2011). Citation analysis is an important tool to study scholarly communications,

therefore number of researches of citation behaviour is constantly growing

(Hellqvist, 2010).

Citation is used for different proposes. Garfield (1965) defines fifteen main reasons

of provision reference citations. Scientists pay homage to pioneers, identify original

publication in which idea was discussed and identify methodology as well as criticize

previous work, correct it and disclaim ideas of others. Citations is used by scientists

as “threads” to link distinct ideas or arguments, which appears in different sources

(Amsterdamska, Leydesdorff, 1989). In addition, citing behaviour of scholars is

varying in different field of science and social science (Hellqvist, 2010). Therefore, it

creates additional difficulties in citation analysis.

Distribution of citation counts for articles is highly skewed (Jishi, Willett, 2010) and

the majority of articles receive few, if any, citations from other researches. However,

Leydesdorff (1987) claims that citation analysis does not often show importance of

the publication for science and just reflects behaviour of researcher. Therefore,

scientist can refer to particular source because of institutional, behavioural, and


16

prestige reasons. In addition, even high quality articles with time become obsolescent

and obsolete, therefore, citation counts dropped off over time (Jishi, Willett, 2010).

Moreover, most scientist agreed that citation counts in general measure individual

contribution to the practice of science (Amsterdamska, Leydesdorff, 1989; Garfield,

1979). Nevertheless, there are a lot of criticism connected with citation analysis and

its ability to measure impact of publication. Garfield (1979) labelled these limitations

as “negative citations, self-citations, and methodological papers”. However, scientist

outlines that negative citations or criticism of the paper has important role in the

process of scientific publication, therefore it is a minor drawback of these technique

(Garfield, 1979).

Another limitation of citation analysis, which is outlined in literature is self-citations

(Bonzi, Snyder, 1991; Garfield, 1979; Lawani, 1982). The term self-citation is used

most commonly for characterization of links between authors of the cited and citing

article and it could be considered as egoism of the researcher in certain cases

(Lawani, 1982). However, Bonzi and Snyder (1991) argued that self-citation might

present “the best and most accurate information on a topic”. In addition, Tagliacozzo

(1977) pointed out that self-citation can be recognized as connection of different

works of one scientist for avoidance of replications and presenting historical

background of the research. Garfield (1979) support this statement and reports that

“a high self-citation count, more often than not, indicates nothing more ominous

than a narrow specialty”. Study conducted by Folly, Hajtman, Nagy and Ruff (1987)

represents that about 30% of all citations with no differences between disciplines are

direct or indirect self-citations, however according to future researches they

concluded “through not their number, but – the effect of self-citations is negligible.”

Final limitation of citation analysis that is proposed by Garfield (1979) connected with

citation counts of methodological papers and assumption that “citation counts

cannot be a valid measure because they favour those who develop research methods

over those who theorize about research findings.” This finding is confirmed by Peritz

(1983) and Folly, Hajtman, Nagy and Ruff (1987) in their studies. However, there is a

consensus among social scientists (Garfield, 1979; Folly, Hajtman, Nagy and Ruff,


17

1987; Peritz, 1983) that even though methodological papers are more cited some of

them are not cited at all.

In addition, Lowry, Karuga and Richardson (2007) outlines that one of the limitation

of citation analysis is biased because citation count depends on time and number of

articles in journal. However, this bias is important in the case when citation counts of

different journals is compared (Lowry, Karuga, Richardson, 2007).

MacRoberts and MacRoberts (1996) identify additional problems connected with

citation analysis such as some citations have no influence, citing is highly biased,

secondary sources is preferred by scientist, informal influences are not cited in

papers, citation rates is varying with disciplines, nationality, time period, size and type

of speciality, some areas of research do not get cited and technical problems

connected with citation count. In addition, Bornmann and Daniel (2008) have found

that citation data are not perfect because of time depend factors, journal depend

factors, article depend factors and availability of publication. This have influence on

citation counts, but citation analysis still used and improved “with advances in

computing capability and the growth of digital libraries and repositories” (Ding, Liu,

Guo, Cronin, 2013).

Literature review of citation analysis shows reasons for usage citation analysis for the

research. In addition, it determines main limitation connected with proposed

bibliometric method. However, this drawbacks have not significant influence on the

results of bibliometric analysis. This happens because negative citations is a part of

scientific process. Effect of self-citations can be neglected because of small influence

on the results and because it represents previous academic portfolio of the

researcher. Methodological paper have impact on future research in both

methodological and theoretical areas, therefore them are more cited. Other

disadvantages of this method more connected with citation behaviour. Some

limitations can be minimized because this study will be based on single journal.


18

2.3 Knowledge diffusion

Different subject areas is open systems. They can receive knowledge from one

discipline and give ideas to another (Urata, 1990). Citations creates “a network of

connections that effectively maps the spread of knowledge from one publication to

another” (Hessey, Willett, 2013). Knowledge diffusion between disciplines is defined

as export-import relationship. This economic term was introduced by Cronin and

Pearson (1990) to measure stability of discipline through import-export ratio of

knowledge. Larivière, Sugimoto and Cronin (2012) suggest that importation of

knowledge takes place when references from other discipline is made and

exportation is present when publication receive citation from other discipline.

Disciplines, which highly rely on import from others, are vulnerable, therefore

exporters of knowledge are robust (Cronin and Davenport, as cited in Cronin and

Pearson, 1990). The result of dividing the value of imports by value of exports can be

described with another economic term – the balance of trade in knowledge (Lockett,

McWilliams, 2005). Therefore, each discipline can have “a positive knowledge flow”

or “a negative knowledge flow” (Yan, Ding, Cronin, Leydesdorff, 2013). However, this

approach is criticized by Bedeian (2005) because high import of knowledge does not

show immaturity of discipline and can be considered as orientation inside the

discipline. Nonetheless, analysis of export-import relationship among the disciplines

can describe the structure of humanities and social sciences (Urata, 1990). In

addition, number of interdisciplinary researches is growing because of complexity of

research problems (Levitt, Thelwall, Oppenheim, 2011) this raise question of effective

evaluation of export-import relationship.

Several studies examined export-import relationship between disciplines based on

articles published in particular journal (Cronin and Pearson, 1990; Lockett,

McWilliams, 2005). However, Stigler (1994) highlight that export-import relationship

in that case also depend on other factors such as geographical distribution of journal,

number of subscribers, size of the journal, the number of issues a year and the age of

the journal.


19

Other concerns, which connected with knowledge diffusion between disciplines, are

outlined by Urata (1990). First, boundaries of academic disciplines are unclear and

this produce problem in classification both cited and citing literature. Second,

researcher can use knowledge of other discipline but does not cite it. This finding

supported by Cronin and Meho (2008) because “common subject interests do not

always or necessarily translate into the expected density of interdisciplinary citation.”

Therefore, clear and unambiguous relationship between disciplines cannot be

established.

Despite the problems associated with evaluation of export-import relationship,

journal analysis of the citation is one of the most commonly used method of their

evaluation. Cronin and Pearson (1990) investigate exportation of knowledge from

information science to other disciplines. They argued that information on available

ideas is transmitted in formal and informal way, but more than 90% of all generated

ideas are never acknowledged by other scientists. However, study by Cronin and

Pearson (1990) shows that high quality publications is cited by other authors. For

example, Cronin and Pearson (1990) find out that 105 analysed publication receives

a total of 1650 citations. However, the most of citations were done by publication in

the tame research area and analysed publications generates only 156 exports.

Stigler (1994) analyzed data about citations of 33 statistical journals including some

econometric journals (Econometrica, The Journal of Econometrics). Scientist

calculated the balance of trade in 1987-1989 of journals and highest exporter of the

ideas in the sample is journal Econometrica. However, according to Yan, Ding, Cronin,

Leydesdorff, (2013) because of acceleration of science and growth of human

knowledge the amount of exported-imported knowledge can increase significantly.

Disciplines establish strong relationship with other disciplines. This helps for them to

solve complex problems, which appears nowadays. However, this increases the

interest to determine flows of knowledge. Export-import relationship shows links

between different subject areas in science. But, evaluation of this links rises some

difficulties. Nonetheless, the most usable method to this evaluation is bibliometric

analysis of the citations in journals with different subject areas.

20

3 Methodology

3.1 Approach

To meet dissertation aim and objectives was required a methodology, which would

be suitable for single journal bibliometric review and evaluation of export-import of

knowledge.

There are wide range of papers, which provide methodological background for the

study. Some of the papers, which were used as methodological background, were

described in Literature review. The methodological background for bibliometric study

is presented by Jena (2006) and Biswas, Roy, Sen (2007). However, these two studies

were not connected with econometrics journal review, they conducted single journal

bibliometric study. Jena (2006) carries out study on nine volumes of Indian Journal of

Fibre and Textile Research published from 1996 to 2004. Author describes year wise

distribution of publication, citation pattern, bibliographical distribution of citations,

authorship pattern, geographical distribution and other bibliometric indicators.

Biswas, Roy, Sen (2007) produce study with similar indicators to describe

performance of journal Economic Botany. However, for this study were chosen

figures as year wise distribution of publication, geographical and organization

distribution of the authors and distribution of citations to the journal Econometrica.

In addition, an attempt for analysis of most cited publications and evaluation of their

contribution to other fields (export of knowledge) in this study.

Stigler (1994) produces the methodological background for evaluation of export-

import of knowledge. Researcher analyzed export-import and balance of trade in

sample of 33 statistic journals including Econometrica and The Journal of

Econometrics. In addition, methodology for knowledge diffusion analysis provided by

Cronin and Meho (2008). They “conducted a large-scale, longitudinal citation analysis

3: Methodology

21

of exports from, and imports to, the literature of IS based on an inclusive population

of journals and conference proceedings in the field.” Cronin and Meho (2008)

describes data collection process and further analysis of it. Previous bibliometric

studies (Cronin and Meho, 2008; Hessey and Willett, 2013) analysed knowledge

exported to other disciplines based on Journal Citation Reports (JCR) subject

categories. This maps information flow within different subjects, therefore provides

approach to a knowledge diffusion evaluation. In addition, Jishi and Willett (2010)

evaluates not only amount of exported knowledge, they produce main reasons for

citing each publication.

Based on this methodological approach research did not include human participants

and bibliometric study was not considered as sensitive topic. Therefore, this was no

risk study.

3.2 Justifying the use of WoS citation index

There are three main citation search services (Jacso, 2005) Web of Science (WoS),

Scopus and Google Scholar. Bergman (2012) states that WoS is the oldest citation

index among this three and it have existed since the 1970s. Scopus and Google

Scholar were launched in the fall of 2004 (Bar-Ilan, 2010).

Google Scholar is free access database that provides general information with Google

Scholar Metrics. However, according to information presented on web-site it covers

only publication published between 2008 and 2012. Therefore, it is not suitable for

bibliometric study of Econometrica, because this journal was published since 1933.

Both WoS and Scopus provide enough tools for retrieving and saving information

from their databases, however WoS was used by many scientist for their researches

(Jacso, 2005).

According to Jacso (2005) WoS is core of “the family of ISI citation index” and includes

“all references cited by papers in the primary (source) documents”. According to

Thomson Reuters web site WoS “provides researchers, administrators, faculty, and

3: Methodology

22

students with quick, powerful access to the bibliographic and citation information

they need to find research data, analyse trends, journals and researchers”.

In current research it was decided to use information retrieved from WoS.

3.3 Data collection process

As was mentioned before data for the research was collected from Web of Science

Core Collection database. Search was conducted for the publication with name

Econometrica. 7058 publications was retrieved. To obtain further data two tools from

the WoS were used.

First, Analyze Results this tool allows retrieve data about authors, editors, funding

agencies and other. In this project, data about publication years, document types,

countries, organizations, and research areas was gathered. In addition, this tool helps

to retrieve information about source titles of the journals that cites the journal

Econometrica.

Second, Create Citation Report tool from the WoS was used to retrieve data for

citation analysis. It allows gathering information about total citations, which was

received by each article. In addition, this tool was used for further analysis of

publications that cites specific article from the journal Econometrica.

3.4 Limitations of available data

Research had some limitations. Most of them connected with data. As was discussed

before, WoS is most appropriate source of data for this research, however, this

citation indexing service does not provide all necessary information.

First, it does not provide full list of information about countries that contribute

publications for the journal. As was mentioned before, there were 7058 publications,

however information about country were available just for 4010 publications.

Second, citation distribution over time for the journal cannot be analysed, because

WoS database provides information about citations as cumulative trend. Therefore,

number of citations is constantly growing.

3: Methodology

23

Finally, WoS databases does not cover all research areas and journals, therefore it

can cause some distortions in the results.

24

4 Results and Discussion: Publication and

Citation analysis

4.1 Introduction

This chapter describes main bibliometric characteristics of journal Econometrica.

It starts from presentation of data about year wise distribution of publication in

Econometrica. It is indicated main trends in number of published items. In addition,

reasons for deviations in publication count is explained.

Then geographical distribution of the authors of the publications in the journals from

1933 to 2014 is outlined. Dynamics for each country in 1999-2013 is discussed.

Then, organisational distribution of contributors is outlined. Next, distribution of

citations is presented. Finally, citation distribution of source titles and research areas

of journal Econometrica is discussed.

4.2 Year wise distribution of publications in journal

Econometrica

Journal Econometrica was firstly published in 1933. Since that time number of

articles, which are published in journal, fluctuated significantly, however, in the last

decade number of published articles did not show dramatic changes. Information

about year wise distribution of articles in journal Econometrica are illustrated on

Figure 1.

In total from the 1933 to beginning of 2014 in the journal Econometrica was issued

7058 publications. In average, it is 86 publications in a year. However, as can be seen

from Fig. 1, more than 8% of the publications were released in 1971 (585

4: Results and Discussion: Publication and Citation analysis

25

publications). For example, in 1970 this figure was 4.6% (327 publications) which is

also high number. Smallest number of articles were published in 1944 – only 16

(0.23% of total amount).

Figure 1: Distribution of publications in journal Econometrica in 1933-2014

There are several possible explanations for this result. For example, from 1941 to

1944 were published only three issues of the journal. However, there was four issues

of the Econometrica in 1933-1943. It can be presumed that reduction of number of

issues in this year is connected with entry of the US in World War II.

From 1944 to 1969, each volume of the journal Econometrica in general consists of

four issues. In addition, as can be seen from Figure 1, number of publications in 1944-

1963 was constantly growing. However, in 1964-1969 the number of publications

fluctuated significantly.

Sharp increase of number of publications started in 1970. It can be presumed that

this happened because each volume of the journal has six issues from the 1970.

However, the most interesting trend is that Econometrica issued 585 publications in

1971. It was more than 8% of total publications in journal. To explain this tendency

data about types of publication was retrieved and presented in Table 1.

0

100

200

300

400

500

600

19

33

19

36

19

39

19

42

19

45

19

48

19

51

19

54

19

57

19

60

19

63

19

66

19

69

19

72

19

75

19

78

19

81

19

84

19

87

19

90

19

93

19

96

19

99

20

02

20

05

20

08

20

11

20

14


26

As can be seen from the table 1 high number of publications occurs when Meeting

abstracts of Book reviews is published in the journal Econometrica. This trend starts

in 1964.

Table 1: Types of published items in journal Econometrica in 1964 - 1974

DOCUMENT TYPES 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974

MEETING ABSTRACT 72 0 156 0 173 0 229 476 0 0 0 BOOK REVIEW 44 44 39 0 0 0 22 31 71 0 0

NOTE 8 15 7 3 4 14 15 20 35 14 13

ARTICLE 33 46 56 29 35 50 59 58 63 73 72 CORRECTION ADDITION 0 2 4 0 5 6 2 0 3 2 2

LETTER 0 0 0 0 0 0 0 0 5 0 0

REVIEW 0 2 0 0 0 0 0 0 0 0 0 ITEM ABOUT AN INDIVIDUAL 0 2 0 0 0 0 0 0 0 0 0

TOTAL 157 111 262 32 217 70 327 585 177 89 87

Meeting abstract in the journal Econometrica is a short publication approximately

two or three pages long. Therefore, this type of publication increases the overall

number of publications, however it does not affect the length of the issue. Influence

of this type of publication can be seen at 1964, 1966, 1968, 1970, and 1971. The

number of published articles increased from 1964 to 1966. After dramatic fall of the

number of published articles in 1966, it shows gradual growth to the 1974. However,

these tendencies did not affect the general number of published items in the journal

Econometrica (Fig. 2).

As can be seen from the Figure 2 there are significant difference in types of published

items in journal Econometrica. However, as was mentioned before, number of

published articles did not changed trends in number of published items significantly.

For example, the biggest amount of meeting abstracts were published in 1970 and

1971. This shows an outcome in number of publications. In 1971 the whole issue (Vol.

39 Issue 3) consisted of meeting abstracts from Second World Congress of the


27

Econometric Society, Tenth Indian Econometric Conference and North American

Regional Conference.

Figure 2: Types of published items in journal Econometrica in 1964 - 1974

The number of publications in journal Econometrica in 1975-1980, however, it leveled

off in 1981 and did not show sharp decreases or increases until now.

4.3 Geographical distribution of contributors in journal

Econometrica

Most publication (6917) in journal Econometrica is in English language, however

some articles was published in French language (149). Last article in French was

published in 1972 and since then Econometrica is published on English. In total

scientist from 40 countries (Fig. 3).

Geographical distribution of contributors to the journal Econometrica is highly

skewed. It can be assumed that distribution follows Zipf law. Therefore, it will be

plotted log-log distribution of the contributors on Figure 4.

0

100

200

300

400

500

600

1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974

ITEM ABOUT AN INDIVIDUAL REVIEW LETTER

CORRECTION ADDITION ARTICLE NOTE

BOOK REVIEW MEETING ABSTRACT


28

Figure 3: Geographical distribution of contributors in journal Econometrica

in 1933 - 2014

*Includes England, Scotland, Ireland and Wales.

**Includes Germany, Federal Republic of Germany and West Germany.

Figure 4: Zipf distribution of contributors in journal Econometrica in 1933 - 2014

0

500

1000

1500

2000

2500U

SA

UK

*

CA

NA

DA

FRA

NC

E

ISR

AEL

JAP

AN

AU

STR

ALI

A

NET

HER

LAN

DS

SPA

IN

BEL

GIU

M

SWIT

ZER

LAN

D

GER

MA

NY*

*

SWED

EN

ITA

LY

IND

IA

DEN

MA

RK

NO

RW

AY

NEW

ZEA

LAN

D

AU

STR

IA

SOU

TH K

OR

EA

PEO

PLE

S R

CH

INA

BR

AZI

L

MEX

ICO

TUR

KEY

SIN

GA

PO

RE

FIN

LAN

D

PO

LAN

D

HU

NG

AR

Y

RU

SSIA

TAIW

AN

PO

RTU

GA

L

CH

ILE

AR

GEN

TIN

A

USS

R

PH

ILIP

PIN

ES

EGYP

T

CZE

CH

OSL

OV

AK

IA

HO

NG

KO

NG

GR

EEC

E

BER

MU

DA

Number of published articles

0

0,2

0,4

0,6

0,8

1

1,2

1,4

1,6

0 1 2 3 4 5 6 7 8 9

LN(C

ou

ntr

y co

un

t)

LN(Number of published articles)


29

As can be seen from the Figure 4 geographical distribution of contributors to the

journal Econometrica cannot be exactly defined as Zipf distribution. The main reason

for this is shortage of data. However, it can be assumed that for the bigger sample it

follows approximately Zipf distribution. Because there will be small amount of

countries with high contribution and higher amount of countries with smaller

contribution to the journal Econometrica.

Data obtained from WoS Core Collection database shows that highest number of

contributors are from the USA. As can be seen from Table 2 scientist from the USA

contributes more than 32% of total number of articles. Scientists from the United

Kingdom is on the second place and contribute nearly 6% of total publications.

Scientists from Canada and France is responsible for 3% and 2.5% respectively.

Table 2: Geographical distribution of top 10 main contributors in journal

Econometrica in 1933 - 2014

Country Number of published articles

% of 7058

USA 2267 32.1 %

UNITED KINGDOM* 403 5.7 %

CANADA 218 3.1 %

FRANCE 181 2.6 %

ISRAEL 130 1.8 %

JAPAN 84 1.2 %

AUSTRALIA 83 1.2 %

NETHERLANDS 64 0.9 %

SPAIN 59 0.8 %

BELGIUM 58 0.8 %

* Includes England, Scotland, Ireland and Wales.

To understand main trends in geographical distribution in recent years it was decided

to merge data and create intervals in 5 years each for each country contributions.

Grows rates for 1999 - 2003 were calculated with data from previous periods, which

is not displayed in a table. Results is presented in Table 3.


30

Table 3: Geographical distribution of contributors in journal Econometrica

in 1999 - 2013

Country

1999-2003 2004-2008 2009-2013

Number Growth rate, %



USA 225 26 205 -9 295 44

UK* 46 77 39 -15 67 72

FRANCE 22 0 17 -23 32 88

ISRAEL 18 125 12 -33 11 -8

CANADA 13 -54 10 -23 29 190

NETHERLANDS 12 50 3 -75 6 100

SPAIN 9 -18 13 44 10 -23

JAPAN 8 33 9 13 8 -11

GERMANY 8 167 15 88 12 -20

DENMARK 7 0 8 14 11 38

BELGIUM 4 -33 4 0 3 -25

AUSTRALIA 4 -20 4 0 0 -100

AUSTRIA 4 33 0 -100 0 0

SOUTH KOREA 4 0 2 -50 7 250

NEW ZEALAND 4 0 7 75 3 -57

SWEDEN 3 50 3 0 6 100

SWITZERLAND 3 0 7 133 15 114

ITALY 2 -50 4 100 11 175

INDIA 2 -33 2 0 0 -100

MEXICO 0 -100 3 0 0 -100

NORWAY 0 -100 0 0 2 0

PEOPLES REPUBLIC CHINA 0 0 7 0 8 14

AUSTRALIA 0 0 0 0 11 0

SINGAPORE 0 0 0 0 5 0

BRAZIL 0 0 0 0 4 0 * Includes England, Scotland, Ireland and Wales.

It can be seen from Table 3 the biggest contribution to the journal Econometrica is

made by the USA scientists. Scientist from the United Kingdom are on the second

place. In addition, high impact on the journal Econometrica has such countries as

France, Israel, Canada, Netherlands and Spain. Therefore, non-English speaking

countries also have an impact on the journal.


31

It is reasonable to look for the dynamics for ten main contributors, because dynamics

for smaller contributors is very volatile. The number of publications for main ten

contributors in 1999 - 2003 comparing for previous period (1994 - 1998) is increased

in general. However, decline shows Canada and Spain. The number of publications

from Canada’s scientists dropped more than a half. This lead to the loss of the

position of Canada’s scientists in journal. However, impact of scientists from

Germany, Israel and UK raised significantly.

In 2004 - 2008 contribution of top six countries is dropped. This leads to the growth

of contribution from other countries. Therefore, authors from Germany, Spain,

Denmark, and Japan increase the impact on journal Econometrica.

In contrast to the previous period, top contributors showed considerable growth of

publications, when scientists from Germany, Spain, and Japan contribute less to the

journal. However, the greatest growth of the publications was shown by the authors

from Canada in 2009 - 2013.

4.4 Organisational distribution of contributors in journal

Econometrica

WoS Core Collection database shows that there are 405 organisations, which

contribute to the journal Econometrica. However, WoS does not provide information

about organisation for all articles. Figure 5 presents organisational distribution of

publications.

Figure 5 shows all organizations which contribution for the journal Econometrica is

more than 100 publication. Organization with smaller contribution is combined into

OTHERS. Nearly 15% of the publications have not information about organization.

As can be seen from the Figure 5 first five organisations, which are biggest

contributors for the journal, is from the USA. Two organisations which contribution

is more than 100 publications are from the United Kingdom. However, the rest is from

the USA.


32

Figure 5: Organisational distribution of publications in journal Econometrica

in 1999 - 2013

*Includes UC Berkeley; UC Davis; UC Irvine; UC Los Angeles; UC Merced; UC Riverside;

UC San Diego; UC San Francisco; UC Santa Barbara; UC Santa Cruz

**Includes 18 individual institutions, e. g. Birkbeck; King's College London; London

Business School; The London School of Economics and Political Science; Queen Mary

University of London; Royal Holloway; St George's; University College London.

4.5 Citations distribution of publication in journal

Econometrica

As was mentioned previously there are 7056 publication in journal Econometrica

from 1933 to July 2014. This publication received 339364 citations.

Distribution of citations is presented on Figure 6. The biggest number of citations

receive publication from Kahneman and Tversky (1979) Prospect theory: An analysis

of decision under risk – 9645. This is 2.8% of the total citations which journal

Econometrica receives since 1933. However, this number is unique for this journal.

UNIVERSITY OF CALIFORNIA SYSTEM*

NORTHWESTERN UNIVERSITY

YALE UNIVERSITY

STANFORD UNIVERSITY

MASSACHUSETTS INSTITUTE OF TECHNOLOGYMITUNIVERSITY OF LONDON**

HARVARD UNIVERSITY

UNIVERSITY OF CHICAGO

PRINCETON UNIVERSITY

UNIVERSITY OF CALIFORNIA BERKELEY

LONDON SCHOOL ECONOMICS POLITICALSCIENCEUNIVERSITY OF PENNSYLVANIA

OTHERS

NOT SPECIFED


33

Next most cited publication receives more than 2000 less citation (7330). This shows

that distribution of citation is highly skewed. In addition, there are 2697 publications

that do not receive any citation. Distribution of the citations, which was received by

the journal Econometrica, is presented on the Figure 6.

Figure 6: Citations distribution of publications in journal Econometrica in 1933-2014

As can be seen from the Figure 6 citations in journal Econometrica follow

approximately Zipf distribution. Therefore, it can be proved by plotting data about

citations on a log-log graph (Fig. 7). Publications, which received zero citations, were

removed from the Figure 7.

Figure 7 proves the assumption that citations, which was received by the journal

Econometrica, can be approximated with Zipf law. In addition, as the figure below

illustrates, there is a small number of highly cited articles and high number of

publication receives few citation.

For example, more than 3% of the publication in Econometrica receive only 2

citations. The percentage of articles which receive 1 citation is 4,6. In addition, 38,2%

publications in journal Econometrica was never cited.

0

500

1000

1500

2000

2500

3000

0 2000 4000 6000 8000 10000

Total citations

Qu

anti

tyo

f ar

ticl

es


34

Figure 7: Zipf distribution of citations of publications in journal Econometrica

in 1933 - 2014

On the other hand, 20 most cited articles is responsible for nearly 20% of all citation

in journal Econometrica.

Twenty most cited articles is presented in Table 4.

Most of the publications presented in Table 4 is methodological papers. Some of the

authors in this publication describes innovative approach to the evaluation of certain

processes. Therefore, they described some tests and methods, which were named

after them. For example, White test, Engle-Granger two-step method, Hausman test,

Dickey-Fuller test, and Chow test.

As can be seen from the table below some articles in journal Econometrica are highly

cited. Some authors repeats several times. For example White and Engle. The oldest

most cited publication is publication by Nash. It was published in 1950. The newest

most cited publication is publication by Johansen, which was published in 1991. There

is one more publication, which was published in 1991, by Nelson. However, it receive

less citation than previous one. Interestingly, the three most cited publication were

published within small interval of time in 1979 - 1980.

0

1

2

3

4

5

6

7

0 2 4 6 8 10

LN(Total Citation)

LN(Q

uan

tity

of

arti

cles

)


35

Table 4: Most cited articles in journal Econometrica in 1933-2014

No. Publication Citations % of total citations

1 Kahneman, D., Tversky, A. (1979). Prospect theory - analysis of decision under risk

9645 2.8

2 White, H. (1980) A heteroskedasticity-consistent covariance-matrix estimator and a direct test for heteroskedasticity

7330 2.2

3 Heckman, J. J. (1979). Sample selection bias as a specification error.

6133 1.8

4 Engle, R. F., & Granger, C. W. (1987). Co-integration and error correction: representation, estimation, and testing.

5892 1.7

5 Engle, R. F. (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation.

4479 1.3

6 Newey, W. K., & West, K. D. (1986). A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix.

3597 1.1

7 Hausman, J. A. (1978). Specification tests in econometrics. 3592 1.1

8 Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators.

2943 0.9

9 Dickey, D. A., & Fuller, W. A. (1981). Likelihood ratio statistics for autoregressive time series with a unit root.

2462 0.7

10 Nash Jr, J. F. (1950). The bargaining problem. 2111 0.6

11 Pratt, J. W. (1964). Risk aversion in the small and in the large.

2090 0.6

12 Sims, C. A. (1980). Macroeconomics and reality. 2065 0.6

13 Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models.

2008 0.6

14 Koenker, R., & Bassett Jr, G. (1978). Regression quantiles. 1966 0.6

15 Perron, P. (1989). The great crash, the oil price shock, and the unit root hypothesis.

1941 0.6

16 Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions.

1925 0.57

17 Hamilton, J. D. (1989). A new approach to the economic analysis of nonstationary time series and the business cycle.

1870 0.6

18 Cox, J. C., Ingersoll Jr, J. E., & Ross, S. A. (1985). A theory of the term structure of interest rates.

1780 0.5

19 White, H. (1982). Maximum likelihood estimation of misspecified models.

1710 0.5

20 Nelson, D. B. (1991). Conditional heteroskedasticity in asset returns: A new approach.

1635 0.5


36

The total citations, which receive journal Econometrica each year, are different.

However, there are trend for growth as can be seen from Figure 8.

Figure 8: Citations distribution of publications in journal Econometrica

in 1933 - 2014

Highest number of citations journal received in 2013. However, this is cumulative

trend and growth of received citations is connected to the growth of the number of

publications.

4.6 Citation distribution of source titles and research areas of

journal Econometrica

Journal Econometrica is one of the most highly cited journal in the field. Journal

impact factor in 2013 was 3.5. Journal Econometrica research areas are Mathematics,

Mathematical Methods in Social Sciences, and Business Economics.

However, Econometrica is most cited by the journals that has connection either for

the econometrics or for the economic studies in general. Table 5 presents ten

journals, which are the largest users of information from journal Econometrica.

0

5000

10000

15000

20000

25000

19

33

19

36

19

39

19

42

19

45

19

48

19

51

19

54

19

57

19

60

19

63

19

66

19

69

19

72

19

75

19

78

19

81

19

84

19

87

19

90

19

93

19

96

19

99

20

02

20

05

20

08

20

11

20

14


37

Table 5: Journals most frequently citing journal Econometrica

Source titles Records

Econometrica 3389

Economics Letters 3242

Applied Economics 2971

Journal of Economic Theory 2774

Journal of Econometrics 2685

American Economic Review 2633

Review of Economics and Statistics 1789

American Journal of Agricultural Economics 1554

Journal of Economic Dynamics Control 1476

Review of Economic Studies 1450

As can be seen from the table above journal Econometrica has high number of self-

citations. However, what is interesting in this data is that number of citations from

Economics Letters is very close to the self-citation level. In addition, field of research

of journal Economics Letters according to WoS is Business Economics. This is similar

to one of the Econometrica research areas. Therefore, strong connection between

these two journals is expected. Next journals presented in the Table 5 also have at

least one similar field of research with Econometrica. Research areas of Journal of

Econometrics and Econometica is similar. However, American Journal of Agricultural

Economics has one different research area – Agriculture. Therefore, it can be

assumed that journal Econometrica has cross-disciplinary connections.

To prove that assumption research areas of the journals that citing Econometrica

were retrieved and presented at the Table 6.

It is clear that most citations journal Econometrica receive from journals with similar

research areas. However, very high number of citation received from journals with

research area Business economics shows that Econometrica has strong influence in

this field. This means that econometrics methods and models are widely used in

economic studies.


38

Citations from the journals, with research areas as mathematics and mathematical

methods in social sciences, shows that there are strong connections and diffusion of

knowledge in this fields.

Table 6: Research areas of the journals most frequently citing journal Econometrica

Research Areas records

Business economics 128482

Mathematics 28840

Mathematical methods in social sciences 20342

Computer science 8472

Operations research management science 8387

Environmental sciences ecology 7844

Engineering 6754

Government law 6333

Public administration 5391

Psychology 5079

However, as can be seen from the Table 6, Econometrica receives number of citations

from journals with different research areas. Data about biggest importers from the

journal is presented on Figure 9.

Data presented on the Figure 9 shows that Econometrica has strong cross-disciplinary

connections with others research areas.

The strongest relationship Econometrica established with journals connected to

Computer science. However, three main journals, which cited Econometrica from this

field, is also connected to Mathematics. Therefore, it can be assumed that

connections of this journals is stronger than data from Figure 9 shows, because it also

can be established through other journals connected to Mathematics.

Operations research management science is another research field from which

Econometrica receives high number of citations. In addition, three most connected

journals to Econometrica from this research area is also indexed as journals

connected to Business economics.


39

Figure 9: Importers of knowledge from journal Econometrica

The most striking result to emerge from the data presented on Figure 9 is that

research area of Environmental sciences ecology is big importer of the knowledge

from Econometrica. However, second and third biggest importers of this field (Journal

of Environmental Economics and Management and Regional Science and Urban

Economics) is also connected to Business economics. Nonetheless, biggest importer

of this field journal Energy Policy has completely different research areas.

Authors from the journal Energy Policy uses methods of econometric modelling and

testing which was introduced in the journal Econometrica. Another interesting

findings is that authors of the journal Energy Policy implement different methods

from the journal to their researches. Some authors cites the journal Econometrica

more than once. Therefore, knowledge from the journal Econometrica can be used

in different research areas for different purposes.

0

2000

4000

6000

8000

Number of recordsCOMPUTER SCIENCE OPERATIONS RESEARCH MANAGEMENT SCIENCEENVIRONMENTAL SCIENCES ECOLOGY ENGINEERINGGOVERNMENT LAW PUBLIC ADMINISTRATIONPSYCHOLOGY AGRICULTURESOCIAL SCIENCES OTHER TOPICS SOCIOLOGYURBAN STUDIES HEALTH CARE SCIENCES SERVICESINTERNATIONAL RELATIONS TRANSPORTATIONPUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH GEOGRAPHY

40

5 Results and Discussion: Citation patterns of

publications

5.1 Introduction

Discussion of main patterns in citation will be based on the most cited publication of

the journal Econometrica.

It was chosen four articles that will be reviewed. First, this is most cited publication

in the journal. It is reviewed number of citations, which was received by the article

and research areas of journals, which cited this article.

Second, it is discussed main trends in citations of the oldest publication in Table 4.

Third, discussion about citation patterns of the newest article in the Table 4 is made.

Then, it is shown main tendencies in citations for the most cited article, which was

published in this century.

Finally, the attempt is made to compare citation patterns for these four articles.

5.2 Citation pattern of most cited publication in journal

Econometrica

The most cited article in the journal Econometrica is publication of two authors

Kahneman and Tversky. It was published in 1979 and named Prospect theory -

analysis of decision under risk. This article has received more than 9645 citation since

1979. This is 2.8% of total number of citations, which was received by the journal

Econometrica. In average, this article receives 271.1 citation per year.

Kahneman and Tversky (1979) in their research describe choices of the person in

different situations connected with risk. They criticise utility theory, because people

5: Results and Discussion: Citation patterns of publications

41

tend to change their attitudes to reward and loss because of level of risk and their

own value function. The decision is not based on maximal utility. It is individual choice

and this choice does not always reflect attitudes to money. In addition, Kahneman

and Tversky (1979) describes probabilistic insurance and differences in attitudes to

the risk when it connected with reward or loss. Differences in risk acceptance appears

because a value function is not a straight line.

Publication describes all this problems and present reasons for risk acceptance in

different situations. Therefore, high number of citations to this publication is

predictable. Data presented on the Figure 10 shows trends in citations of this article.

Year 2014 was removed from the figure, because data about this year is constantly

changing.

Figure 10: Citation over time to the article by Kahneman and Tversky

As can be seen from the Figure above the number of the citation for this publication

is constantly growing. In 1979, it received only four citations. However, in the next

year, this number quadrupled and publication receives 16 citations. The gradual

growth of the number of citations can be observed to the 2003. However, in 1991

and 1993 was present small decline in the number of citations, however, it did not

change the tendency. From the 2005, sharp increase in number of citations to this

publication can be observed. This number peaked in 2013 and was 837 citations.

0

100

200

300

400

500

600

700

800

900

1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2013

Nu

mb

er o

f ci

tati

on

s

Year


42

However, the reasons of citing this article is changed from the 1979. For example, in

1979 some authors used methodology of the Kahneman and Tversky, gave credit for

related work and corrected their work. However, in 2013 most of the reviewed

publications, which cited Kahneman and Tversky, just mention their names, which

can be considered as paying homage to pioneers.

Publication by Kahneman and Tversky Prospect theory - analysis of decision under risk

can be considered as multidisciplinary. Because, on the one hand, it is stongly

connected to the economics, but, on the other hand, it describes psychological

behaviour of the person. To prove that data about research areas of the journals

which cited this article is displayed in the Figure 11.

Figure 11: Research areas most frequently cited Kahneman and Tversky Prospect

theory - analysis of decision under risk

As was expected the two research areas, which cited this publication the biggest

number of times, is Business economics and Psychology. The most striking result to

emerge from the figure above is that Government law is cited this article a large

4834

2458

653516 504 439 405 395 365 354 353 276 226 189 185

2356

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

BUSINESS ECONOMICS PSYCHOLOGY

GOVERNMENT LAW OPERATIONS RESEARCH MANAGEMENT SCIENCE

COMPUTER SCIENCE MATHEMATICAL METHODS IN SOCIAL SCIENCES

MATHEMATICS ENGINEERING

NEUROSCIENCES NEUROLOGY ENVIRONMENTAL SCIENCES ECOLOGY

SOCIAL SCIENCES OTHER TOPICS HEALTH CARE SCIENCES SERVICES

PUBLIC ENVIRONMENTAL OCCUPATIONAL HEALTH TRANSPORTATION

BEHAVIORAL SCIENCES OTHERS


43

number of times. However, after analysing articles from the Government law, it can

be concluded that these articles is closely connected to economy, psychology or

describes voting behaviour of the people.

5.3 Citation pattern of oldest most cited publication in journal

Econometrica

Oldest most cited publication in journals Econometrica is article produced by Nash in

1950. This publication has received more than 2111 citation since 1950. It is 0.6% of

total citations, which was received by the journal Econometrica.

Nash (1950) describes bargaining between two individuals. This is done by deploying

mathematical model. Therefore, author uses Theory of Games and basic concepts of

it in paper. However, Nash (1950) develops his own utility theory of the individual

and introduces main assumption for it in the paper. According to Nash (1950) all

anticipation between two individuals would be based on their utility functions. In

addition, author outlines that he takes into account “only those cases in which there

is a possibility that both individuals could gain from the situation” (Nash, 1950, p.

158). Finally, it is described in paper the solution for the fair bargaining between two

individuals. This is done by finding best solution for them based on their utility

functions and the set of possibilities. Nash (1950) proposes the solution when two

utility function is maximized, however this is not an ultimate bargaining for one

individual, this is fair bargaining.

It is oldest article from table 4, therefore it receives less citation per year in average

than others. This number is 32.6. However, average number does not describe

tendencies in citation patterns of this article. Therefore, data about trends in citations

of Nash is presented on Figure 12. Year 2014 was removed from the figure, because

data about this year is constantly changing.


44

Figure 12: Citation over time to the article by Nash

As can be seen from the Figure 12 the number of citation to this paper show gradual

growth. It was not cited in first 3 years after publication. The most striking results

from to emerge the figure above is that number of citation rise slowly. This increase

was not connected with either the award of the John von Neumann Theory Prize in

1978 or the Leroy P. Steele Prize in 1999. In addition, award of the Nobel Memorial

Prize in Economic Sciences in 1994 did not affect the number of citation. However,

sharp grows of the citation count in 2005 did not connected with any award. This was

an outcome of growing citations from Business economics journals. However, it can

be assumed that movie about John Nash “Beautiful Mind”, which was released in

2001 and received four Academy Awards in 2002, Golden Globe Award and several

BAFTA Awards is connected to grows of citation count.

However, high number of citation from Business economics research areas can be

explained because publication is connected to this area. In addition, publication by

Nash cited by other research areas, as can be seen from the Figure 13.

0

20

40

60

80

100

120

140

160

19

53

19

55

19

57

19

59

19

61

19

63

19

65

19

67

19

69

19

71

19

73

19

75

19

77

19

79

19

81

19

83

19

85

19

87

19

89

19

91

19

93

19

95

19

97

19

99

20

01

20

03

20

05

20

07

20

09

20

11

20

13

Nu

mb

er o

f ci

tati

on

s

Year


45

Figure 13: Research areas most frequently cited Nash The bargaining problem

Citations from research areas such as Business economics, Mathematical methods in

social sciences and Mathematics are citations from the same field. However, citations

from Computer science are considered as export of knowledge. Citations to this

publications from Computer science papers were mostly made when publication is

connected to the application, which use Nash approach in the Game Theory. This

shows effective import of knowledge from this publication to the publication in

different research areas.

The trend for citations has changed for this paper since it was published. In 1953 it

was cited by the two publications from Econometrica. In addition, one citation was

made by Nash. These citations were made to present previous researches in this field.

However, in 1958 this publication starts to import knowledge for other disciplines. It

was cited by the journal World Politics as well as with journals from Business

economics research area. In the next year it was cited by the journal from

Government law American Political Science Review this shows interdisciplinary

connections of publication by Nash. In 2013 this publication was cited 119 times. This

0

200

400

600

800

1000

1200

1400

BUSINESS ECONOMICS COMPUTER SCIENCE

MATHEMATICAL METHODS IN SOCIAL SCIENCES MATHEMATICS

ENGINEERING OPERATIONS RESEARCH MANAGEMENT SCIENCE

GOVERNMENT LAW SOCIAL SCIENCES OTHER TOPICS

PSYCHOLOGY TELECOMMUNICATIONS

INTERNATIONAL RELATIONS ENVIRONMENTAL SCIENCES ECOLOGY

AUTOMATION CONTROL SYSTEMS PHILOSOPHY

PUBLIC ADMINISTRATION OTHERS


46

was made by the authors from different research areas. It can be assumed that they

pay homage to the pioneers as well as provide background reading.

5.4 Citation pattern of newest most cited publication in journal

Econometrica

Newest most cited publication in journals Econometrica is article produced by

Johansen in 1991. Since 1991 Estimation and hypothesis testing of cointegration

vectors in Gaussian vector autoregressive models has received more than 2008

citations. In average it is 84.7 times in a year. This is 0.6% of total citations of the

journal Econometrica.

Johansen (1991) describes “maximum likelihood estimators and likelihood ratio tests

for cointegration in Gaussian vector autoregressive models” in his paper. Based on

data on money demand from two countries author outlines probability theory when

processes in one country can make asymptotic inference to the processes in another.

First, this paper describes cointegration model and the tests, which can measure,

cointegration rank.

As can be seen from the Figure 14 citations for this article has a tendency to growth.

However, in 1997 the number of citation for this paper started to fall. This tendency

has completely changed since 2003. It has shown gradual growth since 2003 and

peaked in 2011. In 2012 and 2013 it can be seen small decline in number of citation.

Nonetheless, small decrease in citation count to the publication by Johansen can be

considered as temporal.

Publication by Johansen is methodological paper with strong accent mathematical

methods and models of economic processes. This models mostly used only in

economic field therefore, this article has not strong connection with other disciplines.

This can be seen from the Figure 15, because highest number of citations this paper

receives from the papers in the same research areas. Scientists, who cited this article,

adopt this methodology to their researches. Therefore, this publication is strongly

oriented inside the discipline.


47

Figure 14: Citation over time to the article by Johansen

Figure 15: Research areas most frequently cited Johansen Estimation and hypothesis

testing of cointegration vectors in Gaussian vector autoregressive models

However, there are some disciplines, which cited this article and import knowledge.

For example, Environmental science ecology researchers adopt this methodology to

0

20

40

60

80

100

120

140

160N

um

ber

of

cita

tio

ns

Year

0

500

1000

1500

2000

BUSINESS ECONOMICS MATHEMATICS

MATHEMATICAL METHODS IN SOCIAL SCIENCES ENVIRONMENTAL SCIENCES ECOLOGY

COMPUTER SCIENCE ENERGY FUELS

AGRICULTURE INTERNATIONAL RELATIONS

OPERATIONS RESEARCH MANAGEMENT SCIENCE ENGINEERING

SOCIAL SCIENCES OTHER TOPICS PUBLIC ADMINISTRATION

GOVERNMENT LAW URBAN STUDIES

TRANSPORTATION OTHERS


48

their own publications. Some publications in Computer science research area has

strong connection to Business economy. Publication from Energy fuels research area

also adopted methodology proposed by the Johansen in their researches.

5.4 Citation pattern of most cited publication in journal

Econometrica that was published in XXI century

The most highly cited article which was published in this century is article by Melitz.

It was published in 2003 and named The impact of trade on intra-industry

reallocations and aggregate industry productivity. It has been cited more than 1293

times. It is 107.8 times each year in average.

Article, published by Melitz (2003), develops “dynamic industry model with

heterogeneous firms to analyse the role of international trade as a catalyst for these

inter-firm reallocations within an industry”.

Data about citations of this article from 2004 to 2013 is presented on Figure 16. Year

2014 was removed from the figure, because data about this year is constantly

changing.

Figure 16: Citation over time to the article by Melitz

0

50

100

150

200

250

300

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

Nu

mb

er o

f ci

tati

on

s

Year


49

Article published by Melitz was firstly cited in 2004. In that year it received 13

citations. Since 2004, it has shown rapid growth in number of citations. In 2013 it was

cited by 252 other publications.

Figure 17: Research areas most frequently cited Melitz The impact of trade on intra-

industry reallocations and aggregate industry productivity

This is methodological paper, which focus on economic problem, therefore, it can be

suggested that publication does not receive high number of citation from different

research areas. Data from Figure 17 proves this assumption. It is highly cited by the

journal from the same research area. Therefore, this publication orients inside the

discipline.

However, publication by Melitz receives citations from publications that is connected

to International relations. This publication also have connection to economy and uses

to provide background reading and pay homage to pioneers. Similar to this pattern is

presented in papers from Public administration research area.

0

200

400

600

800

1000

1200

1400

BUSINESS ECONOMICS INTERNATIONAL RELATIONS

PUBLIC ADMINISTRATION MATHEMATICAL METHODS IN SOCIAL SCIENCES

ENVIRONMENTAL SCIENCES ECOLOGY GOVERNMENT LAW

GEOGRAPHY MATHEMATICS

AGRICULTURE URBAN STUDIES

OPERATIONS RESEARCH MANAGEMENT SCIENCE ENGINEERING

AREA STUDIES NUTRITION DIETETICS

FOOD SCIENCE TECHNOLOGY TRANSPORTATION

SOCIAL SCIENCES OTHER TOPICS


50

5.5 Comparison of citation patterns

The publications described earlier are the most cited publication in journal

Econometrica. However, this publications are cited by other scientists in different

ways. For example, publication by Kahneman and Tversky or Nash is big exporters of

the knowledge to the other disciplines. Publications by Johansen or Melitz also export

knowledge to other research areas, however, this publications is more oriented

inside the discipline. It is hard to determine reasons for citing these articles, however

it can be assumed that scientists from other filed use these publications as

methodological base for their researches.

Article published by Melitz in first year after publication received biggest number of

the citations. It can be assumed that articles became more available for the citations

because of the Internet. In addition, this article shows the fastest growth of the

citations count. In 2010, it has higher number of the citations than publications by

Nash or Johansen.

In addition, article published by Melitz gets higher number of citations than article

published by Nash or Johansen. However, paper by Kahneman and Tversky is still

most cited article in the journal Econometrica and has much higher number of

citations each year (Figure 18). Figure 18 contains information about citation count

to the year 2013 because data about 2014 is constantly changing.

As can be seen from the Figure 18 number of citations received by the articles is

growing. Most rapid growth shows the article published by Kahneman and Tversky.

Therefore, it can be assumed that this article will be the most cited publication in the

journal Econometrica for several more years.


51

Figure 18: Citation over time to the article by Kahneman and Tversky, Nash,

Johansen, Melitz

This shows that articles published in journal Econometrica can be interested for

scientists from different research areas. This fact has great impact on citations that is

received by each publication. It was shown that some articles in a journal

Econometrica is oriented inside the discipline and others are importer of knowledge.

The most cited papers in journal is methodological papers, which propose new

approaches to solution of different problems. However, citation count is not highly

related to the time of the publication. Publications, which were produced in recent

years, can receive high number of citations.

0

100

200

300

400

500

600

700

800

900

19

53

19

55

19

57

19

59

19

61

19

63

19

65

19

67

19

69

19

71

19

73

19

75

19

77

19

79

19

81

19

83

19

85

19

87

19

89

19

91

19

93

19

95

19

97

19

99

20

01

20

03

20

05

20

07

20

09

20

11

20

13

Kahneman and Tversky Kahneman and Tversky average

Nash Nash average

Johansen Johansen average

Melitz Melitz average

52

6 Conclusions

6.1 Key findings

From the first part of the research it can be concluded, that there were few

fluctuations in number of publications in journal Econometrica. This fluctuations can

be explained either general situation in the world (minimal level of publication in

1944) or policy of the editorial board. For example, increase from 1945 to 1964 can

be explained as development of the journal Econometrica. Maximal number of

publication in 1971 is the results of edition of big amount of meeting abstracts from

Second World Congress of the Econometric Society, Tenth Indian Econometric

Conference and North American Regional Conference.

Than it was find out that, there are significant differences in contribution for the

journal Econometrica from the scientists of different countries. Greatest contribution

is made by USA’s scientists. Second place is contributions from the UK’s scientists and

third is Canada. However, in recent years greatest contributors to the journal are

scientists from USA, UK and France, which shows the international impact of the

journal. In addition, the internationalization of the journal can be observed because

of growth in number of publications from authors from Israel, Canada, Netherlands

and Spain.

University of California System consists of ten individual universities and makes the

biggest contribution to the journal. The biggest contribution to the journal

Econometrica is made by institution that situated in USA. However, universities from

the United Kingdom also make significant contribution to the journal. For example,

University of London, which consists of 18 individual institutions, is on the sixth place

for its contribution.

6: Conclusions

53

Journal Econometrica is highly cited journal. However, different publications received

different number of citations. For example, twenty most cited articles receives almost

twenty percent of total citations. However, 2697 publications in the journal

Econometrica were never cited by others. This shows that distribution of citations is

highly skewed. In addition, in the Chapter 4 was proposed that distribution of

citations could be approximated with Zipf law

From the data analysed in chapter four can be concluded that journal Econometrica

has influence on journals from other fields. It is obvious that strongest relations

journal has with journals from similar research areas. However, knowledge from

Econometrica is imported by Computer science, Operations research management

science, Environmental sciences ecology and other sciences and social sciences.

In the Chapter Five citation patterns of four articles were analysed. It were

publications by Kahneman and Tversky (1979), Nash (1950), Johansen (1991) and

Melitz (2003).

Publication by Kahneman and Tversky is by far the most highly cited publication in

journal Econometrica. In 1979 it introduces new ideas and approaches of evaluation

decision making processes under risk. This publication was highly cited. Because of

this, it becomes popular among wide range of the scientist. However, nowadays

some citations is made because scientists pay homage to pioneers. In addition, high

number of citation, which receives this article, is connected to the fact that it is

interesting for different research areas.

Article published by Nash in 1950 was not cited in first years after publication.

However, it shows gradual growth of citation till nowadays. It was cited by the wide

range of scientists from different research areas. This shows that this article is

exporter of knowledge to other disciplines.

Publication by Johansen is newest one of the most highly cited article in journal

Econometrica. It can describes econometric methods and it can be suggested that it

is oriented inside the discipline. However, other disciplines use methodology

developed by Johansen as well as Business economic. For example, scientists from

6: Conclusions

54

Environmental science ecology and Energy fuels adopt this methodology to their own

researches.

Melitz’s publication The impact of trade on intra-industry reallocations and aggregate

industry productivity is one of highly cited article which was produced in recent years.

This article become highly cited in a short period. However, this publication describes

economic problem and receives most citations from the journals with the same

research areas. This shows that publication orients inside the discipline. In addition,

some citations from other research areas is mostly made because topic of the

research is highly connected to economy.

6.2 Meeting the aim and objectives

The overall aim of the study was ‘to make bibliometric analysis of journal

Econometrica and investigate export relationship between econometrics and other

research areas based on this journal’. To reach that aim next objectives were set up:

Objective 1: To carry out literature review presenting previous bibliometric analysis

of different journals and their impact on different scientific areas.

Chapter 2 of the study presents literature review. This part includes review of the

bibliometric studies of single journal, citation analysis and knowledge diffusion.

Therefore, impact of the journal on different scientific areas was assessed with

citation analysis and diffusion of knowledge from one research area to others.

Objective 2: To outline year wise distribution of publications in the journal

Econometrica.

The year wise distribution of publication in the journal Econometrica was presented

and analysed in second part of Chapter 4. It includes discussion of existed patterns

and explanation of unexpected results.

Objective 3: To identify geographical distribution of contributors to the journal

Econometrica.

6: Conclusions

55

Geographical distribution of contributors to the journal Econometrica is presented in

third part of Chapter 4. Distribution of contributors from different countries to the

journal Econometrica in 1933 - 2014 was discussed. In addition, recent trend in

contributions from different countries was presented and analysed.

Objective 4: To identify main organisations contributors to the journal

Econometrica.

Main organizations that contributes to the journal Econometrica is presented in forth

part of Chapter 4.

Objective 5: To describe citation trends of the journal Econometrica.

Citation trends in to the journal Econometrica is presented and discussed in fifth part

of Chapter 4. Analysis of total number of citation, which were received by to the

journal Econometrica. In addition, twenty most cited articles were presented. Finally,

total number of citation, which was received by the journal Econometrica, is

presented.

Objective 6: To explore which disciplines and journals export knowledge from the

journal Econometrica.

Export of knowledge from the journal Econometrica was discussed in final part of

Chapter 4 and in Chapter 5. Sixth part of Chapter 4 presents data about export of

knowledge by the journals and to the different research areas from the journal

Econometrica in general. Chapter 5 is focused on four most cited articles from the

journal. In this part was described export of knowledge from specific articles.

6.3 Recommendation for further research

Carry out similar study to the other econometric journals and compare

received results with findings of this study.

Analyse more specific publications from the journal Econometrica and add

received results to findings of this study.

6: Conclusions

56

Make deeper analysis of reasons for citing specific publication by the authors

from the other research areas.

Total word count: 10,131

57

7 Bibliography

Amsterdamska, O., Leydesdorff, L. (1989). Citations: indicators of significance?.

Scientometrics, 15(5-6), 449-471.

Anyi, W. U. K., Anuar, N. B., Zainab, A. N. (2009). Bibliometric studies on single

journals: a review. Malaysian Journal of Library & Information Science, 14(1),

17-55.

Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS,

Scopus and Google Scholar. Scientometrics, 82(3), 495-506.

Bergman, E. M. L. (2012). Finding Citations to Social Work Literature: The Relative

Benefits of Using Web of Science, Scopus, or Google Scholar. The Journal of

Academic Librarianship, 38(6), 370-379.

Biswas, B. C., Roy, A., Sen, B. K. (2007). Economic botany: a bibliometric study.

Malaysian Journal of Library & Information Science, 12 (1), 23-33.

Bedeian, A. G. (2005). Crossing disciplinary boundaries a epilegomenon for Lockett

and McWilliams. Journal of Management Inquiry, 14(2), 151-155.

Bonzi, S., Snyder, H. W. (1991). Motivations for citation: A comparison of self citation

and citation to others. Scientometrics, 21(2), 245-254.

Borkenhagen, A., Decker, O., Brahler, E., Strauss, B. (2002). The Journal of the Society

for Psychotherapy Research and the diversity of Psychotherapy Research: a

compilation and comparison of North American and European contributions.

Psychotherapy Research, 12(4), 491-514.

Bornmann, L., Daniel, H. D. (2008). What do citation counts measure? A review of

studies on citing behavior. Journal of Documentation, 64(1), 45-80.

7: Bibliography

58

Cronin, B., Meho, L. I. (2008). The shifting balance of intellectual trade in information

studies. Journal of the American Society for Information Science and

Technology, 59(4), 551-564.

Cronin, B., Pearson, S. (1990). The export of ideas from information science. Journal

of Information Science, 16(6), 381-391.

Elster, A. D., Chen, M. Y., Jenkins, C. B. (1994). The internationalization of the

American Journal of Roentgenology: 1980-1992. AJR. American Journal of

Roentgenology, 162(3), 519-522.

Damrosch, L.F. (2006). The "American" and the "international" in the American

Journal of International Law. American Journal of International Law, 100, 2-

16.

Ding, Y., Liu, X., Guo, C., & Cronin, B. (2013). The distribution of references across

texts: Some implications for citation analysis. Journal of Informetrics, 7(3),

583-592.

Folly, G., Hajtman, B., Nagy, J. I., & Ruff, I. (1981). Some methodological problems in

ranking scientists by citation analysis. Scientometrics, 3(2), 135-147.

Garfield, E. (1965, December). Can citation indexing be automated? In Statistical

association methods for mechanized documentation, symposium

proceedings, 189-192.

Garfield, E. (1979). Is citation analysis a legitimate evaluation tool?. Scientometrics,

1(4), 359-375.

Hammarfelt, B. (2011). Interdisciplinarity and the intellectual base of literature

studies: Citation analysis of highly cited monographs. Scientometrics, 86(3),

705-725.

Hellqvist, B. (2010). Referencing in the humanities and its implications for citation

analysis. Journal of the American Society for Information Science and

Technology, 61(2), 310-318.

7: Bibliography

59

Hessey, R., Willett, P. (2013). Quantifying the value of knowledge exports from

librarianship and information science research. Journal of Information

Science, 39(1), 141-150.

Jacso, P. (2005). As we may search: Comparison of major features of the Web of

Science, Scopus, and Google Scholar citation-based and citation-enhanced

databases. Current Science, 89(9), 1537-1547.

Jena, K. L. (2006). A bibliometric analysis of the journal 'Indian Journal of Fibre and

Textile Research', 1996–2004. Annals of Library and Information Studies,

53(1), 22-30.

Jishi, R. A., Willett, P. (2010). The Journal of Chemical Documentation and the Journal

of Chemical Information and Computer Sciences: publication and citation

statistics. Journal of Chemical Information and Modelling, 50(11), 1915-1923.

Johansen, S. (1991). Estimation and hypothesis testing of cointegration vectors in

Gaussian vector autoregressive models. Econometrica: Journal of the

Econometric Society, 59(6), 1551-1580.

Kahneman, D., Tversky, A. (1979). Prospect theory: An analysis of decision under risk.

Econometrica: Journal of the Econometric Society, 47(2), 263-291.

Lawani, S. M. (1982). On the heterogeneity and classification of author self‐citations.

Journal of the American Society for Information Science, 33(5), 281-284.

Larivière, V., Sugimoto, C. R., Cronin, B. (2012). A bibliometric chronicling of library

and information science's first hundred years. Journal of the American Society

for Information Science and Technology, 63(5), 997-1016.

Levitt, J. M., Thelwall, M., & Oppenheim, C. (2011). Variations between subjects in

the extent to which the social sciences have become more interdisciplinary.

Journal of the American Society for Information Science and Technology, 62(6),

1118-1129.

Leydesdorff, L. (1987). Towards a theory of citation?. Scientometrics, 12(5), 305-309.

7: Bibliography

60

Lockett, A., McWilliams, A. (2005). The balance of trade between disciplines: Do we

effectively manage knowledge?. Journal of Management Inquiry, 14(2), 139-

150.

Loebbecke, C., Leidner, D. (2012). The contribution of top IS publications to

subsequent research: A citation analysis. Communications of the Association

for Information Systems, 30, 423-439.

Lowry, P. B., Karuga, G. G., Richardson, V. J. (2007). Assessing leading institutions,

faculty, and articles in premier information systems research journals.

Communications of the Association for Information Systems, 20(1), 142-203.

MacRoberts, M. H., MacRoberts, B. R. (1989). Problems of citation analysis: A critical

review. Journal of the American Society for Information Science, 40(5), 342-

349.

MacRoberts, M. H., MacRoberts, B. R. (1996). Problems of citation analysis.

Scientometrics, 36(3), 435-444.

McElwee, G., Atherton, A. (2005). Publication trends and patterns in

entrepreneurship: The case of the International Journal of Entrepreneurship

and Innovation. Journal of Small Business and Enterprise Development, 12(1),

92-103.

Melitz, M. J. (2003). The impact of trade on intra‐industry reallocations and aggregate

industry productivity. Econometrica: : Journal of the Econometric Society,

71(6), 1695-1725.

Narin, F. (1976). Evaluative bibliometrics: the use of publication and citation analysis

in the evaluation of scientific activity. Washington, USA: Computer Horizons.

Nash Jr, J. F. (1950). The bargaining problem. Econometrica: Journal of the

Econometric Society, 18(2), 155-162.

Noruzi, A. (2005). Google Scholar: The new generation of citation indexes. Libri, 55,

170-180.

7: Bibliography

61

Peritz, B. C. (1983). Are methodological papers more cited than theoretical or

empirical ones? The case of sociology. Scientometrics, 5(4), 211-218.

Ravichandra Rao, I. K., Bhusan, S. B. (2008). Distributions of Multiple Authors in Two

Journals (JASIST and Scientometrics) and in the subject Software Studies: A

Case Study. Retrieved from: http://www.collnet.de/Berlin-

2008/RaoWIS2008dma.pdf.

Stigler, S. M. (1994). Citation patterns in the journals of statistics and probability.

Statistical Science, 94-108.

Tagliacozzo, R. (1977). Self-citations in scientific literature. Journal of Documentation,

33(4), 251-265.

The Concise Encyclopedia of Economics. 2008. "Ragnar Frisch." Library of Economics

and Liberty. Retrieved June 23, 2014 from the World Wide Web:

http://www.econlib.org/library/Enc/bios/Frisch.html

Torricella-Morales, R. G., Van Hooydonk, G., Araujo-Ruiz, J. A. (2000). Citation analysis

of cuban research. Part 1. A case study: the Cuban Journal of Agricultural

Science. Scientometrics, 47(2), 413-426.

Urata, H. (1990). Information flows among academic disciplines in Japan.

Scientometrics, 18(3), 309-319.

Willett, P. (2007). A bibliometric analysis of the Journal of Molecular Graphics and

Modelling. Journal of Molecular Graphics and Modelling, 26(3), 602-606.

Yan, E., Ding, Y., Cronin, B., & Leydesdorff, L. (2013). A bird's-eye view of scientific

trading: Dependency relations among fields of science. Journal of

Informetrics, 7(2), 249-264.

Information School Research Ethics Panel

Confirmation of a ‘no-risk’ application

Date: 12th

June 2014

TO: Olha Rodzen

The Information School Research Ethics Panel has examined the following

application:

Title: A bibliometric analysis of the journal ‘Econometrica’

Submitted by: Olha Rodzen

The Panel has concluded that the proposed research is classed as ‘no-risk’, and as

such does not require ethics approval. No further action needs to be taken.

This letter is the official record of ethics approval by the School, and should

accompany any formal requests for evidence of research ethics status.

Effective Date: 12th

June 2014

Dr Angela Lin

Research Ethics Coordinator

Information

School.

Access to Dissertation

A Dissertation submitted to the University may be held by the Department (or School) within which the

Dissertation was undertaken and made available for borrowing or consultation in accordance with

University Regulations.

Requests for the loan of dissertations may be received from libraries in the UK and overseas. The

Department may also receive requests from other organisations, as well as individuals. The conservation

of the original dissertation is better assured if the Department and/or Library can fulfill such requests by

sending a copy. The Department may also make your dissertation available via its web pages.

In certain cases where confidentiality of information is concerned, if either the author or the supervisor so

requests, the Department will withhold the dissertation from loan or consultation for the period specified

below. Where no such restriction is in force, the Department may also deposit the Dissertation in the

University of Sheffield Library.

To be completed by the Author – Select (a) or (b) by placing a tick in the appropriate box

If you are willing to give permission for the Information School to make your dissertation available in

these ways, please complete the following:

✔ (a) Subject to the General Regulation on Intellectual Property, I, the author, agree to this dissertation

being made immediately available through the Department and/or University Library for

consultation, and for the Department and/or Library to reproduce this dissertation in whole or

part in order to supply single copies for the purpose of research or private study

(b) Subject to the General Regulation on Intellectual Property, I, the author, request that this

dissertation be withheld from loan, consultation or reproduction for a period of [ ] years from

the date of its submission. Subsequent to this period, I agree to this dissertation being made

available through the Department and/or University Library for consultation, and for the

Department and/or Library to reproduce this dissertation in whole or part in order to supply

single copies for the purpose of research or private study

Name Olha Rodzen

Department Information School

Signed

Date 29.08.14

To be completed by the Supervisor – Select (a) or (b) by placing a tick in the appropriate

box

(a) I, the supervisor, agree to this dissertation being made immediately available through the

Department and/or University Library for loan or consultation, subject to any special restrictions

(*) agreed with external organisations as part of a collaborative project.

*Special

restrictions

(b) I, the supervisor, request that this dissertation be withheld from loan, consultation or

reproduction for a period of [ ] years from the date of its submission. Subsequent to this period,

I, agree to this dissertation being made available through the Department and/or University

Library for loan or consultation, subject to any special restrictions (*) agreed with external

organisations as part of a collaborative project

Name

Department

Signed Date

THIS SHEET MUST BE SUBMITTED WITH DISSERTATIONS BY DEPARTMENTAL REQUIREMENTS.

a bibliometric analysis of the journal...

Documents