the geography of distance education research - bibliographic characteristics of a journal network

The Geography of Distance Education Research - Bibliographic Characteristics of a Journal Network

DEHub/ODLAA 2011 SummitSydney, Australia16 February 2011

Olaf Zawacki-Richter

University of Oldenburg, GermanyCenter for Lifelong LearningFaculty of Educational and Social Sciences

Prof. Olaf Zawacki-Richter Slide / 432

Overview

Part I: Previous research into journals Part II: Scientific network of DE journals Part III: Validated structure of distance education research areas Part IV: Review of research between 2000 and 2008 Conclusions and implications

(Zawacki-Richter, 2009; Zawacki-Richter, Bäcker & Vogt, 2009; Zawacki-Richter & von Prümmer, 2010; Zawacki-Richter, Anderson & Tuncay, 2010; Zawacki-Richter & Anderson, forthcoming)


Online Distance Education / Flexible Learning

Flexible Learning

Distance Learning

E-learning

OnlineLearning

M-learning

Paper-based

Distance Learning

Contact Learning

(residential/face-to-face)

(based on Brown, 2004)


Part I: Previous research into journals

Academic review and publishing Quality, perceived value and impact of journals Flow of information in a scientific knowledge network


Quality, perceived value and impact: two approaches

Perception data based on expert's opinions (critique: e.g. Everett & Pecotich, 1993)

Citation analysis:[…] the study of relationships among authors, articles, journals, concepts, etc. as measured by references in documents. Citation analysis is one research tool in studying the sociology and structure of science, including topics such as the existence and changes in disciplinary and subdisciplinary boundaries; growth or decline of paradigms; patterns of communication within and across research fields, institutions, and authors; status differences in methods, research topics, and researchers; identification of research topics.(Rice, Borgman, Bednarski, Hart, 1989, p. 257)


Quality, perceived value and impact: previous research

Gross & Gross (1927): Journal of the American Chemical Society Daniel & Louttit (1953): Citation patterns and clustering of journals Garfield (1972): Impact factor Hirsch (2005): Hirsch Index (h) Zawacki-Richter, Anderson &

Tuncay (2010): Impact of DE journals

Pioneers of the "impact factor":http://www.garfield.library.upenn.edu/papers/history/heritagey1998.html


Journals as scientific knowledge networks

Journals as important carriers of knowledge in a discipline Research builds upon and refers to previous research Previous research is acknowledged by citations Social Network Analysis (SNA): The nodes are journals and the ties in

the network are citations. The journal network reveals relationships and patterns of scientific

information exchange, the "intellectual structure" of a discipline (Liu, 2007).


Journals as scientific knowledge networks

Marketing, accounting and management (Baumgartner & Pieters, 2003; Danell, 2000; Wakefield, 2008)

Geography (Gatrell & Smith, 1984; Doreian, 1988) Demography (Liu & Wang, 2005) Psychology (Doreian, 1985; Kawano, Kehle, Clark & Jenson; 1993,

Liu, 2007) Social work (Baker, 1992) Fisheries & aquatic science

(McCain, 1994)


Limitations of bibliometrics

"Extreme positions extend from those who would employ a citation measure in the granting of tenure to an individual faculty member to those who deny the validity of any application of citation data beyond the use in literature search" (Pinski & Narin, 1976, p. 297)

Validity? Reasons for citing and not citing


Part II: Distance Education Journals

What are the major journals in the field and how can they be grouped in terms of impact and perceived quality?

How do the journals relate to each other in a network of giving and receiving citations?

What are the most central and prestigious journals in the network and which journals are peripheral or isolated?


Selected Journals


Journal Citation Network

Mean citation rate across all journals: 22 ("tie density") Standardized mean citation rate: 0.23 cites/paper


UCINET 6 (Borgatti, Everett & Freeman, 2002)


Journal Citation Network – Image Matrix

(cf. Doreian & Fararo, 1985)


Journal Citation Network – Image Matrix


Journal Citation Network – SNA Measures


Multidimensional Scaling (MDS)

Young, F. W. (1985). Multidimensional scaling. In Kotz-Johnson (Ed.), Encyclopedia of Statistical Sciences. http://forrest.psch.unc.edu/teaching/p208a/mds/mds.html

http://forrest.psch.unc.edu/teaching/p208a/mds/mds.html



MDS Map of Distance Education Journals


Perceived quality

Journal Prestige Survey: Editorial Board Members (83 of 277). Rating scale according to Nelson, Buss & Katzko (1983):

Please assign numbers to the journals listed below according to how you would rate them, where 5 = outstanding, 4 = excellent, 3 = good, 2 = adequate, 1 = poor. You may leave blanks if you do not recognize a journal.


Perceived quality and prestige


Part III: Research Areas in Distance Education

What are the most important research areas in DE and how can they be categorized?

What are the most neglected research areas?


Research Areas: Delphi Study

Expert panel: 25 individuals from 11 countries (average of 27 years of professional experience in distance education)

1. round: Experts were asked to list 10 important research areas 2. round: Experts were asked to rate research areas on a scale of

importance; What are the most neglected research areas?


Classification of Research Areas

Three broad research perspectives or levels with 15 research areas:

1. Macro level: Distance education systems and theories→ global system level: 5 research areas

2. Meso level: Management, organisation and technology→ institutional level: 7 research areas

3. Micro level: Teaching and learning in distance education→ individual level: 3 research areas


Classification of Research Areas

Macro level: Distance education systems and theories

1. Access, equity and ethics2. Globalization of education and cross-

cultural aspects3. Distance teaching systems and

institutions4. Theories and models5. Research methods in distance education

and knowledge transfer

Micro level: Teaching and learning in DE13. Instructional design14. Interaction and communication in learning

communities15. Learner characteristics

Meso level: Management, organisation and technology

6. Management and organisation7. Costs and benefits8. Educational technology9. Innovation and change10. Professional development and faculty

support11. Learner support services12. Quality assurance


Neglected Research Areas

Macro level: cross-cultural aspects of transnational education Meso level: innovation and change management, educational

management and organisation, costs and funding of DE Micro level: added value of Web 2.0 applications (instructional

design)


Part IV: Publication and Authorship Patterns

What are the topics with the highest and lowest number of papers? What kind of research methods are applied? Do female and male researchers tend to choose different research

topics? If yes, what are their priority areas? Who are the leading authors and where do they come from?


Literature Review

Review of articles that were published in 5 prominent DE journals 9 years: 2000 to 2008 N=695 (675 English, 20 French)

Journals: Open Learning (UK) Distance Education (Australia) American Journal of Distance Education (USA) Journal of Distance Education (Canada) IRRODL (Canada)


Research Trends and Gaps

No significant trends between 2000 and 2008:

0

5

10

15

20

25

2000 2001 2002 2003 2004 2005 2006 2007 2008

Year

No

. o

f ar

ticl

es

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

N=695


Research areas and levels

13

13

24

31

62

12

13

18

23

41

41

48

113

121

122

0 20 40 60 80 100 120 140

7 Costs and benefits

2 Globalisation of education andcross-cultural aspects

5 Research methods in distanceeducation and knowledge transfer

9 Innovation and change

6 Management and organisation

11 Learner support services

4 Theories and models

1 Access, equity and ethics

10 Professional development andfaculty support

12 Quality assurance

8 Educational technology

3 Distance teaching systems andinstitutions

15 Learner characteristics

13 Instructional design

14 Interaction and communication inlearning communities

1 Macro level

2 Meso level

3 Micro level

N=695


Gender differences in research areas (single authors)

0.4%

-1.8%

-10.4%

-5.5%

-1.9%

-1.7%

-5.6%

-3.9%

-0.5%

2.5%

3.8%

-0.3%

6.0%

11.6%

7.4%

15 Learner characteristics14 Interaction and communication in learning communities13 Instructional design12 Quality assurance11 Learner support services10 Professional development and faculty support9 Innovation and change8 Educational technology7 Costs and benefits6 Management and organisation5 Research methods in DE and knowledge transfer4 Theories and models3 Distance teaching systems and institutions2 Globalisation of education and cross-cultural aspects1 Access, equity and ethics

preferred by men

preferred by women

χ2 = 38.07, df=14, p < .001

Cramer's V = .36, p < .001

N=302159 (M)143 (F)


Research Methods and Collaboration

Trend towards more empirical research? Berge & Mrozowski (2000): 76 % descriptive (N=727 articles; AJDE,

DE, JDE and OL, between 1990 and 1999). 2000-2008: 38 % descriptive


Research Methods and Collaboration

Collaboration is operationalized through co-author relationships Average number of authors per article is 2.7. Hunter & Leahey (2008) found an average number of 2.6 in leading

sociology journals.


Authorship Patterns: Leading Contributors

1,148 different authors contributed to the 695 articles in this study. 48 authors who published at least 3 articles:

Canada (17) USA (9) UK (8) Australia (5), China (5) Israel (2) Japan/South Korea (1) New Zealand (1)


Authorship Patterns: Countries

The origin of the first author was taken into consideration (54 countries) The vast majority (>80%) comes from only 5 countries AJDE: strong North-American focus


Conclusions and implications


Conclusions and implications

Relationship between DE and adult education / lifelong learning Ulrich's Periodicals: 29 peer-reviewed AE journals Research collaboration: authors, institutions, countries, projects Morrison et al. (2003): 444 research projects in New Zealand


References

Bakeman, R., & Gottman, J. M. (1997). Observing interaction - an introduction to sequential analysis. Cambridge: Cambridge University Press.

Baker, D. R. (1992). A structural analysis of social work journal network. Journal of Social Service Research, 15(3), 153-168.

Baumgartner, H., & Pieters, R. (2003). The structual influence of marketing journals: A citation analysis of the Discipline and its subareas over. The Journal of Marketing, 67(2), 123-139.

Berge, Z., & Mrozowski, S. (2001). Review of research in distance education. American Journal of Distance Education, 15(3), 5-19.

Borgatti, S. P., Everett, M., & Freeman, L. C. (2002). UCINET for Windows: software for social network analysis. Harvard, MA: Analytic Technologies.

Brown, T. (2004). The role of m-learning in the future of e-learning in Africa. In D. Murphy, R. Carr, J. Taylor, & W. Tat-meng (Eds.), Distance education and technology: issues and practice (pp. 197-216). Hong Kong: Open University of Hong Kong Press.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.

Danell, R. (2000). Stratification among journals in management research: a bibliometric study of interaction between European and American journals. Scientometrics, 49(1), 23-38.

Daniel, R. S., & Louttit, C. M. (1953). Professional problems in psychology. New York: Prentice Hall. Doreian, P. (1985). Structural equivalence in a psychology journal network. Journal of the American Society

for Information Science, 36(6), 411-417.


Doreian, P., & Fararo, T. J. (1985). Structural equivalence in a journal network. Journal of the American Society for Information Science, 36(1), 28-37.

Doreian, P. (1988). Testing structural-equivalence hypotheses in a network of geographical journals. Journal of the American Society for Information Science, 39(2), 79-85.

Everett, J. E., & Pecotich, A. (1993). Citation analysis mapping of journals in applied and clinical psychology. Journal of Applied Social Psychology, 23(9), 750-766.

Fleiss, J. L. (1981). Statistical methods for rates and proportions. New York: Wiley.Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471-479. Gatrell, A., & Smith, A. (1984). Networks of relations among a set of geographical journals. The

Professional Geographer, 36(3), 300-307.Gross, P. L. K., & Gross, E. M. (1927). College libraries and chemical education. Science, 66(1713), 385-

389. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the

National Academy of Sciences of the United States of America, 102(46).Hunter, L., & Leahey, E. (2008). Collaborative research in sociology: trends and contributing factors.

The American Sociologist, 39(4), 290-306.Kawano, T., Kehle, T. J., Clark, E., & Jenson, W. R. (1993). School psychology journals: Relationships with

related journals and external and internal quality indices. Journal of School Psychology, 31(3), 407-424.

Liu, Z. (2007). Scholarly communication in educational psychology: a journal citation analysis. Collection Building. doi:10.1108/01604950710831915

Liu, Z., & Wang, C. (2005). Mapping interdisciplinarity in demography: a journal network analysis. Journal of Information Science, 31(4), 308-316.

McCain, K. W. (1994). Islands in the stream: mapping the fisheries & aquatic sciences literatures. Fisheries, 19(10), 20-27.

42

http://dx.doi.org/10.1108/01604950710831915


Morrison, P. S., Dobbie, G., & McDonald, F. J. (2003). Research collaboration among university scientists. Higher Educational Research & Development, 22(3), 275-296.

Nelson, T. M., Buss, A. R., & Katzko, M. (1983). Rating of scholarly journals by chairpersons in the social sciences. Research in Higher Education, 19(4), 469-497.

Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Information Processing & Management, 12(5), 297-312.

Rice, R. E., Borgman, C. L., Bednarski, D., & Hart, P. J. (1989). Journal-to-journal citation data: Issues of validity and reliability. Scientometrics, 15(3-4), 257-282.

Wakefield, R. (2008). Networks of accounting research: A citation-based structural and network analysis. The British Accounting Review, 40(3), 228-244.

Young, F. W. (1985). Multidimensional scaling. In Kotz-Johnson (Ed.), Encyclopedia of Statistical Sciences. http://forrest.psch.unc.edu/teaching/p208a/mds/mds.html

Zawacki-Richter, O. (2009). Research areas in distance education – a Delphi study. International Review of Research in Open and Distance Learning, 10(3), 1-17.

Zawacki-Richter, O., Bäcker, E. M., & Vogt, S. (2009). Review of distance education research (2000 to 2008) – analysis of issues, methods and authorship patterns. International Review of Research in Open and Distance Learning, 10(6), 21-50.

Zawacki-Richter, O., & von Prümmer, C. (2010). Gender and collaboration patterns in distance education research. Open Learning, 25(2), 95-114.

Zawacki-Richter, O., Anderson, T., & Tuncay, N. (2010). The growing impact of open access distance education journals - a bibliometric analysis. Journal of Distance Education, 24(3).

Zawacki-Richter, O., & Anderson, T. (forthcoming). The geography of distance education – bibliographic characteristics of a journal network.

43



Thanks for your attention!

Olaf Zawacki-Richter, Ph. D.

University of Oldenburg

Faculty of Educational and Social Sciences

Center for Lifelong Learning (C3L)

Germany

[email protected]


Inter-rater Reliability

Research areas and methods were coded by a team of three researchers.

Cohen's kappa (κ) (Cohen, 1960) which is a coefficient for the degree of consistency among raters based on the number of codings in the coding scheme:

κ = .40 - .60 (fair) κ = .60 to .75 (good) κ > .75 (excellent) (Fleiss, 1981; Bakeman & Gottman, 1997)

κAreas = .675

κMethods= .855

the geography of distance education research - bibliographic characteristics of a journal network

Education

review of research

research fields

research tool

strand of research

overview of previous

zawackirichter anderson

olaf zawackirichterslide

outlets of research