scholarly communication research groupracoss.idi.hr/roundtable_croatia_zagreb_kulczycki.pdf1986 1989...
TRANSCRIPT
![Page 1: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/1.jpg)
Scholarly Communication Research Group
Changing landscape of journals in social sciences and humanities in Poland
Zagreb, 5th of April 2018Emanuel Kulczycki
The lessons from Polish science policy
![Page 2: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/2.jpg)
1. Evaluation of Scientific Institutions in Poland.
2. Polish Journal Ranking as the key instrument of evaluation.
3. The landscape of scholarly journals in Poland.
4. How has journal evaluation changed Polish journals in social sciences and humanities?
5. Toward a new model of evaluation.
6. Lessons from the Polish case.
Structure of Presentation
2
![Page 3: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/3.jpg)
3
1.Evaluation of Scientific Institutions in Poland
![Page 4: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/4.jpg)
4
The context and the scale of evaluation exercise
Warsaw
Poland
38 million citizens 430 higher education
institutions110.000 scholars
3.000 scientific journals
![Page 5: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/5.jpg)
5
1991 1998 2010 2018
Poland started the evaluation of scientific institutions
Parametric model and journal evaluation
Polish Journal Ranking (Version 1 – old)
Polish Journal Ranking (Version 2 – new)
The History of Polish Research Evaluation
![Page 6: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/6.jpg)
6
2Polish Journal Ranking as the key instrument of
evaluation
![Page 7: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/7.jpg)
the 2016 Polish Journal Ranking
List (part) Points Number of
Journals What journals are indexed
A list 15–50 11,271 Journals indexed in the Journal Citation Reports (JCR)
B list 1–152,209(over
1,200 from the SSH)
Polish Journals without an Impact Factor and not indexed in the European Reference Index for the Humanities (ERIH)
C list 10–25 4,111Journals indexed in the European Reference Index for the Humanities (ERIH)
7
![Page 8: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/8.jpg)
Formal Evaluation
Bibliometric Evaluation
Expert-based Evaluation
Three dimensions of evaluation(The B list) Parameters
Percentage of authors from foreign countriesPublishing without postponements greater than 6 months
10 parameters
for instance:
2 indicators Predicted Impact Factor (PIF)
Scimago Impact Factor (SIF)
Recommendations by the Committees of the Polish Academy of Sciences
8
![Page 9: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/9.jpg)
0
10
20
30
40
50
Points per publication
50
252515
5
Book Chapter Article – the B list (max)Monograph Article – the C list (max)Article – the A list (max)
Journals from Social Sciences: 50 points (max) if indexed in the Journal Citation Reports (very high Impact Factor in the subject category)
Journals from Humanities: 25 points (max) if indexed in the ERIH and many citations in Scopus
![Page 10: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/10.jpg)
10
2.Landscape of scholarly journals
in social sciences and humanities in Poland
![Page 11: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/11.jpg)
11
Polish Journals
356Scopus**
257Web of Science**
over 3,000 Polish scientific journals in all fields of science
2476Polish Journal Ranking***
* ** ***
~10% of those journals is from social sciences and humanities~20% of those journals is from social sciences and humanities~60% of those journals is from social sciences and humanities
![Page 12: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/12.jpg)
Landscape of Polish scientific journals
Social sciences and humanities
![Page 13: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/13.jpg)
Humanities
![Page 14: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/14.jpg)
Social Sciences
![Page 15: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/15.jpg)
15
Active Polish scientific journals in 2017N
umbe
r of J
ourn
als
0
10
20
30
40
50
60
70
80
90
100
Year of journal founding (1st year of publishing)
182018731879188718931897190219081911191419191922192519281931193419371940194719501953195619591962196519681971197419771980198319861989199219951998200120042007201020132016
![Page 16: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/16.jpg)
16
Editorial standards of Polish Journals: situation at December 2017Sh
are
of jo
urna
ls
0%10%20%30%40%50%60%70%80%90%
100%
A number of points on the Polish Journal Ranking0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 20 25 30 40
ORCIDDOIAnti-plagiarism systemJournal management and publishing system
![Page 17: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/17.jpg)
63.4%
26.4%
68.3%
61.8%
78.7%
45.9%25.8%
17.2%
Percentage of peer reviewed publications in English, local language(s) and other languages in the social sciences and humanities in 2014
EnglishLocal language(s)Other language(s)
Norway
Finland
Poland
Czech Republic
Flanders (Belgium)
Slovakia
Slovenia
Denmark
DOI :10.1007/s11192-018-2711-0
% of publications in English
![Page 18: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/18.jpg)
4.How has journal evaluation changed Polish journals in
social sciences and humanities
![Page 19: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/19.jpg)
It is easier to establish a new journal than publish a good paper
From to 2012 to 2015, nearly 600 Polish journals were added to the Polish Journal Rankings.
2012
2013
2015
0 750 1 500 2 250 3 000
2 212
1 807
1 639
19
![Page 20: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/20.jpg)
20
What is the actual effect of science policy oriented on internationalization?
% of members of the editorial advisory board from foreign countries
% of authors from foreign countries
% of reviewers from foreign countries
% of articles published in the so-called congress languages, i.e. English, German, French, Spanish, Russian, and Italian
Journals submitted information for the last two years in each edition.
Science policy’s goal: Internationalization
![Page 21: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/21.jpg)
In all groups of sciences, the primary effect of science policy has been a change in the composition of editorial advisory boards.It was the easiest way to play the game.
Journals from all sciences increased % of reviewers from foreign countries.
% of authors and % articles in the congress languages did not change significantly in the analyzed period.
21
Effects of Science Policy
![Page 22: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/22.jpg)
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
2012 2013 2015 2012 2013 2015 2012 2013 2015 2012 2013 2015
Social SciencesHumanitiesHard Sciences
Internationalization of Authors
Internationalization of Reviewers
Internationalization of Languages
Internationalization of Editorial Boards
22
![Page 23: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/23.jpg)
5.Toward a new model of evaluation
![Page 24: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/24.jpg)
24
1. We need a bibliometrics indicator (not formal criteria) to assess journals from all fields.
2. Journals from the humanities need to be able achieve the highest number of points.
3. A part of local Polish journals need funding to improve their editorial practices.
Three main assumptions of the new model
![Page 25: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/25.jpg)
25
Evolution of the Polish Journal Ranking
The A list
Journals indexed in the Journal Citation Reports
Polish journals without the Impact Factor
Journals indexed in the ERIH.
The B list
The C list
Old model New model
Journals indexed in SCOPUS (SNIP)
+Journals indexed in WoS and not in Scopus
+
up to 250 local Polish journals mostly from SSH
15–50 points
1–15 points
10–25 points
Three lists One list
Extra funding(grant program)
![Page 26: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/26.jpg)
6.Lessons from the Polish case
![Page 27: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/27.jpg)
Lessons
Each country has its own starting point (e.g. the share of publications in English) and the evaluation criteria must be fitted to it.
Web of Science is not appropriate for the evaluation of SSH journals.
If a bibliometric indicator is needed, SNIP is one of the best options for the SSH.
Using formal criteria does not allow to achieve science policy goals.
Local SSH journals need stable funding (for years!) to improve their editorial practices.
Expert-based assessment (made by the members of the Polish Academy of Sciences’ committees) was a total failure.
27
![Page 28: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/28.jpg)
Recommendations for policy makers
A system of journal evaluation should acknowledge two types of journals in the SSH:
➡ those that are actually prestigious and internationally oriented or might become such;
➡ those that are actually locally oriented and have implemented the highest standards of editorial practices.
A system of journal evaluation should encourage small journals (e.g. from a single faculty or region) to consolidate.
A system of evaluation can promote good practices (establishing journal websites) but – most often – it promotes playing the game. Thus, criteria have to be clear, well-matched, and systematically revised.
28
![Page 29: Scholarly Communication Research Groupracoss.idi.hr/Roundtable_Croatia_Zagreb_Kulczycki.pdf1986 1989 1992 1995 1998 2001 2004 2007 2010 2013 2016 16 Editorial standards of Polish Journals:](https://reader036.vdocument.in/reader036/viewer/2022071001/5fbd8fbb299da3491e1307fe/html5/thumbnails/29.jpg)
Presented results were based mostly on below publications:
1. Kulczycki, E. (2017). ‘Assessing publications through a bibliometric indicator : The case of comprehensive evaluation of scientific units in Poland’, Research Evaluation, 45/1: rvw023. DOI: 10.1093/reseval/rvw023
2. Kulczycki, E. , Korzeń, M., & Korytkowski, P. (2017). ‘Toward an excellence-based research funding system: Evidence from Poland’, Journal of Informetrics, 11/1: 282–98. DOI: 10.1016/j.joi.2017.01.001
3. Kulczycki, E. , Engels, T. C. E., Pölönen, J., Bruun, K., Dušková, M., Guns, R., Nowotniak, R., et al. (2018). ‘Publication patterns in the social sciences and humanities: evidence from eight European countries’, Scientometrics. DOI: 10.1007/s11192-018-2711-0
4. Sorokowski, P., Kulczycki, E. , Sorokowska, A., & Pisanski, K. (2017). ‘Predatory journals recruit fake editor’, Nature, 543/7646: 481–3. DOI: 10.1038/543481a
5. Kulczycki, E. , & Rozkosz, E. A. (2017). ‘Does an expert-based evaluation allow us to go beyond the Impact Factor? Experiences from building a ranking of national journals in Poland’, Scientometrics, 111/1: 417–442. Springer Netherlands. DOI: 10.1007/s11192-017-2261-x