scival research analyticsrdo.psu.ac.th/.../scivaltrainingdeckdec2016.pdf · scival modules ....
TRANSCRIPT
| 1 | 1
SciVal Research Analytics Prince of Songkla University
prepared by Alexander van Servellen Consultant, Research Management, Elsevier
December 2016
| 2 | 2
Agenda
Part I
Part II
• Demonstrative analysis
• Research Metrics
• Non-Performance Variables
• CiteScore
• Hands on SciVal training
• Setting up organizational structure
• Analysis
• Excercises
| 3 | 3
Part I Demonstrative Analysis
| 4 | 4
Part I – Demonstrative Analysis
Demonstrate how analytics from SciVal can help Prince of Songkla University
achieve its goals.
1.) What are the key subject strengths & weaknesses of PSU?
2.) How well is our faculty of Science doing?
3.) Who are the top performing researchers of Faculty of Science?
4.) How much do the researchers in Faculty of Science publish in top quality journals?
5.) How has PSU’s research contributed to Innovation?
6.) How does PSU publication output compare to other Thai Institutions?
7.) Who are the key collaboration partners of PSU?
8.) How well does PSU perform in “Rubber” research?
9.) Who are the top performing institutions and researchers “Rubber” research in the
world?
Objective
Topics
| 5 | 5
1.) What are the key subject
strengths & weaknesses of PSU?
| 6 | 6
Overview Module Prince of Songkla University
PSU’s research cited same as
world average level
| 7 | 7
PSU Top Subject areas Overview Module
| 8 | 8
Output, Impact & Growth by Subject Bubble size = # publications X-axis = FWCI Y-axis = Growth (publications)
Engineering
Chemistry
Biochemistry, Genetics and Molecular Biology
Materials Science
Physics and Astronomy
Chemical Engineering
Pharmacology, Toxicology and Pharmaceutics
Computer Science
Environmental Science
Mathematics
Energy
Nursing
Arts and Humanities
Neuroscience
Medicine
Agricultural and Biological Sciences
Immunology and Microbiology Dentistry
Earth and Planetary Sciences
Veterinary
-50.0%
-30.0%
-10.0%
10.0%
30.0%
50.0%
70.0%
90.0%
110.0%
0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6
GR
OW
TH
Field Weighted Citation Impact
World Field Weighted Citation Impact = 1 High Growth –
High Impact
Low Growth – High Impact
High Growth – Low Impact
Low Growth – Low Impact Data from
Overview Module
| 9 | 9
2.) How well is PSU faculty of Science
doing?
| 10 | 10
PSU– Faculty of Science
Overview Module
| 11 | 11
PSU– Faculty of Science
Overview Module
| 12 | 12
3.) Who are Faculty of Science’s top
performing Researchers?
| 13 | 13
Highest Impact Researchers
Overview Module
| 14 | 14
Each Author has a Profile
Overview Module
| 15 | 15
4.) How much do the Departments
in Faculty of Science publish in top
quality journals?
| 16 | 16
Faculty of Science Departments ranked by
%papers in Top Quality Journals
Benchmarking Module
| 17 | 17
5.) How has PSU’s research
contributed to Innovation?
| 18 | 18
Patent to Article citations for PSU
Overview Module
| 19 | 19
Patents citing PSU scientific papers Overview Module
| 20 | 20
6.) How does PSU’s publication
output compare to other Thai
Institutions?
| 21 | 21
Top 15 Institutions in Thailand
| 22 | 22
7.) Who are the key collaboration
partners of PSU?
| 23 | 23
PSU top collaboration partners
| 24 | 24
8.) How much does PSU collaborate with
Industry?
| 25 | 25
PSU top corporate collaboration partners
| 26 | 26
8.) How well does PSU perform in
“Rubber” research?
| 27 | 27
PSU in “Rubber” Research
| 28 | 28
9.) Who are the top performing
institutions and researchers “Rubber”
research in the World?
| 29 | 29
Top Rubber Research Institutes in the World
| 30 | 30
Research Metrics
| 31 | 31
Elsevier’s metrics approach in a nutshell
Figure 1 of http://www.elsevier.com/online-tools/research-intelligence/resource-library/resources/response-to-hefces-call-for-evidence-independent-review-of-the-role-of-metrics-in-research-assessment
| 32 | 32
Metrics
Productivity metrics Scholarly Output h-indices (h, g, m)
Citation Impact metrics Citation Count Citations per Publication Cited Publications h-indices (h, g, m) Field-Weighted Citation Impact Publications in Top Percentiles Publications in Top Journal Percentiles Collaboration Impact (geographical) Academic-Corporate Collaboration Impact
Views metrics Views Views per publication Field-Weighed Views Impact
Collaboration metrics Authorship Count Number of Citing Countries Collaboration (geographical) Academic-Corporate Collaboration Disciplinarity metrics Journal count Journal category count
Economic Impact metrics Citing Patents Patent-Cited Scholarly Output Patent-Citations Count Patent-Citations per Scholarly Output
Societal Impact metrics Mass Media Mentions Media Exposure Field-Weighted Mass Media Input metrics Awarded Grants
Journal Metrics CiteScore SJR SNIP
| 33 | 33
When used correctly, research metrics together with qualitative input give a balanced, multi-dimensional view for decision-making
Two Golden Rules for using research metrics
Always use both qualitative and quantitative input into
your decisions
Always use more than one research metric as the
quantitative input
| 35 | 35
Example: importance of using multiple metrics from
the basket - compensate for weaknesses
35
Field-Weighted Citation Impact = 2.53
Compensates for differences in field, type and age
Meaningful benchmark is “built in” – 1 is average for a subject area
× People may not like small numbers × Complicated; difficult to validate × No idea of magnitude: how many
citations does it represent?
Citations per Publication = 27.8
with
Large number Simple, easy to validate Communicates magnitude of
activity
× Affected by differences in field, type and age
× Meaningless without additional benchmarking
| 36 | 36
Non-Performance Variables
| 37 | 37
When making comparisons, we must take into
consideration “Non-performance variables”
Size
Discipline
Publication-type
Database coverage
Manipulation
Time
In some cases the difference in a metric value between two entities (authors for example) might not reflect a difference in performance. Instead the difference can be caused by non-performance variables.
| 38 | 38
Publications types receive differing levels of citations
Because some publication types are cited more often than others, we should not compare different types without applying normalization.
| 39 | 39
Citation practices differ between disciplines
Because some subject areas are cited more often than others, we should not compare papers from different fields without applying normalization.
| 40 | 40
Size and Time
Citation count is directly related to
volume
CPP normalizes for differences in
publication volume
FWCI normalizes for subject area,
document type, and time
Do you think the declining trend is indicative of a
decrease in quality or is it caused by a non-
performance variable?
| 41 | 41
Please don’t be shy…
| 42 | 42
CiteScore
| 43 | 43
Serial title metrics are important!
• Serial title metrics are still important complements to new and alternative metrics
• Many serial titles are missing transparent and replicable
metrics that are easy to access • SNIP & SJR are complex and difficult to replicate • We have introduced CiteScore to compliment them
| 44 | 44
Filling the gap in the Scopus basket of journal metrics
44
SNIP and SJR
Compensates for differences in field, type and age
Meaningful benchmark is “built in” – 1 is average for a subject area
× People may not like small numbers × Complicated; difficult to validate × No idea of magnitude: how many
citations does it represent?
CiteScore and associated metrics
with
Large number Simple, easy to validate Communicates magnitude of
activity
× Affected by differences in field, type and age
× Meaningless without additional benchmarking
| 45 | 45
A
CiteScore is a simple metric for all Scopus serial titles
B
CiteScore Impact Factor A = citations to 3 years of documents
A = citations to 2 or 5 years of documents
B = all documents indexed in Scopus, same as A
B = only citable items (articles and reviews), different from A
Note: at launch, all serial titles in the May 2016 title list, and with some documents indexed in 2016, will have CiteScore metrics
CiteScore 2015 value B
=
A
| 46 | 46
CiteScore is one of a family of related metrics,
available for FREE.
| 47 | 47
Journalmetrics.scopus.com website
Static values 2011-2015 for reporting, showcasing and exporting
| 48 | 48
| 49 | 49
CiteScore 2015 correlates 75% with Impact Factor
R² = 0.7524
-20
0
20
40
60
80
100
120
140
0 10 20 30 40 50 60 70
Im
pact
Fac
tor 2
015
CiteScore 2015
2015 Impact Factor and 2015 CiteScore
| 50 | 50
Serials with CiteScore only cover all levels of performance
0
50
100
150
200
250
300
350
400
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69 71 73 75 77 79 81 83 85 87 89 91 93 95 97 99
All CS %iles JIF %iles Unique CS %iles HIGH LOW
Num
ber o
f doc
umen
ts
• 22,256 serial titles have CiteScore 2015 • 22,620 serial titles have CiteScore Tracker 2016
| 51 | 51
Comprehensive
• Based on Scopus, the
world’s broadest database
• A CiteScore will be available for all serials, not just journals
• CiteScore can be calculated for portfolios
Transparent
• CiteScore and
associated metrics will be available for free
• CiteScore is easy to calculate for yourself
• The underlying database is available for you to interrogate
Current
• CiteScore Tracker is
updated monthly
• New serial titles will have CiteScore metrics the year after they are indexed in Scopus
Main advantages of CiteScore
| 52 | 52
Part II Hands on SciVal Training
| 53 | 53
53
SciVal offers quick, easy access to the research performance of 220 nations and 7500 research institutions worldwide (including 774 corporate organizations)
SciVal is Elsevier’s Research Intelligence Tool.
Data updated weekly from Scopus
| 54 | 54
Scopus is the main data source for SciVal. It includes content from
more than 5,000 publishers and 105 different countries
21,568 peer-reviewed journals 361 trade journals • Full metadata, abstracts
and cited references (ref’s post-1995 only)
• Funding data from acknowledgements
• Citations back to 1970
Physical Sciences 7,443 Health Sciences 6,795 Social Sciences 8,086 Life Sciences 4,492
90K conference events 7.3M conference papers Mainly Engineering and Computer Sciences
531 book series 30K Volumes / 1.2M items 119,882 stand-alone books 974K items
Focus on Social Sciences and A&H
62M records from 22K serials, 90K conferences and 120K books • Updated daily • Records back to 1823 • “Articles in Press” from > 3,750 titles • 40 different languages covered • 3,715 active Gold Open Access journals indexed
BOOKS CONFERENCES
Source: November 2015 title list at https://www.elsevier.com/solutions/scopus/content
JOURNALS
27M patents From 5 major patent offices - WIPO - EPO - USPTO - JPO - UK IPO
PATENTS
| 55 | 55
| 56 | 56
Accessing SciVal
| 57 | 57
Accessing SciVal: go to www.scival.com
Who does not have access to SciVal? Please raise your hand.
If you do not have a username and password for SciVal, then please
register here.
| 58 | 58
Master List Creating Organization Structure in SciVal
| 59 | 59
Master List of Researchers
| 60 | 60
| 61 | 61
Quick Overview of SciVal
| 62 | 62
SciVal Modules
Overview module provides dashboard style reports based on a 3 or 5 year period. You can analyze countries, institutions, any researcher, group of researchers and research areas you have created. It gives a comprehensive overview of one entity at a time.
Benchmarking module is flexible, it allows you to select a time period anywhere from 1996 to one month ago. You can choose from over 17 metrics, and compare across different types of entities (i.e. you directly can compare researchers with departments, institutions, countries in one analysis)
Collaboration module looks at specifically at co-publication trends. It allows you to identify the top collaboration partners of an institution or country, as well as identify potential collaboration partners. Research areas can be applied here as a subject filter.
| 63 | 63
| 64 | 64
Assignments
| 65 | 65
Assignment 1
1.) What data is SciVal based on? a.) Custom dataset b.) Web of Science c.) Scopus 2.) How many papers did PSU produce in Medicine in 2011? a.) 105 b.) 159 c.) 125 3.) Who produced the most papers in “Energy” research in 2015: a.) Khon Kaen University b.) Prince of Songkla University c.) The both produced the same number of papers in Energy 4.) How many citations did Prince of Songkla receive in “Engineering” (2011-2015)? a.) 25% more than world average b.) 5% less than world average c.) same as world average 5.) Which GOVERNMENT institution did PSU co-author with most often (2011-2015)? a.) Armed Forced Research Institute of Medical Sciences, Thailand b.) National Cancer Institute, Thailand c.) Thailand Ministry of Public Health
| 66 | 66
Answers 1
1.) What data is SciVal based on? a.) Custom dataset b.) Web of Science c.) Scopus 2.) How many papers did PSU produce in Medicine in 2011? a.) 105 b.) 159 c.) 125 3.) Who produced the most papers in “Energy” research in 2015: a.) Khon Kaen University b.) Prince of Songkla University c.) The both produced the same number of papers in Energy 4.) How many citations did Prince of Songkla receive in “Engineering” (2011-2015)? a.) 25% more than world average b.) 5% less than world average c.) same as world average 5.) Which GOVERNMENT institution did PSU co-author with most often (2011-2015)? a.) Armed Forced Research Institute of Medical Sciences, Thailand b.) National Cancer Institute, Thailand c.) Thailand Ministry of Public Health
| 67 | 67
Assignment 2
1.) What percentage of PSU papers were published in the TOP 10% best journals based on CiteScore journal metric (2011-2015)? a.) 17.1% b.) 18.7% c.) 180 2.) How many research articles did PSU produce in Environmental Science in 2013? a.) 39 b.) 44 c.) 222 3.) What percentage of PSU papers published in 2011 in Medicine are cited often enough to belong to the top 1% highest cited papers world-wide, excluding self-citations? a.) 1.9% b.) 1.0% c.) 1.1%
| 68 | 68
Answers 2
1.) What percentage of PSU papers were published in the TOP 10% best journals based on CiteScore journal metric (2011-2015)? a.) 17.1% b.) 18.7% c.) 180 2.) How many research articles did PSU produce in Environmental Science in 2013? a.) 39 b.) 44 c.) 222 3.) What percentage of PSU papers published in 2011 in Medicine are cited often enough to belong to the top 1% highest cited papers world-wide, excluding self-citations? a.) 1.9% b.) 1.0% c.) 1.1%
| 69 | 69
Assignment 3
Create a Research Group for PSU Faculty of Engineering. You can download the sheet here: https://www.dropbox.com/s/5f7sdnx0pnuyqcw/PSU%20Faculty%20of%20Engineering.xlsx?dl=0
| 70 | 70
www.elsevier.com/research-intelligence
Thanks for your attention Alexander van Servellen [email protected]
| 71 | 71
Questions
1.) What data is SciVal based on? a.) Custom dataset b.) Web of Science c.) Scopus 2.) How many papers did UI produce in Medicine in 2011? a.) 105 b.) 95 c.) 125 3.) Who produced the most papers in “Materials Science” in 2014: a.) Universitas Indonesia b.) Institut Pertanian Bogor c.) Gadjah Mada University 4.) How many citations did Indonesia receive in Medicine (2011-2015)? a.) 43% more than world average b.) 5% less than world average c.) same as world average 5.) Which corporate institution did UI co-author with most often (2011-2015)? a.) Samsung b.) Unilever c.) RAND
| 72 | 72
Comparison with nearest peer
Scopus 22,245
Web of Science 12,140
Scopus 7,443 (+73%)
WoS 4,291
Scopus 6,795 (+96%)
WoS 3,472
Scopus 4,492 (+50%)
WoS 3,002
Scopus 8,086 (+99%)
WoS 4,060
Physical Sciences Health Sciences Life Sciences Social Sciences
~12K titles (Core Collection) 3,300 publishers Updated weekly
~22K titles >5,000 publishers Updated daily
Source: Web of Science Real Facts, Web of Science title list and Scopus’ own data (April 2015)
| 73 | 73 | 73
Scopus
Competitor
0
2,000
4,000
6,000
8,000
0
200
400
600
800
+160%
+266%
0
2,000
4,000
6,000
8,000
10,000
12,000 +168%
0
500
1,000
1,500
2,000
2,500
Asia Pacific
+333%
Eastern Europe incl Russia
0
500
1,000
1,500
0
200
400
600
800
1000
+305% Latin America
Middle East & Africa
Western Europe
0
100
200
300
400
500
+248%
+252%
North America
Australia/New Zealand
What does Scopus’s content advantage mean for
emerging countries?