monitoring and assessing research performance€¦ · dr. walid hassan research consultancy...
Post on 07-Nov-2020
3 Views
Preview:
TRANSCRIPT
November 2017
Monitoring and assessing research performance
November 2017
On ResearchBenchmarking and Valuation (Translation)
Dr. Walid HASSANResearch Consultancy Services Manager
The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man”..George Bernard Shaw, Man and Superman (1903) "Maxims for Revolutionists"
4
CLARIVATE ANALYTICS: A GLOBAL FOOTPRINT
1. With more than 4,200 staff worldwide, Clarivatesupports customers through curated editorialcontent, leading professional services, customproduct and service development and muchmore.
2. Over 7,000 leading government and academicinstitutions, and research intensive corporationsuse the Web of Science for its 100 years worth ofresearch.
3. Headquartered in the US, we are a global,information-led company operating in over 100countries around the world.
5
BIG DATA CAPABILITIES
THOMSON INNOVATION
• DWPI – world’s most comprehensive patent data set
• 4.5 Release and Smart Search – find what your are looking for quickly, easily
• ThemeScape – easily visualize how competitor portfolios overlap
6
OUR PEOPLE: CONNECTING ACROSS THE GLOBE
7
Content
� Top 2016 Discoveries
� Situation Analysis- Brazil
� Back to basics, Challenges
� On R&D Strategies
� Benchmarking
� Case Studies
8
Top 2016 DiscoveriesSlight Genetic Mutation 800 Million Years Ago Leads To Multicellular Life
Researchers found that an ancientmolecule, GK-PID, is the reason single-celled organisms started to evolve intomulticellular organisms approximately 800million years ago . The molecule hasbeen found to be like a molecularcarabiner able to pull chromosomestogether to latch them onto the inner wallof a cell membrane when division occurs.This allows cells to copy properly andavoid becoming cancerous.
Photo credit: TenOfAllTrades /Wikimedia
9
A Ninth Planet Was Discovered In The Solar System
California Institute of Technologypresented evidence that a ninth planettruly does exist with an orbital period of15,000 years.The astronomers who published theirdiscovery have calculated that there is“only a 0.007 percent chance, or aboutone in 15,000, that the clustering could bea coincidence.” Presently, Planet Nineremains hypothetical, but astronomershave calculated its orbit to be quitemassive. If it does exist, the planet wouldlikely be approximately 2–15 times themass of Earth and orbit between 200 and1,600 Astronomical Units (AU) from theSun. An AU is 150,000,000 kilometers,which means that the planet could orbit asfar from the Sun as 240,000,000,000kilometers.
Photo credit: Wikimedia
Top 2016 Discoveries
10
An Eternal 5D optical data Storage
University of Southampton: Scientists havesuccessfully used nano-structured glass to create aprocess for recording and retrieving data. The storagedevice is a small glass disk about the size of anAmerican quarter that can hold 360TB of data andremain intact up to 1,000°C. This means that itsaverage shelf life when held at room temperaturewould be approximately 13.8 billion years (Roughlythe same amount of time the universe has existed).
Photo credit: southampton.ac.uk
Top 2016 Discoveries
11
Stem Cells Injected Into Stroke Patients Re-Enable Patient To Walk
A clinical trial held at StanfordUniversity School of Medicineinjected modified human stem cellsdirectly into the brains of severalchronic stroke patients. Theprocedures were all successful withno negative effects described fromthe injection and only mildheadaches as a result of theprocedure, which was performedon mildly anesthetized patients. All18 showed significant healing longafter any healing is expectedfollowing a stroke (a period of sixmonths)Photo credit: Nissim Benvenisty
Top 2016 Discoveries
12
Dr. Walid Hassan
11/13/2017
Private Company SpaceX Successfully Lands A Rocket Vertically
Top 2016 Discoveries
13Where We Are
14How We Look
15Who?
16How? (% of Documents Cited)
17How? (% of Hot Papers)
Scope
19Who is concerned?
Research Process
PhD Students/Res
earchers
Research Managers`
Decision Makers
� Researchers: PhD Students and Researchers� Research Managers: Research unit managers, department managers, etc.. � Decision makers: Research Laboratory Managers, Institution Management, etc..
20Challenges and Repartition
The Start Point
Researchers are challenged at the first moment, when defining their research subject area
22
Topics Strategy
35 USD billions has been wasted developingthings that are already documented in a patentspecification .(Source: British Patent Office,2000)
23Patents Versus Publications
Publications
Patents
R&D STRATEGY
25
—TYPES OF RESEARCH
26
—PATHWAY FROM RESEARCH TO IP, STRATEGY
Fundamental
Publications
Citations, Visibility
Research Community (Academia )
Applied
Patents
Licensing
Industry
Theories, methodologies
Applications
IPKnow HowV
alua
tion
27
Positioning of needs: University
28
University: Opportunities and Challenges
PUBLISH, READ AND CITE
30
Publish or Perish
Publish ………. or Perish
They Say:
Something is Missing !!!!!!!!
31
Publishing: Problem Positioning
32
Today: What to Read
33
Contradictions?
Need a Benchmark
Research
papers?
Need a Model
34
EUGENE GARFIELD’S “ASSOCIATION OF IDEAS INDEX”
A benchmark..
A Model
35
Challenges
Why to publish?
Where ?
When ?
How?
With Whom?
36
Why we Publish
Diffusion of Knowledge
Promotion
Sharing Ideas
Community belonging
Research Valorization
Getting Feedback
Professional Profile
37
Where to Publish
• Which journal? Indexed?
• How journals are classified? Which is better?
• Should I look at the Impact factor?
• Would my publications be accepted in high-IF journals?
• Open Access?
BENCHMARKING
The main Pillar of Strategic Planning
39
BENCHMARKINGBENCHMARKING
Strategizing the Step from A to B
40BENCHMARKING IN RESEARCH
41GOALS AND OBJECTIVESIn
pu
ts Human Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
42KPI’s
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
Where is our
funding coming
from?
How much
funding do we
receive from
various
sources?
Who are the Who are the leaders in interested
fields?
What connections can we take
advantage of to recruit top
talent?
How much
research do we
disseminate?
What
publications
channels do we
use most
often?
How do we
compare to
similar
institutions?
How are we
perceived by
the research
community?
43PLANNING
ESTOU FAZENDO ÀS COISAS CERTAS?
• Theory Evaluations
• Mission statement development
• Needs assessments
• Strategic planning
ESTOU FAZENDO ÀS COISAS CERTAS CERTO?
• Process Evaluations
• Regular progress reports
• Monitoring databases and user interface systems
• Resource usage tracking
ESTOU FAZENDO O SUFICIENTE DAS COISAS CERTAS PARA FAZER A DIFERENÇA?
• Outcome Evaluations
• Output tracking
• Performance benchmarking
• Longitudinal attribution
44
Who are the stakeholders?
Executives
Researchers
Funders
Partners
Public
What domains are being addressed?
Need
Design
Implementation
Impact and outcome
Efficiency
How is the information presented?
Summary
Detailed report
Presentation
Infographic
Campaign
What are the expected returns?
Awareness
Advocacy
Accountability
Action
Assets
CAPITALISE
45
CAPES
46Where We Are
47Organizations
48
Não é possível exibir esta imagem no momento.
Organizations (Documents in Top 1%)
Dr. Eng. Walid HassanResearch Consultancy Services Manager
Email: walid.hassan@clarivate.com
Twitter: @Walid_Hassan_TR
Linked In: Walid Hassan, PhD
50
Case Study: University X (Optional)
Overview of a Brazilian University’s
research performance.
52
Evaluation of
University X’s
Research Activity,
2000-2016
Use case: exploring how bibliometric data can indicate institutional performance
across multiple components of a university research activity framework.
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
53
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
54
Funding
55
Acknowledged
Funding
(2008-2016)
56
Acknowledged
Funding
(2008-2016)
CNPq and CAPES excluded
57
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
58
Partnerships
59
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
60
Publication output
by year
Total (2000-2016): 142,077 publications
61
Publication output
by CAPES area
6 - Health Sciences, 41.0%
3 - Biological Sciences, 27.2%
5 - Exact and Earth Sciences, 25.8%
4 - Engineering, 12.2%
1 - Agricultural Sciences, 9.1%
9 - Multidisciplinary, 9.1%
2 - Applied Social Sciences, 3.7%
7 - Humanities and Social Sciences, 2.4%
8 - Linguistics, Literature and Arts, 0.2%
Note: Does not sum to 100% due to
publications in multiple categories
62
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
63
Collaborations
64
Brazilian
Collaborations
65
International
Collaborations
66
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
67
Impact as
Citation counts
68
Times Cited
and
Document count
69
Times Cited
and
Normalized Citation
Impact
70
Scientific ImpactUsing normalized citation measures
to indicate the impact of inputs, activities, and outputs
Example: Category Normalized Citation Impact
Where:
e = the expected citation rate or baseline,
c = Times Cited,
p = the number of papers,
f = the field or subject area,
t = year,
d = document type,
n = the number of subjects a paper is assigned to and
i = the entity being evaluated (institution, country, person, etc).
The Category Normalized Citation Impact (CNCI) of a document is calculated by dividing the actual
count of citing items by the expected citation rate for documents with the same document type, year
of publication and subject area.
CNCI
0,5
0,6
0,7
0,8
0,9
1
1,1
World Avg, 1.0
University X, 0.88
71
Acknowledged
Funding
(2008-2016)
CNPq and CAPES excluded
US DOE
US NIH US NSF
72
Acknowledged
Funding
(2008-2016)
CAPESNCPq
73
Output and Impact
by CAPES category6 – Health Sciences
5 – Exact and Earth Sciences
7 – Humanities and Social Sciences
74
Sub-analysis of
CAPES category
“Exact and Earth
Science”
75
Sub-analysis of
CAPES category
“Humanities
and
Social Sciences”
76
Collaborations,
Country-level
USA
Netherlands
77
Percentage of
papers with
international
collaborations by
year
Total percentage
2000-2016: 29.72%
78
Journal analysis
using citation
impact normalized
to the journal
79
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
80
Evaluation of
University X’s
Research Activity
Inp
uts Human
Resources
Funding
Facilities
Administration
Social Capital
Act
ivit
ies Research
Education & Training
Partnership
Recruitment
Thought Leadership
Ou
tpu
ts Publications
Graduates
Collaborations
Innovation
New Resources
Ou
tco
me
s Scientific Impact
Societal Impact
International Reputation
Increased Funding
81
Case Study: The Impact Factor and it’s
Usage
(Optional)
82
The Impact FactorThe impact factor is a measure of the frequency with which the average
article in a journal has been cited in a particular year. The JCR also lists
journals and their impact factors and ranking in the context of their
specific field(s).
83
Citations and IF variations
Dr. Walid Hassan
11/13/2017
84
Categories/Quartiles
- J1 IF1
- J2 IF2
- J3 IF3
- J4 IF4
- J5 IF5
- J6 IF6
- J7 IF7
- ------------
25%- Q1
25%- Q2
25%- Q3
25%- Q4
Q1
Q1
85
Case Study: Building a PhD Subject
(Optional)
86
—SELECTING THE RIGHT RESEARCH TOPIC
Example: Image Reconstruction
87
—TREND (NUMBER OF DOCUMENTS PER YEAR IN WOS)
88
—NORMALIZED CITATION IMPACT
89
—PATENT ANALYSIS: IMAGE RECONSTRUCTION
90
—THEMESCAPE (DERWENT INNOVATION)
Red Dots: Alive Patents
91
WEB OF SCIENCE (WOS) DATABASES
SCIENCE CITATION INDEX EXPANDED
•41.2 million records
•1900-present
SOCIAL SCIENCES CITATION INDEX
•7.9 million records
•1900-present
ARTS & HUMANITIES CITATION INDEX
•4.6 million records
•1975-present
CONFERENCE PROCEEDINGS CITATION INDEX
•8.2 million records
•1990-present
BOOK CITATION INDEX
•977,199 records
•2005-present
CURRENT CHEMICAL REACTIONS
•212,622 records
•1985-present
INDEX CHEMICUS
•337,305 records
•1993-present
DATA CITATION INDEX
•2.8 million records
•1900-present
CURRENT CONTENTS CONNECT
•18.1 million records
•1998-present
BIOSIS CITATION INDEX
•22.8 million records
•1926-present
ZOOLOGICAL RECORD
•4 million records
•1864-present
MEDLINE
•22 million records
•1950-present
DERWENT INNOVATION INDEX
•22.9 million basic inventions
•1963-present
A comprehensive gateway to the most relevant research information: reliable, integrated, multidisciplinary
top related