A CRIS driven by research community:

benefits and perspectives

Sergey Parinov,

CEMI RAS, Moscow, Russia

euroCRIS DRIS-BP Task Group Leader

Modern challenges of the Science System

• In Russia, the same as in many other countries, government and funding organizations would like to improve science regulations:

• 2006 - a registry of intellectual property created by scientists on money from public budget (where the state has a share)

• 2010 – a new approach to assess scientific organizations outputs and new rules to finance research from public budget

• But the efficiency of the new regulations is bottled up by some shortcomings of experts panels and existed bibliometric on research performance (e.g. citation statistics)

Research information systems and regulations of the Science

• Many countries have created regulatory RIS and will improve it, other will build it from the scratch

• In Russian Academy of Sciences (RAS) from 2010 is working a RIS “Results of Intellectual Activity” (RIA) of 400 institutions of RAS with 30000 people in staff

• Main RIA RAS goals: to provide new research performance statistics and quality research evaluation to improve funding rules according new science regulators

• In this year CEMI RAS is one of main RIA performers• One of our aims is a concept of virtual research

environment that removes shortcomings and increases the RIA RAS efficiency

Our initial view on virtual research environment

• A combination, at least, of two types of RIS:– RIS working to support science regulators (like RIA

RAS), evaluations, management (organizational CRIS), etc. and driven by some organizations

– RIS working to support decentralized professional interactions and driven by a research community (like Socionet and many other)

• Data exchange between two types of RIS, e.g.– The first (regulatory) RIS can motivate researcher to

use the second (community) one– The 2nd (community) RIS can create some data and

conditions which will be used by the first RIS to realize the regulators more efficiently

Known weaknesses and shortcomings

• No transparent and public quality data on research performance– Traditional citation indexes within an article do

not consider variations in cited scientific objects and types of its usage

• Experts panels make not transparent and often not predictable decisions

• Regulatory RIS are national, but research performance has international nature

CRIS driven by research community: to compliment regulatory RIS

• We have created an initial concept of the CRIS and starting its pilot implementation

• This CRIS will provide– for scientists - new tools/services, scientific

circulation mechanism – for regulatory RIS - new scientomentrics

• Using new scientometrics in regulatory RIS for research assessments will motivate researcher to use community CRIS

The main idea• To exclude human experts, where it’s possible,

from creating research performance statistics by– giving researchers a method to register research

results as computer readable scientific Objects-for-Reuse (OfR)

– organizing a scientific circulation of OfR including methods for its using by research community in a computer readable form

– running automated monitoring services which forming/updating statistic of research producing/using

• The statistics have to be produced by the Science System, not by humans

Main elements of the proposed system

1. CRIS/CERIF model to build a core of virtual research environment driven by community of scientists

2. Establishing of scientific Objects-for-Reuse (OfR) and its circulation mechanism

3. Automated monitoring over virtual research environment and personalized notifications

4. Scientometrics database creating by monitoring

5. New types of research performance indicators

Objects-for-reuse: basic features

• A definition: any fragment of research article which has cited or can be cited by scientists in their research articles

• As an object of scientific electronic environment OfR exist only in electronic form

• OfR is created by an author (as a granulation of articles) or any researcher using electronic citation utility within depositing tools

Objects-for-reuse: fields

• OfR is specified by a set of fields:– A fragment of text (a text of the citation)– A title of this object– A source of the citation– Keywords, classification codes for areas

where it can be used and usage notes– Linkages with related objects different types

Objects-for-reuse: semantic of linkages

• Semantic linkages can specify:– Types of scientific usage of cited object– Types of scientific inference relation with

existed scientific knowledge– Types of technical relations with other objects

• If other metrics are contributed by researchers it can be added to the system

Current “usage” metrics

• (1) the cited research result (or just a citation) is a basement for author’s output;

• (2) the citation approves (or is approved by) author’s output;

• (3) the citation illustrates of (or has another logical connection with) author’s output;

• (4) the citation is wrong or disproved by author’s output.

“A needle is made of steel, and magnets attract steel”

Source: Barend Mons and Jan Velterop. Nano-publicationfor community annotation and Interoperability

Objects-for-reuse: semantic networks

• At least three types of semantic networks arise within scientific information space– Usage linkages (what, by whom, in what

sense were used in research)– Scientific inference linkages (what scientific

statement s are the basis, with what logical relation)

– Other types of relations (language versions, etc.)

How to use objects-for-reuse

• Browse by disciplines - thematic collections – titles and search by keywords

• Navigate by different types networks of semantic linkages

• Select the best object-for-reuse for citing in your article among similar (a competition …)

• Easy link it with your article/material and specify usage quality characteristics

Monitoring of virtual research environment

• Automated monitoring of all changes in structure of information objects and linkages between them– Notifications of researchers about important

changes (improve scientific circulation)– Accumulating of scientometric database

(improve statistic on research performance)

Notification system informs:

• the author who is changing his/her article, if the article has cited in other articles, that she/he can violate (by this action) links and citations that have established with the changed fragments of the article;

• the authors of articles, if their articles include citation links to the changed article, about a fact of made changes in the cited article, so they should reconsider and, if it necessary, to correct corresponding citations;

• the readers of electronic article that certain citations in reading text can be violated because of the cited articles were changed, and an author of the reading article has not updated suspicious citations.

Scientometrics produce:1. quantitative characteristics of research players and results of their activity

presented by CERIF style organized information objects;

2. quantitative data about all existed relations between information objects, e.g. number of persons linked with organization unit, number of publications linked with a person, number of citations linked with a publication, and so on;

3. qualitative data about all existed relations between information objects, like a graph topology of linkages and semantic values assigned to each edge of the graph, e.g. a set of relations with the semantic value "member of staff" between an organization of unit and persons; a set of relations with the semantic value "basement" between a publication and objects-for-reuse; and so on;

4. data about views/downloads aggregated for each information objects according linkages, e.g. numbers of views/downloads for all publications related with a person or a sum of these numbers for all persons related with an organization unit and so on.

Statistical portrait includes: • Personal data or a researcher's profile presented as a complex

information object of DIS with existed network of semantic linkages to other information objects (organizations, research results in different forms, events, etc.), including a history of changes of this data.

• Growth characteristics of this complex information objects as time series, e.g. numbers of produced by this person research results (new objects-for-reuse/papers/articles/patents, etc.), and also a numbers of usage facts for the results (like a traditional citation index).

• Usage activity characteristics of a researcher as a number of citations made by the researcher, including a distribution of usage characteristics, which were specified by the researcher as quality attributes of have made citations.

• Usage/impact characteristics for researcher's outputs/results built as a distribution of quality attributes for citations made by other scientists for researcher’s materials.

Conclusion

• Proposed virtual research environment will provide transparent and public data for regulatory RIS:– How many OfR currently exist for selected

article and/or for selected author, project, organizations and its dynamic

– What usage characteristics currently exist for selected OfR, authors, etc, and how it has been changing in time

Conclusion

• Regulatory RIS will form data for research assessment, including its performance, in terms of – Research results (not just articles)– Usage characteristics as an indicator of

quality– Relations between research results as an

indicator of impact

Conclusion

• Many national regulatory RIS need better data about research performance, it is a new challenge for CRIS community

• Someone should design/support a concept and provide an approach to develop virtual research environment

• Some international organization should provide an expertise, coordination and best practice

A message to RIS professional community

• To be efficient regulatory RIS has to be complemented by a specially designed international virtual research environment

• The virtual research environment should be design as a single and unified system for international community of scientists

• Development of such environment should be coordinated by some international body, e.g. euroCRIS

A message (2)

• European Commission is one of many who can benefit from such virtual research environment

• National funding organizations also can support this innovations in the Science System

• What should we do to respond on the challenges?

Benefits and perspectives for the Science System

1. Visualizing in computer readable form of scientific knowledge network (research results as nodes and existed relations between them) and its development

2. Visualizing of established scientific inference network (deduction relations between research results) and its development

3. Visualizing of research results usage types and its characteristics (what, how and by whom are used)

4. Monitoring of all changes over knowledge network and personalized notifications about its important events

5. Collecting of all scientometric data and forming of open professional scientific “signaling system”

6. Building of better research performance indices and using it for researchers motivation