Measuring and Characterizing Indian Research

Dr Vivek Kumar SinghProfessor

Department of Computer ScienceBanaras Hindu University, Varanasi, India

http://www.viveksingh.in

Outline

• Facts & Statistics

• Disciplinary Distribution

• Institutional Rankings

Measuring Indian Research

• University-Industry-Government Collaboration

• Open Access Levels & Patterns

• Gender Distribution

Characterizing Indian Research

• Social Media Attention

• Sciento-text Framework

• Inferences & Implications

New Dimensions

Measuring Indian Research

Research in India

• Diverse Institutional Setup-

– Multidisciplinary Universities

– Discipline Specific Institutions

– Government Research Labs

– Private Organizations

• Top 100 most productive institutions contributemore than 40% of research output.

3.38 3.423.54 3.58

3.71

4.56

4.925.20

5.03

5.52

2.88 2.973.10 3.14 3.07

0.00

1.00

2.00

3.00

4.00

5.00

6.00

2014 2015 2016 2017 2018

PER

CEN

TAG

E SH

AR

E

YEAR

WoS Scopus Dimensions

India’s Contribution to Global Research Output

Rank Web of Science Scopus Dimensions

Country Output Country Output Country Output

1

USA 3,092,011 United States 3,408,827

United

States 3,443,870

2China 1,694,149 China 2,590,897 China 2,048,128

3 United

Kingdom 776,071

United

Kingdom 1,035872

United

Kingdom 1,038,954

4Germany 654,978 Germany 897,212 Germany 967.016

5 Japan 501,847 India 760,030 Japan 788,499

6 Canada 463,470 Japan 658,507 India 679,221

7 Italy 449,724 France 614,615 France 616,690

8 France 446,878 Italy 577,561 Italy 546,944

9 Australia 422,881 Canada 541,888 Canada 545,721

10 India 379,398 Australia 499,074 Australia 487,143

Top 10 countries by Research Output

Disciplinary Distribution of Indian Research

CS & IT Research in India

• Analysis of 2010-18 research output data taken from Web of Science and Scopus.

[Ref: A. Uddin & V.K. Singh, “A Quantity-Quality Composite Ranking of Indian Institutions inComputer Science Research”, IETE Technical Review, Vol. 32, No. 4, pp. 273-283. Taylor & Francis]

Institutional Research Performance Assessments

• Only few Indian institutions figure in global research rankings

• ARWU 2019: 11 Indian institutions in 1000.

• QS 2019: 24 Indian institutions in 1000

• THE 2019: 49 Indian institutions in 1258.

• Need for national rankings and assessments.

• NIRF is not a research ranking.

[Ref: A. Basu, S.K. Banshal, K. Singhal & V.K. Singh, “Designing a Composite Index for ResearchPerformance Evaluation at the National or Regional Level: Ranking Central Universities in India”,Scientometrics, Vol. 107, No. 3, pp. 1171-1193, June 2016]

• Data for 39 CUs for 1990-2014.

• Combined Output of CUs is less than Cambridge/ Stanford.

Institutional Research Performance Assessments- Central Universities

[Ref: Marisha, S.K. Banshal & V.K. Singh, “Research Performance of Central Universities in India”,Current Science, Vol. 112, No. 11, pp. 2198-2207, June 2017]

Institutional Research Performance Assessments- IITs

[Ref: S.K. Banshal, V.K. Singh, A. Basu & P.K. Muhuri, “Research performance of the IndianInstitutes of Technology”, Current Science, Vol. 112, No. 5, pp. 923-932, March 2017]

Institution TP TC ACPP HiCP ICP h-IndexPapers Per

Capita

Old IITs

IIT Kharagpur 5871 26991 4.6 35 1394 43 10.1

IIT Madras 4905 20413 4.2 42 1396 44 9.7

IIT Bombay 4680 24118 5.2 85 1470 50 8.3

IIT Delhi 4574 20480 4.5 31 1198 38 10.8

IIT Kanpur 3789 17135 4.5 25 1008 35 10.1

IIT Roorkee 3452 16330 4.7 39 805 39 7.6

IIT Guwahati 1754 9032 5.1 19 472 34 4.9

New IITs

IIT Varanasi 1443 2110 1.462 11 410 19 6.2

IIT Hyderabad 501 1545 3.084 7 165 15 3.7

IIT Indore 389 1950 5.013 19 161 22 5.1

IIT Ropar 304 1446 4.757 5 99 17 5.6

IIT Mandi 152 271 1.783 1 73 9 2.5

IIT Gandhinagar 147 382 2.599 4 65 9 1.7

IIT Bhubaneswar 141 422 2.993 0 41 11 2.4

IIT Patna 119 418 3.513 1 19 10 1.6

IIT Jodhpur 87 165 1.897 0 31 6 2.4

• Most productive IIT is 5 times less productivethan MIT and 3 times less productive than NTU.

Institutional Research Performance Assessments- IITs

• Data for 31NITs for 2005-16.

• NITscontributeabout 3% tototal researchoutput ofIndia.

Institutional Research Performance Assessments- NITs

[Ref: S.K. Banshal, T. Solanki & V.K., “Research Performance of the National Institutes ofTechnology in India”, Current Science, Vol. 115, no. 11, pp. 2025-2036, 2018]

• Data for 5 IISERsfor 2010-14.

• Percentagecontribution toIndian researchOutputincreased from0.46% in 2010to 1.25% in2014.

Institutional Research Performance Assessments- IISERs

[Ref: T. Solanki, A. Uddin & V.K. Singh, “Research Competitiveness of Indian Institutes of ScienceEducation & Research (IISERs)”, Current Science, Vol. 110, No. 3, pp. 307-310, Feb. 2016]

• Data for 2010-16.

• 40 Private Universitiesproduce 31,675 papers.

• Pvt Univ set contributes5.1% to India’s 1% HiCPPas compared to IITs(15.5%), CUs(12.6%) andNITs (2.7%).

• In others, CSIR~6.5%,ICAR~3.5%, DRDO~1%.

Institutional Research Performance Assessments- Private Universities

[Ref: S.K. Banshal, V.K. Singh & P. Mayr “Comparing Research Performance of PrivateUniversities in India with IITs, Central Universities and NITs”, Current Science, Vol. 116, No. 8, pp.1304 – 1313, April 2019]

Institutional Research Performance Assessments- Private Universities

Characterizing Indian Research

University-Industry-Government Collaboration

[Ref: Rupika, A. Uddin & V.K. Singh, “Measuring the University-Industry-Government Collaboration in Indian Research Output”, Current Science, Vol. 110, No. 10, pp. 1904-1909, May 2016]

0

10000

20000

30000

40000

50000

60000

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

No

. of

Pu

blic

atio

ns

U

I

G

UI

UG

IG

UIG

2005-14 Research

Output Data for India

Open Access Levels & Patterns

• 2014-18 WoS data, about 24% papers are available as OA.

• Gold OA is most prevalent followed by Green.

[Ref: R. Piryani, J. Dua & V.K. Singh, “Open Access Levels and Patterns in Scholarly Articles from India”, Current Science, Vol. 117, No. 9, pp. 1435-1440, Nov. 2019]

Open Access Levels & Patterns

[Ref: R. Piryani, J. Dua & V.K. Singh, “Open Access Levels and Patterns in Scholarly Articles from India”, Current Science, Vol. 117, No. 9, pp. 1435-1440, Nov. 2019]

AGR

6%AH

0%BIO

12%

CHE

7%

ENG

3%ENV

5%

GEO

4%INF

2%MAR

5%MAT

2%

MED

17%

MUL

13%

PHY

23%

SS

1%

• Physics, Medical Science have higher OA percentages.

Open Access Levels & Patterns

• CSIR system has more papers available as OA.

0

100

200

300

400

500

600

700

800

900

1000

0 1000 2000 3000 4000 5000 6000

No

. of

arti

cles

th

at a

re O

A

WoS Records with DOI

DST

ICAR

IISc Banglore

TIFR

AIIMSDBT

PGIMER

IIT Kharagpur

IIT Roorkee

SNIP

IIT (ISM) Dhanbad

BARCIIT Delhi

MU

BHUAMU

CSIR

Open Access Levels & Patterns-IRs

S. No. Name of Institutional Repository Period of

Records

Total Records

1 DST-DBT Central Repository (sciencecentral.in) 1920-2019 1,25,595

2 CSIR Institutional Repository (csircentral.net) 2006-2019 1,00,609

3 ICAR Central Repository (krishikosh) NA 18,486

• Major Indian IRs do not have significant number of papers deposited.

• Estimates suggest that all IRs taken together has less than 20% ofIndian research papers deposited.

Sci-Hub and Indian Research Output

No. of articles in WoS

Articles with DOI No. of articles that are on Sci-Hub

Sci-Hub articles as Percentage of total articles1

76530 67857 61706 90.93

• Data for India for 2016 as indexed in WoS.

• 90% of the papers are in Sci-Hub.

0

1000

2000

3000

4000

5000

6000 Unique Downloads Total Downloads

Date

Re

ads

Sci-Hub and Indian Research Output

• Who is downloading Indian papers from Sci-Hub? (2017 Access Log)

Gender Distribution• Data sample from 55 most productive institutions/ institution systems

from WoS for 2008-17.

• This constitutes about 65% output from India for this period.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0

10000

20000

30000

40000

50000

60000

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Fem

ale/

Mal

e ra

tio

Tota

l pap

ers

in W

oS

Year

Total papers in WoS Female 1st Authored

Male 1st Authored Female/Male ratio

Gender Distribution

• Biological Sciences,Medical Sciences,EnvironmentalScience and SocialScience disciplineshave better femalerepresentation.

Subjects Total

papers

Total papers

for which

Gender is

determined

Female 1st

Authored

Male 1st

Authored

Female/

Male ratio

AGR 31,414 17,825 5,667 12,158 0.47

AH 11,669 7,263 1,769 5,494 0.32

BIO 1,12,697 83,168 30,640 52,528 0.58

CHE 1,00,382 69,294 18,833 50,461 0.37

ENG 52,120 31,098 6,352 24,746 0.26

ENV 30,926 18,478 5,635 12,843 0.44

GEO 33,000 18,975 5,183 13,792 0.38

INF 20,071 14,106 2,919 11,187 0.26

MAR 85,180 47,638 12,466 35,172 0.35

MAT 20,607 12,463 2,714 9,749 0.28

MED 97,177 69,512 22,256 47,256 0.47

MUL 14,616 10,092 3,124 6,968 0.45

PHY 1,21,003 66,659 16,564 50,095 0.33

SS 19,595 14,332 4,408 9,924 0.44

New Dimensions

Social Media Attention

• Altmetrics is emerging as a new source of researchperformance assessment.

• It gives a good estimate of societal impact ofresearch.

• These indicators are available early and could beeasily processed using computational methods.

• Studies have suggested that altmetrics could be anearly indicator of a paper’s impact and that theyoften correlate with citations.

Social Media Attention

[Ref: S.K. Banshal, V.K. Singh, P.K. Muhuri & P. Mayr, “How much Research Output from Indiagets Social Media Attention?”, Current Science, Vol. 117, No. 5. Sep. 2019. pp. 753-760]

• Data for India for the year 2016 from WoS.

• 28.5% of the papers from India get some social media attention ascompared to world average of 47%.

Mention Type TP Percentage (%)* Total Mention Mention Per Paper

Mendeley 20,815 27.2 353817 16.998

Attention Score 18,449 24.1 136222 7.384

Twitter 16,569 21.6 102176 6.167Facebook 3,594 4.7 6960 1.937

News Mentions 1,455 1.9 9528 6.548

Blog 949 1.2 1892 1.994Google 517 0.7 1695 3.279Wiki 496 0.6 760 1.532Reddit 229 0.3 270 1.179Policy 157 0.2 229 1.459Peer Review 149 0.2 315 2.114

F1000 137 0.2 151 1.102Patent 68 0.1 80 1.176

Social Media Attention• Disciplinary variations exist in social media attention of

research.

[Ref: S.K. Banshal, V.K. Singh, P.K. Muhuri & P. Mayr, “How much Research Output from Indiagets Social Media Attention?”, Current Science, Vol. 117, No. 5. Sep. 2019. pp. 753-760]

Applying Text Analytics Methods

• Advances in AI, ML, NLP coupled withavailability of research papers in digital formsallow machine reading and analysis of researchpapers.

• Computational methods can analyse text ofresearch papers to identify emergence oftopics, thematic trends, fine-grained thematicclassification of articles etc.

CS & IT Research in India

[Ref: V. K. Singh, A. Uddin & D. Pinto, “Computer Science Research: The Top 100Institutions in India and in the World”, Scientometrics, Vol. 104, No. 2, pp. 529-553. Springer]

CS & IT Research in India

[Ref: V. K. Singh, A. Uddin & D. Pinto, “Computer Science Research: The Top 100Institutions in India and in the World”, Scientometrics, Vol. 104, No. 2, pp. 529-553. Springer]

Sciento-text Framework

Fusion of techniques, approaches and know-how of Text Analytics and Scientometrics for improved outcomes.

Scientometrics

Text Analytics

Sciento-text

[Ref: A. Uddin, J. Kumar, Marisha & V.K. Singh, “A Sciento-text Framework to CharacterizeResearch Strength of Institutions at Fine-grained Thematic Area Level”, Scientometrics, Vol. 106,No. 3, pp. 1135-1150, March 2016]

Sciento-text Framework

• Deals with issues related to evaluation, academic searchand recommendations.

• Can produce better outcomes of evaluations and canalso identify research strengths of institutions.

• Also includes problems like searching for orrecommending academic papers, patents, venues (i.e.,conferences or journals), authors, experts (e.g., peerreviewers), references (to be cited to support anargument), and datasets.

• It’s a hot topic investigated by both academia (ex.ArnetMiner, CiteSeerx, DocEar) and Industry (ex.Microsoft Academic Search, Google Scholar, SemanticScholar)

Identifying Centres of Excellence

• Applied for Identifying centres of excellence in afine-grained thematic area of research.

• A small prototype developed:

www.universityselectplus.com

• Captured by a Current Science Correspondence.

https://www.currentscience.ac.in/Volumes/114/04/0711.pdf

Application Areas

• Identifying centres of excellence in a researcharea/ Finding a place of research.

• Funding Agencies- Differential Funding.

• Applications to Research Evaluations.

• Finding a researcher to collaborate/ ReviewerAssignment.

• Bibliometric Enhanced Information Retrieval.

Inferences & Implications

• Indian contribution to global research output isincreasing but much more needs to be done,including possibilities of performance-linkedincentives for scientists.

• IRs in India need to be strengthened and incentivemechanisms be created for submissions.

• Ideas of evidence-based and differential funding ofresearch need to be seriously considered.

• Studies on societal impact of research are neededand their inferences seriously considered.

Concluding Remarks

• Newer methods of characterizing,understanding and evaluating research areemerging and India can not be indifferent tothem.

• Capacity building in using the newer approachesand their applications is required.

• Our research community need to be outwardlooking and need to persuade and push for useof the outcomes in policy perspectives.

THANK YOU!

Email: vivek@bhu.ac.in