beyond citation counts - the potential of academic social network sites for scientific impact...
DESCRIPTION
Millions of researchers all around the world have profiles on academic social network sites, such as ResearchGate, Academia.edu, or Mendeley. Still these channels are hardly used for impact assessment. While scientific impact has traditionally been measured with bibliometrics, social media provide new avenues for influence measurement (Altmetrics). We focus on one specific type of social media, namely academic social network sites. How can such platforms provide insights into scientific impact and add to Altmetrics? To answer this question, we rely on a social network analysis of a research community on ResarchGate. The underlying data was provided by the platform provider. It contains detailed interaction and publication information of 55 faculty members of a Swiss public university. We apply a structural perspective and use centrality measures as core indicators of influence within the network. Our analysis proceeds in three steps: First, we describe the network structure in terms of classical SNA metrics. Second, we analyze whether researchers’ network centrality is associated with other metrics of influence, namely: (a) activity on the platform (b) traditional metrics of scholarly influence (i.e. mainly bibliographic criteria), and (c) academic position. Third, we compare the network structure with that of participants' co-authorship pattern. Our findings show that activity on the platform is the best predictor of impact within the network, while publication success and academic play less of a role. Implications for research and practice are provided.TRANSCRIPT
![Page 1: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/1.jpg)
Beyond Citation Counts The Potential of Academic Social Network Sites for Scientific Impact Assessment
Christoph Lutz
ASNA Conference 2013
Zurich, August 27 2013
![Page 2: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/2.jpg)
ASNA Zurich, August 29 2013
Page 2 Executive Summary
• Relational aspects neglected in scientific impact assessment
Academic SNS as a new source for SNA in science
• Case study of ResearchGate: low activity and density, high homophily and interesting centrality effects
Network position partly explanable by other measures of influence
• Following structure overlaps significantly with co-authorship
![Page 3: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/3.jpg)
ASNA Zurich, August 29 2013
Page 3
Measuring Impact in Science 1
![Page 4: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/4.jpg)
ASNA Zurich, August 29 2013
Page 4 Academic Impact Is A Multi-Faceted Construct
![Page 5: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/5.jpg)
ASNA Zurich, August 29 2013
Page 5 Academic Impact: New vs. Old
Bibliometrics from Peer-reviewed Journals Usage-based Metrics
![Page 6: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/6.jpg)
ASNA Zurich, August 29 2013
Page 6 Usage-based metrics
• Publication on the web enables new metrics:
Webometrics
Scientometrics 2.0/Altmetrics
• Goal: achieving a more current and differentiated picture of impact
• Using social media data for person-based and article-based metrics (Priem & Hemminger, 2010; Shema, Bar-Ilan & Thelwall, 2013; Thelwall et al., 2013)
![Page 7: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/7.jpg)
ASNA Zurich, August 29 2013
Page 7 Example: ImpactStory
![Page 8: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/8.jpg)
ASNA Zurich, August 29 2013
Page 8 What about Social Capital and Relations?
• Relations with other researchers are important resources
- Promotion
- Publication
- Invitation
- Collaboration…
• Social capital matters
Bringing in the relational aspect in impact measurement… beyond citations
![Page 9: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/9.jpg)
ASNA Zurich, August 29 2013
Page 9
The ResearchGate Project 2
![Page 10: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/10.jpg)
ASNA Zurich, August 29 2013
Page 10 What is ResearchGate?
• Biggest SNS for scientists with 3 million users
• Based in Berlin and founded in 2008
• More than 30 million publication entries
• Vivid community, especially popular with young researchers in emerging countries
![Page 11: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/11.jpg)
ASNA Zurich, August 29 2013
Page 11
![Page 12: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/12.jpg)
ASNA Zurich, August 29 2013
Page 12 The Project
• Evaluating new forms of impact in management research
• Intensity of interaction, followers, followees, activity, publication resonance, RG score etc.
• 55 participants from 11 institutes: 80 percent male
• 50 percent PhD, 30 post-doc/assistant profs and 20 percent full profs
![Page 13: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/13.jpg)
ASNA Zurich, August 29 2013
Page 13 The Network
• 55 nodes, 10 isolates
• Av. Degree=3
• Density=0.06
• Diameter=6
• Av. Path Length=2.43
• E-I Index=-0.08
(expected: 0.69)
• Clustering=0.48
![Page 14: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/14.jpg)
ASNA Zurich, August 29 2013
Page 14 Network and Institute Membership
E-I Index: -0.08 (expected: 0.69)
Clustering: 0.48 (0.27)
![Page 15: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/15.jpg)
ASNA Zurich, August 29 2013
Page 15 Prediciting Network Centrality
Indegree Closeness Betweenness Eigenvector
Seniority (Reference: Master student=0) PhD Student (1) Post-Doc without Professorship (2) Junior/Assistant Professor (3) Full Professor (4)
.11** .28*** n. s. n. s. .46** .71*** n. s. n. s.
.66*** .39** .89*** .83***
n. s. n. s. n. s. n. s.
.22** n. s. .70*** n. s.
Publication Success (WOS h-Index; off platform)
.37* n. s.
n. s. n. s. n. s.
Publication Resonance (on platform)
n. s. .43*
n. s. n. s. .36*
Online Activity .50*** .40**
.21** n. s. .41***
ResearchGate Score n. s. n. s. n. s. n. s.
R2 70% 50%
37% 41% 54%
![Page 16: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/16.jpg)
ASNA Zurich, August 29 2013
Page 16 Network and Academic Position
Blue=PhD
Black=Postdoc/Project Leader
White=Assistant Professor
Red=Full Professor
![Page 17: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/17.jpg)
ASNA Zurich, August 29 2013
Page 17 The Co-Authorship Network
Coloring=Institute
Size=Indegree
![Page 18: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/18.jpg)
ASNA Zurich, August 29 2013
Page 18 QAP-correlating Follower and Co-Authorship Network
Do the networks overlap?
Do researchers who write articles together also follow each other?
Yes… but the correlation is not very strong: 0.09 (p-value: 0.025)
![Page 19: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/19.jpg)
ASNA Zurich, August 29 2013
Page 19 Summary
• Indegree and closeness allrounders, betweenness most distinct
• Activity most important predictor of centrality (reciprocity)
• Academic position important but differentiated picture (inverted u-shape?)
• Publication success (h-index) important for overall indegree but not for faculty-specific indegree
• High homophily and clustering and low activity
![Page 20: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/20.jpg)
ASNA Zurich, August 29 2013
Page 20 The Road Ahead
• Sample extension to other research areas and universities
• More sophisticated data by including different social media
• Combining centrality with other criteria
• Looking at other publications (proceedings, book chapters…)
• ERGMing the s*** out of the data
![Page 21: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/21.jpg)
ASNA Zurich, August 29 2013
Page 21
Thank You For Your Attention
![Page 22: Beyond Citation Counts - The Potential of Academic Social Network Sites for Scientific Impact Assessment](https://reader033.vdocument.in/reader033/viewer/2022052822/554c183cb4c905ec518b4eef/html5/thumbnails/22.jpg)
ASNA Zurich, August 29 2013
Page 22 Interested in the Project and Collaboration?
Mail: [email protected]
ResearchGate: http://www.researchgate.net/profile/Christoph_Lutz/
Academia.edu: http://independent.academia.edu/ChristophLutz
Mendeley: http://www.mendeley.com/profiles/christoph-lutz1/
@lutzid