Do We Need Standards for

Alternative Metrics?

What is NISO hoping to accomplish?

Todd Carpenter, Executive Director, NISO NISO Alternative Assessment Initiative

CNI Fall Project Briefing Washington, DC | December 9, 2013

• Non-profit industry trade association accredited by ANSI with 150+ members

• Mission of developing and maintaining technical standards related to information, documentation, discovery and distribution of published materials and media

• Volunteer driven organization: 400+ spread out across the world

• Responsible in various wasy for standards like ISSN, DOI, Dublin Core metadata, DAISY digital talking books, OpenURL, MARC records, and ISBN

About

Phase 1 Discussion

Forums

NISO Alternative Assessment Metrics Initiative

!

Brainstorming Meetings October 9, 2013 // December 11, 2013 //

January 23, 2014

Washington, DC | San Francisco, CA | Philadelphia, PA

Define altmetrics, please

Usage Based Metrics

Clickstream Analysis

Social Media Metrics

Image: Danny Brown, The State of Social Media Marketing 2012

Clickstream data yields high-resolution maps of science

Johan Bollen, Herbert Van de Sompel, et al. PLoS One, February 2009

Clickstream network analysis

Behavioral Metrics

Multivariate analysis

Sentiment Analysis

Image: Domenico, Caron, Davis, et al.

Citation of non-traditional content

Citations to new forms of communication aren’t as simple as one might think. !

Define a use of this?

We’ve got what, now how?

Image: Flickr user karindalziel

What can be measured?

How do we measure things?

At what granularity do we measure things?

Which measurements are valuable?

How do we ensure the validity of those measurements?

How to (or even if we should) deal with other issues

such as gaming?

Issues we face with alternative metrics

Basic Definitions !

(So we are all talking about the same thing)

Comparison across providers

Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.

Consistency in what is counted

Source:  Scott  Chamberlain,  Consuming  Article-­‐Level  Metrics: Observations  And  Lessons  From  Comparing  Aggregator  Provider  Data,  Information  Standards  Quarterly,  Summer  2013,  Vol  25,  Issue  2.

Authorship -----

Disambiguation &

Contributor Roles

Open exchange of component data

TRUST

What is NISO trying to do?

Alterna(ve  Assessment  Ini(a(ve  

!

Phase  1  Mee(ngs October  9,  2013    -­‐  San  Francisco,  CA December  11,  2013  -­‐  Washington,  DC

January  23-­‐24  -­‐  Philadelphia,  PA !

Phase  1  report  expected  in  May  2014

Alterna(ve  Assessment  Ini(a(ve   !

Phase  2 Presenta(ons  of  report  (June  2014) Priori(za(on  Effort  (June  -­‐  Aug,  2014)

Project  approval  (Sept  2014) Working  group  forma(on  (Oct  2014)

Consensus  Development  (Nov  2014  -­‐  Dec  2015) Trial  Use  Period  (Dec  15  -­‐  Mar  16)

Publica(on  of  final  recommenda(ons  (Jun  16)

Criticism

Why now? Isn’t it too soon?

Can NISO to do this?

How soon is now?

These things take time

It will be hard to change

Capital “S” Standards vs

standards (i.e., best practice)

“All I see now is blonde, brunette,

redhead”

ISQ Altmetrics Issue Now Available Open Access!

www.niso.org/publications/isq/

One Very Last Thing…

Image from Will Lion: http://www.flickr.com/photos/will-lion/2646213692/

Thank you! Todd Carpenter, Executive Director

tcarpenter@niso.org National Information Standards Organization (NISO)

3600 Clipper Mill Road, Suite 302 Baltimore, MD 21211 USA

+1 (301) 654-2512 www.niso.org