clements - visualizing the impact of mission driven organisations

Visualizing the impact of mission driven organizations: Multiple perspectives on knowledge flows and impact from research

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Margaret M. Clements, Center for Knowledge Diffusion Molly Macauley, Resources for the Future [email protected], [email protected], Please do not cite or quote without permission of the authors. The financial support of the US National Aeronautics and Space Administration under Contract NNJ12GA17C is gratefully acknowledged.

© 2016, Center for Knowledge Diffusion, All rights reserved

mailto:[email protected]



Science as New Knowledge: ( “delta K over K”) (NASA Chief Scientist 2011)

Common challenges in corporate R&D labs, at universities, and in government • How to measure and count new knowledge • How to characterize its significance for the agency, or corporate shareholders,

or promotion and tenure review, or funders, and … for society • How best to use this information to encourage new science and innovation

(and manage internal resources as science managers and leaders) and • How, what, by whom, and why to communicate credibly to diverse

stakeholders and in turn, their constituencies • Revision of knowledge (major as well as minor revision – delta k / k is a ratio –

revise textbook)


White House, June 28, 2010, The National Space Policy of the United States of America • “(The ISS is to be utilized) for scientific, technological, commercial,

diplomatic, and educational purposes; • Support activities requiring the unique attributes of humans in space; • Serve as a continuous human presence in Earth orbit; and • Support future objectives in human space exploration” US GAO, March 28, 2012, NASA: Significant Challenges Remain for Access, Use, and Sustainment of the International Space Station • “NASA must ensure that the management of the ISS national laboratory

results in effective utilization of the station for its primary purpose – scientific research.”

• “The ISS was constructed to support three activities: scientific research, technology development, and development of industrial applications”

ISS Science Mission


US House of Representatives, 113th Congress, 1st Session, “National Aeronautics and Space Administration Authorization Act of 2013, June 12, 2013 -- Discussion Draft • “The (NASA) Administrator shall provide…criteria for defining the ISS as a research

success” and a strategic plan for conducting competitive, peer-reviewed research in physical and life sciences and related technologies on the ISS.”

White House 8 January 2014, ISS Extension to at least 2024 • The ISS is a unique facility that offers enormous scientific and societal benefits. The

Obama Administration’s decision to extend its life until at least 2024 will allow us to maximize its potential, deliver critical benefits to our Nation and the world, and maintain American leadership in space.

ISS Science


White House, Winter 2015, FY2016 Budget Request Executive Summary, p. SUM-5

This budget request strongly supports ISS research. Scientists and engineers will develop

and execute experiments and technology demonstrations in diverse disciplines including

physics, biology, materials science, robotics, communications, and human physiology.

Insights gained from these studies will be essential for planning future deep space

missions, including to Mars. The Center for the Advancement of Science in Space (CASIS),

the research management organization for the ISS National Laboratory, will continue to

enable federal, academic, and commercial research activities. Exciting research will

include potential medicines and interventions that will improve human health both in

space and here on Earth.

ISS Science Mission, continued

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Research is motivated by:

Consideration for Applied Use

Quest

for F

unda

menta

l Und

erstan

ding

Quadrant Model of Scientific Research (Stokes, 1997)

ISS Science in Theories of R&D

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

US National Aeronautics and Space Administration http://www.nasa.gov/mission_pages/station/main/onthestation/facts_and_figures.html

About the size of a football field, with the cubic capacity of a five-bedroom house, orbiting about 220 miles above the Earth

The ISS: Physical Capacity and Operating Characteristics

Credit:http://www.bing.com/images/search?q=diagram+of+space+station&id=8CBCF3B31624B0AE910A1E565DD2AA9C60F0928B&FORM=IQFRBA#view=detail&id=8CBCF3B31624B0AE910A1E565DD2AA9C60F0928B&selectedIndex=0

The ISS and International Collaboration

OECD Handbook on Measuring the Space Economy UCSD Map of Science US Science of Science Policy Historical Studies in the Impact of Spaceflight

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Approach Background

Approach Rationale Grant agencies, Administrators, R&D managers: to monitor grant flows, research developments, funding strategies and patterns; to identify scientific frontiers, the dynamics of scientific fields, and emerging complementary fields. NSF, NIH, USDA, NOAA, etc. Researchers and students: to detect social networks, “invisible colleges,” potential and cross-disciplinary collaborators, the interconnectedness of science Industry, national security agencies: to identify major science results, ascertain how knowledge is diffusing, see gaps and promise in research directions that industry might want to pursue Agencies and Librarians: unique visual interface to digital resources Society: to dramatically improve the access to, communication of, and utilization of the public’s investment in science

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Table 1. ISS PSD Summary of Outputs (from ISS Office) Research Program Journal Conference Patent Technical

Paper Other

Biology and Biotechnology

235 8 1

Earth and Space Science

52 4

Human Research 171 11 3 5

Physical Science 94 33 2 Educational Activity 6 6 4

Technology 27 69 14 Exploration 5 24 5 2

Summary/Review 13 23

Data: a subset of ISS research program areas

Note: entries not updated; cohort totals 888 records and over 4300 first-order citations to them – as of November 2013 (bulk of main data, new entries can be added)

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Disciplines Linked by Shared Investigations • Each segment of the arc represents a discipline

• Link shows that two disciplines share at least one investigation.

• Wider links indicate more shared investigations. • Disciplines are linked based on the investigations cited in published

paper.

• If a paper in one discipline cites an investigation and a paper in another discipline also cites that investigation, the two disciplines are linked by that investigation.

• Hover over disciplines to show how closely linked it is to other disciplines. Click on a link to see which investigations are shared between them.

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Disciplines and Shared Investigations

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

• Science of science – a type of science analysis and assessment -- is a key federal initiative, led by OSTP/OMB/White House and NSF & with the participation of the science-oriented agencies (NIH, USDA, DoI, DoE, EPA, NASA)

• Science of science approaches seek to go beyond mere “counts” of publications (outputs) to ascertain and demonstrate “delta K over K” (outcomes) – that is, the generation and diffusion of new knowledge through networks of people and their formal and informal exchanges of ideas

• Salient attributes of our approach are that it is objective and external to syntheses prepared by the agency

Key Findings

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

• ISS science – in particular, as inventoried and curated by the ISS science office -- can be described by the science of science approach (that is, methods, analytical studies, and visual displays of results) – for use by decisionmakers & to help inform the science community as well as the interested public

• Using science of science assessment, the ISS science portfolio is extraordinarily impressive in breadth (the range of science disciplines) and impact (growth and rates of growth in citations to ISS research) both within specialized space journals – evidence of the diffusion of specialized knowledge to benefit space activities

• In mainstream (non-space) journals – evidence of the diffusion of knowledge for broader societal benefit

• Salient attributes of our approach are that it is objective and external to syntheses prepared by the agency

© 2

016,

Cen

ter f

or K

now

ledg

e D

iffus

ion,

All

righ

ts

rese

rved

Acknowledgements • Michael J. Stamper

• Thomas Grenfell Smith

• Joseph Cottam • David Ernst

• Project Steering Committee


clements - visualizing the impact of mission driven organisations

Data & Analytics