understanding and communicating the value of research...
TRANSCRIPT
![Page 1: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/1.jpg)
Understanding and communicating the value of research enterprises
Nick BerenteUniversity of Notre Dame
NSF CI TEAM #1240160 NSF RCN # 1148996
![Page 2: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/2.jpg)
My Research:Next Generation Research Enterprise
![Page 3: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/3.jpg)
Research
![Page 4: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/4.jpg)
![Page 5: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/5.jpg)
Distributed, Computationally Intensive Science
Collaborative
Multi/Inter/Trans-disciplinary
Data & Visualization
Global & International
Shared Resources / Digital Infrastructures
Unprecedented Scale
Scrutiny – Funding Uncertainty
“Science Enterprises”Larger-scale, Persistent Organizations
![Page 6: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/6.jpg)
Research Enterprises…
ProjectInstituteCenterCollaborativeResourceSoftwareFacilityLaboratoryFoundationAgencyDepartment
![Page 7: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/7.jpg)
Why should we continue to fund you?Why should we fund you in the first place?
Why should we increase your funding?Why shouldn’t we decrease your funding?
![Page 8: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/8.jpg)
Value
“the importance, worth, or usefulness of something”
![Page 9: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/9.jpg)
Value: Relative to the Stakeholder
(Mark Rothko No. 6
$186 Million)
![Page 10: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/10.jpg)
What researchers are interested in
What funders are interested in
![Page 11: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/11.jpg)
What interests researchers:
- Number of publications
- Status of journal
- Number of citations
- M-index, H-index, impact factor
- Grant funding…
![Page 12: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/12.jpg)
Logic Model
Researchers Funders
![Page 13: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/13.jpg)
Four Dimensions of Value for Sustained Science Enterprise
Scientific Findings
- Solving pressing problems
- Overall science trajectories
- Publications, citations
Economic Return
- Dollars / matching dollars
- Multipliers local / national
- Industry creation –potential & analogous
Workforce Development
- Training & education – grad students and practitioners
- Capabilities & capacity
- Diaspora & graduation
Innovations
- Technologies / patents
- New ventures
- Future benefits –tranformational vs. incremental
- Industrial product development
![Page 14: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/14.jpg)
Communicating Value
![Page 15: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/15.jpg)
Before you can communicate value,
you need to understand value
Sustained Research Enterprises
![Page 16: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/16.jpg)
Value of…
… Center… Resource… Project… Software… Facility… Institute… Lab… University… Scientist
Be specific about which research enterprise.
![Page 17: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/17.jpg)
… Federal Science Agency… Congress… State / Region… University… General population… Scientists… Globe
Value for…
Be specific about which stakeholder.
![Page 18: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/18.jpg)
… Scientific findings… Economic… Workforce… Innovation
Be specific about which dimension of value.
Value for…
![Page 19: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/19.jpg)
1. Identify important stakeholders
2. Value propositions for those stakeholders
3. Develop KPIs
4. Bring KPIs into the cultureof the enterprise
Understanding & Communicating Valueof a Science Enterprise
![Page 20: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/20.jpg)
1. Identify key stakeholders (vertical axis)
Science Economic Workforce Innovation
Nation
State
University
![Page 21: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/21.jpg)
2. Value Propositions for those Stakeholders
Science Economic Workforce Innovation
Nation (NSF)
State(IL)
University(UIUC)
![Page 22: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/22.jpg)
Science Economic Workforce Innovation
Nation (NSF)
ImpactQuantity Translation
State(IL)
Generate funds to region
Workforcedevelopment Commerce
University(UIUC)
Bring funds to University Impact students
2. What Brings those Stakeholders Value? – specifically!
![Page 23: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/23.jpg)
Stakeholder SegmentationIdentify key stakeholders and specific value propositions
Operational Science Economic Workforce Innovation
Congress 1
Congress 2Public 1Public 2NSF 1NSF 2
State 1State 2
University 1
University 2
![Page 24: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/24.jpg)
Your turn: Stakeholders & Value
Bad metrics:- Overall time- Quality- Speed - Performance
24
![Page 25: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/25.jpg)
“All the good business leaders I know are maniacal about measuring things. They know their sales data and customer-satisfaction numbers, which divisions of their company are beating expectations and which are lagging behind…
Measurement is a big part of mobilizing for impact. You set a goal, and then you use data to make sure you’re making progress toward it.”
-Bill Gates
![Page 26: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/26.jpg)
Metrics
Metric – specific measurement through which we can evaluate performance toward a goal
Rules about metrics:- Must be specific- Must be measurable
Quantity and quality metrics- Quantity sells!
Bad metrics:- Overall time- Quality- Speed - Performance
26
![Page 27: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/27.jpg)
Key Performance IndicatorsKPIs describe the important criteria that you will use to analyze and redesign processes. Each KPI should have:
Title: In a word or two say what is important [e.g., “Minimize Defects”]
Description: Describe what this means [ e.g., “A Defect invoice is any invoice that does not accurately list the customer information, product information, price, tax and shipping information.”]
Metric: How will this be measured [e.g. “Defects per month” or perhaps, “Defects per 1000 invoices”]
Target: What is the acceptable metric? [e.g. “1 defect per month” or “2 defects per 1000 invoices”]
27
![Page 28: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/28.jpg)
Goals
• Who is the stakeholder?
• What is the value proposition?
• Does the value proposition align with strategic goals?
• What are specific objectives with respect to this stakeholder?
• How can we measure progress toward these objectives?
• Develop Key Performance Indicators
28
Make metrics and KPIs part of the culture of the science enterprise
![Page 29: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/29.jpg)
“Numbers beat no numbers every time.”
![Page 30: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/30.jpg)
We completed 820 student-hours of training…
![Page 31: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/31.jpg)
Contextualize Numbers
• Cost / ROI• History / Trend• Benchmark• Target• Visualizations
![Page 32: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/32.jpg)
We completed 820 student-hours of training…
… Our target training hours are 600 per year
![Page 33: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/33.jpg)
We completed 820 student-hours of training…
… This is 10% above last year, and 20% above the year before that
![Page 34: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/34.jpg)
We completed 820 student-hours of training…
… For our $50,000 budget, that’s $61 per student hour of training
![Page 35: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/35.jpg)
Goals
Image from: http://www.valueprism.com/strategyreview.html
Strategic
Operational
35
![Page 36: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/36.jpg)
Operational Science Economic Knowledge Innovation
Congress 1
Congress 2National Competitiveness
Public 1Public 2NSF 1NSF 2
State 1State 2
University 1
University 2
Stakeholder Segmentation & Key Performance Indicator
![Page 37: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/37.jpg)
![Page 38: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/38.jpg)
![Page 39: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/39.jpg)
![Page 40: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/40.jpg)
Your turn: Stakeholders & Value
Bad metrics:- Overall time- Quality- Speed - Performance
40
![Page 41: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/41.jpg)
Understanding & Communicating Value
Make metrics and KPIs part of the culture of the science enterprise
![Page 42: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/42.jpg)
Additional Materials
https://www.ssrn.com/abstract=2881752 https://ssrn.com/abstract=2416247
![Page 43: Understanding and communicating the value of research ...docs.asee.org/public/ERC/2018-PGW/Berente.Nick.pdfInnovation Be specific about which dimension of value. ... Develop KPIs 4](https://reader033.vdocument.in/reader033/viewer/2022050309/5f718c4b18e01c7d20115628/html5/thumbnails/43.jpg)
Additional Materials
http://distributedscience.ischool.utexas.edu/