consortium metrics discussion with iom drug forum
DESCRIPTION
Presentation made to the IOM Forum on Drug Discovery, Development, and Translation to explore the possibility of metrics that evaluate the performance of biomedical research consortiaTRANSCRIPT
Strengthening the
Effectiveness and Impact of
Collaboration Through Metrics
Mark D. Lim, PhD
Forum on Drug Discovery, Development, and TranslationInstitute of Medicine
October 7, 2014
Shared scientific challenge
Widely-usable tool
Virtual team to create / qualify
research-by-consortium
Sci. Trans. Medicine, June 2014
http://bit.ly/STMConsortia
Operational Framework Landscape
consortiapedia.fastercures.org
Database
Planned release:
end of 2014
Metrics for consortia?
We need to value collective successes of a consortium…
and get rewarded for that.Eric Green, NHGRI Director
Whatever can be done in qualifying how consortium work, how
quality is being measured and recognized, would help big
companies like ours to work in these consortium.Marc Bonnefoi, Sanofi
FasterCures Hill Briefing, July 2014
Objective measurements – performance and outcome
Apples and apples - normalized across consortia
Suite of metrics by project type? Application (biomarker development, clinical trial methods, etc)
Therapeutic area
Stage of innovation
Others?
Considerations:
Since we last met….
Conversations
• Consortium leaders and participants
• Pharmaceutical companies
• FDA
• Collaboration management experts
Presentations
• Science of Team Science
• Pharmaceutical campus
Other
• Hill Briefing on consortia
An evolving conversation…
qualitative
metrics…asking
participants what the
vision/mission is…
what are indicators
for future success?
is this consortium
doing the best it can?
what are common
challenges that all
consortia are facing?
Where we want to go…
Reality check on value of framework
General consensus on:
Design
Implementation
Optimization
Who are implementation partners?
What is the output? What does success look like?
Evidence to inform best practices?
Metrics of collaboration?
Retrospective Prospective
• Return on investment
• Impact on organization
• Impact on society
• Improved efficiency
• Focus vs flexibility
Output and outcomes
Sponsors/Participants
• What did we get from the collaboration?
• Would we participate in a similar collaboration again?
Managers/Organizers
• Were there lessons learned?
• How do we design/execute a better collaboration?
Value of outcomes: who is asking?
Government • public health
• regulatory science
• de-risk innovation
• economic growth
• state-of-science research guidance documents
Industry • accelerate pipeline
• new therapeutic area
• access resources
• de-risk innovation
• access intellectual
capital
Academia • access resources
• opportunities for
publications
• training opportunities
• identify collaborators
Patient
organizations
• accelerate pipelines
• advance basic
research
• de-risk medical
product development
Consortium
researchers
• simplify day jobs
• access resources
• networking
• training / education
Output to outcome metrics
* Adapted from Critical Path Institute and Innovative Medicines Initiative
Non-competitive biomarkers
Clinical trial designs and processes
Data standards
Best practices / procedures
Tools (PRO, prediction, data-sharing)
Output
Implementation / adoption
IP – filed / awarded / licensed
Policy change / regulatory science
Publications
Job / company creation
Impact
De-risked innovation
Decreased R&D cost/time
Meaningful data-sharing
Improved clinical trials
Improved human health
Outcome
Metrics of collaboration?
Retrospective Prospective
• Return on investment
• Impact on organization
• Impact on society
• Improved efficiency
• Focus vs flexibility
Processes
Sponsors/Participants
• Is the collaboration on-track and focused?
• Are resources available/leveraged?
Managers/Organizers
• How can we do better?
• Are we continuing to address an unmet need?
Processes & Team Performance
Wooten, Univ. Houston
Tuckman & Jensen, Group and Organizational Studies (1977)
Burke et al., J. Applied Psychology (2006)
Hall et al, Trans Behavioral Med (2012)
Identifying shared challenge
Consensus solution
Framework for collaboration
Defining success
Convene
Defining accomplishments
Integrating deliverables
Communicating
Tracking adoption/impact
Transition
Execute
Participant/sponsor buy-in
Maintaining vision/focus
Work stream monitoring
Conflict resolution
Discussion points
? Gut reaction?
Structuring discussion to develop framework?
• How? (retrospective/prospective)
• What? (end-user)
• Who? (stakeholders)
• Where? (host)
Next-steps
• Follow-up with consortia, industry, FDA
• Partnering For Cures – Nov 16-18 (NYC)