report: who informal workshop monitoring …...page 1 of 16 report: who informal workshop –...
TRANSCRIPT
Page 1 of 16
REPORT: WHO Informal workshop – monitoring financial flows in support of
health research & development
14th February 2013 at the London offices of the Wellcome Trust
Background
This workshop brought together people and organizations who are experienced in the area of collecting
data on resource flows in support of health R&D and related initiatives that present that data in
platforms and online resources. The specific aims of this workshop were:
1. To share details of learning and experience on major initiatives that have been concerned with
efforts to map international R & D resource flows with a focus on health (e.g. G Finder and
RICYT) and related studies (e.g. OECD and UNESCO R&D surveys and statistics).
2. To explore related issues for example research classification or how the health product pipeline
for the health needs of developing countries might be mapped.
3. To discuss current opportunities and challenges in doing R&D resource flow mapping from the
perspective of political, economic, social, technical, legal and ethical/environmental aspects.
4. To explore the opportunities to develop a more coordinated approach to R&D resource flow
mapping at a global level.
The agenda and list of participants are attached at the end of this report.
Page 2 of 16
Morning session: Overview of R&D mapping past and present
The morning session consisted of presentations from the participants on initiatives past and present in
the area of mapping R&D and health R&D measures. The presentations are available from the WHO
website at http://www.who.int/phi/workshop14022013/en/index.html
Summary of Presentations
1. Stephen Matlin and Allison Young, ex-Global Forum for Health Research:
Monitoring R&D resource flows. There has been a steady growth of expenditures that has grown from
USD$ 30 billion in 1986 to USD$ 240 billion in 2009 indicating a growth of about USD$ 20 billion yearly.
Typically the larger spender has been the private sector , followed by public (each close to 45%) and
philanthropic contributions make up about 8%. However, for neglected diseases the public sector takes
care of approximately two thirds of the expenditure. There are large gaps on data and lack of uniformity
that makes it difficult to analyse or aggregate data. There is a lack of data transparency especially with
the private sector and a lack of data at the disease level are noticeable particularly in the private sector.
In the public sector there is lack of data beyond OECD countries.
In collecting R&D data it varies between countries which body is tasked, if at all, with the collection of
data. A question is posed as to on whether it is better to have health R&D expenditure in the health
accounts or, as it can be located in other ministries, whether it should be disaggregated from national
accounts. It remains unclear what incentives exist for agencies to invest effort in collecting data within
poorer countries on neglected diseases where this can be a considerable effort for small sums of money.
Incentives will need to be identified that will create a comprehensive and disaggregated data from all
funders. Potential sources and key partners include: UNESCO, RICYT, AU, EUROSTAT, AFRISTAT.
Page 3 of 16
With regard to the establishment of an health R&D Observatory there is a promising model in the
UNESCO Institute for Statistics based in Montreal outside of the HQ of UNESCO. This Institute has
brought a gold standard for collating information on education. RICYT is another model with a regional
perspective (see below).
2. Allison Young, the OECD approach to collecting R&D data
The methodology is developed from the Frascati manual1. which is being revised in 2013. Its guidelines
underpin national R&D surveys and the annual joint OECD/EUROSTAT/UNESCO survey which covers 50
mostly High-Income countries. The Manual is written by a group of experts from these countries, they
organize the surveys, most are drawn from the national statistical offices or ministries of Science &
Technology, and some academics. The overall R&D picture is reported in national and international
publications and databases but estimates for health have to be built as it is not a specific category. There
is little specific interest in health in the expert group, the focus is mostly on R&D with a direct economic
application.
R&D data can be surveyed using two different starting points i.e. i) collecting data from those that Fund
(ministries, agencies foundations etc.) or ii) collecting date from those that are paid to carry out
(Perform) the R&D (universities, hospitals, firms etc.).
OECD and Eurostat publish only one set of funder-reported data, GBAORD = central government budget
appropriations or outlays for R&D. The 3 countries funding the most health R&D via GBAORD are the
USA, Sweden and Netherlands (see slide above). However, this illustrates one (of the many) challenges
in mapping as it demonstrates the huge differences in reporting by socio-economic objective (SEO)
which reflect the institutional arrangements for health R&D in the countries concerned. In the USA
government funded R&D related to health is concentrated in the NIH and thus mostly reported under
the health SEO, in Sweden, where there is a general research council, it mainly credited to Advancement
of Knowledge. In the Netherlands where advancement of knowledge can be further broken down –
most health R&D is funded via General University Funds, the block RD/Education grant to medical
1 Proposed Standard Practice for Surveys on Research and Experimental Development The Frascati Manual 2002:.
OECD; 2002
Page 4 of 16
faculties and hospitals. Some funding categories will be more susceptible to a breakdown by type of
disease than others.
Estimates can also be compiled from the data sets reported by the performers which add to Gross
Domestic Expenditure on R&D (GERD) the standard measure of national R&D efforts. Two health related
categories are R&D in the medical sciences and R&D for health as a socio-economic objective plus, for
business, R&D in health-related industries or products. Building an estimate is therefore complicated.
Data are fragmented, there are problems with response rates and the quality and choice of proxies.
What data can be compiled provides only a very broad measure of the cost of health-related R&D
carried out in each country.
People working in this field have their own terminology and for future work it would be advisable to
follow the recommendations of the international methodology in the Frascati manual as far as possible
in designing a new system of analysis. Few people have used the health R&D data that has been
generated to date. Other than by the Global Forum for Health Research, it has been used in two reports
of the European Science Foundation. The health GBAORD data are quoted in the biennial OECD Science
and Technology and Industry Scoreboard publication.
3. Rohan Pathirage, UNESCO Institute for Statistics (UIS)
The UNESCO Institute for Statistics (UIS) survey on Research and Experimental Development (R&D)
Statistics forms part of the Institute's strategy to improve its statistical programme and to develop and
deliver timely, accurate and policy-relevant statistics in the field of Science, Technology and Innovation
Page 5 of 16
(STI). The objective of this survey is to collect the most recent data on human and financial resources
devoted to R&D. The UIS R&D survey is conducted biennially to gather data from more than 215
countries and territories, following OECD Frascati Manual guidelines. The UIS sends out the
questionnaire to about 140 countries which are not covered through the OECD, Eurostat and RICYT data
collections. In agreement with these three organizations, their data (OECD and Eurostat – 44 and RICYT
– 19) are directly transferred into the UIS database. A joint data collection with the African STI Indicators
Initiative of the AU/NEPAD is currently under discussion. Since 2004, five data collection rounds have
been carried out and the results of the survey can be accessed through the UIS website at
http://stats.uis.unesco.org, where data are available from 1996 onwards.
Among the range of data that are collected and published, there is limited information in relation to
field of health, namely ‘researchers in the field of medical and health sciences by sector of employment
and gender’ and ‘R&D expenditure in the field of medical and health sciences’.
Issues in relation to quality and availability of data from developing countries:
Lack of data and gaps in survey responses from small islands/territories and also from some
countries in Africa and Arab States. Availability of data from the latter two has been improving
lately.
Lack of coverage from some sectors of economy (especially from Business enterprise and Private
non-profit sectors are under-reported).
Lack of data related to R&D Personnel in Full-time equivalents (FTE).
Problems of reporting data in line with Frascati Manual guidelines, leading to comparability
issues.
Unavailability of data for detailed breakdowns (e.g. by field of science) though data are available
at the aggregated level.
Lack of time series data due to ad-hoc national surveys.
Challenges in relation to R&D data collection at the national level in developing countries:
Developing countries are facing various challenges in collecting R&D data. These challenges are multiple,
and play at different levels. We can distinguish between challenges at the political level, the institutional
level and the technical level. These challenges as well as number of actions that could be taken to
minimize them are discussed in detail in UIS Technical Paper no. 5, “Measuring R&D: Challenges Faced
by Developing Countries” (UIS, 2010: http://www.uis.unesco.org/Library/Documents/tech%205-
eng.pdf) and in the “Annex to the Frascati Manual: Measuring R&D in Developing Countries” (OECD,
2012: http://www.uis.unesco.org/ScienceTechnology/Documents/oecd_frascati_annex_rd_2012.pdf).
UIS R&D data are used not only in its own periodicals but are also widely cited in many reports prepared
by UNESCO, more importantly the UNESCO Science Report with a substantial Statistical Annex. Data are
also published in World Bank’s World Development Indicators, UN Statistical Yearbook and are cited in
related reports prepared by other UN agencies as well as public and private institutions or individuals
worldwide.
Page 6 of 16
4. Rodolfo Barrere and Analia Porras, RICYT PAHO/WHO
The network provides indicators for Ibero-America and works with national agencies for science &
technologies in more than 28 countries. RICYT works with UNESCO and they are funded by the
IberoAmerican States Organization, and they have collaborations with other organizations such as
PAHO. The scope includes the production and diffusion of information, reaching methodological
agreements, consolidating skills and developing new indicators. The framework for the observatory
nested in the Regional Platform on Access and Innovation for Health Technologies (funded by FDA, MoH
of Brazil ) is presented. Information is taken from different sources and there is a peer review process
for the application of indicators. The funding is not from MoH (just Brazil, US and Canada would have
some significant funding); funding comes from Science & Technology mostly.
In Latin America information for R&D is not disaggregated for health. Argentina, Brazil, Colombia and
Mexico have partnered with PAHO in a pilot to develop a framework to map research, development and
innovation (RDI) capacities in Latin America. The framework included 8 elements that included RDI
inputs and outputs together with IP management, Access, Regulatory capacities.
Categorization remains a big challenge: what to include under health, what to include in budgets and
funds, private funding is hard to measure. Even degrees awarded by the universities have different
categories that do not translate well between countries. Journals and access to information and
networks are also assessed. RICYT pulls together “hard indicators” as well as measures of context
specific data in an attempt to pull all the information together. There is a tonne of data and reducing
those indicators and choosing key indicators for countries, those that are better predictors, remains a
challenge. There are also efforts to assess the efficiency, productivity and economic returns on investing
in R&D. For human resources RICYT works with agencies in countries. They have mapped collaborations
between countries to characterize how collaborations are taking place (See Brazil example above).
One issue is about balancing openness with confidentiality. For example the information about
regulatory agencies includes a confidential assessment by peers which remains closed. Currently many
of the indicators collected are not public and follow a confidentiality agreement. This is particularly true
for data that is collected from the private e.g. pharmaceutical sector.
Page 7 of 16
5. Javier Guzman, G-FINDER
G-FINDER is part of Policy Cures and funded by the Bill & Melinda Gates Foundation since 2008, five
years initially and now it has been extended 3 more years. G-FINDER measures: 31 neglected diseases;
134 product areas (drugs, vaccines, diagnostics, microbicides and vector control products); Platform
technologies (adjuvants, delivery technologies, diagnostic platforms) and all types of product-related
R&D.
They measure funding on 31 neglected diseases which they designated using a panel of international experts predominantly focusing on those diseases that disproportionally affect low and middle income countries, where there is a R&D gap (i.e. there is no existing product OR improved or additional products are needed) and there is market failure (i.e. there is insufficient commercial market to attract R&D by private industry) . Data is collected from working with funding agencies and product developers, not with countries. They cover research from basic to phase 4 clinical trials. They get data from around240 organizations globally with dominance from public sector. They follow the money and they report on 56 countries, 29 of which are high income, yet 90% of the data is from those high income countries. To obtain data from Brazil, India and South Africa has proven to be expensive and resource intensive and the returns of data were low.
The reports have been extensively used and this is why B&MG has extended their grant. Development
agencies are using their data to prioritize. It has provided the evidence to set up agendas and priorities
by other partners. It has allowed standardizing reporting for example with the industry.
The limitation of this type of survey is that the process is intensive and expensive and has only one
source of funding to support it (B&MG). When asked everyone says they like the resource and value it –
but no one (else) wants to pay for it. There is no data on health systems research as the focus is
products. There are opportunities in the future through linkage with other tracking efforts and an
effort has been made to identify the added value and the use the data is put to. This is slowly changing
attitudes towards the value of providing more data.
Page 8 of 16
6. Luis Gabriel Cuervo and David Abreu, Health Research Web (HRWeb):
HRWeb provides an online platform to both upload and share information on R&D initiatives, policies
and strategies. It has been constructed as a wiki to allow the uploading information by people in
countries and, after initially seeding the site with available public information, it now relies on people
contributing data. Once a critical amount of information became available to publish analysis and
reports, benchmarking against other – also referred to as “envy and embarrassment”, have played a
role in motivating contributions: people want to showcase their capacities and no one wants to see their
country as having nothing when there are worthy resources.
The aim is to provide practical information and includes publications and analysis of health systems. For
example, there is information on over 1100 ethics review committees in the Americas providing what
perhaps is the most comprehensive mapping of such committees in Latin America.
It is seen as a bottom up approach and has the advantage of being a partnership with the countries that
have ownership over their data. Most of the published data includes links to public sites and documents.
With many different contributors there is a drive towards continuous improvements to the site including
an intention, in the PAHO region, with the International Clinical Trial Database (ICTRP) into HRWeb to
provide a one-stop shop repository with descriptors of national health research systems. Of course this
model provides challenges as well with different approaches to presenting data and changes in country
staff leading to varied quality. Agreeing nomenclature and the classification of what is and is not
research and its subsets is a big challenge. In the future the intention is that the data and the processes
that HRWeb offer can be used by countries for the management of their own research portfolios at
national sub-national and institutional level. The model has allowed developing and publishing in
different media analysis about national health research systems in Latin America, and engaging
delegates from the science & technology and health sectors in the Latin American Conferences on
Research and Innovation for Health (www.paho.org/LACRIH).
Page 9 of 16
7. James Herrington, Fogarty International Center, NIH, and Sandeep Somaiya, Office of Extramural
Research, NIH - World RePORT
The National Institutes of Health (NIH) Fogarty International Center , one of 27 Institutes and Centers at
NIH, along with the NIH Office of Extramural Research, lead an effort on behalf of the NIH Director, Dr.
Francis Collins, to map and describe biomedical and behavioral science activities funded by NIH and
several members from the Heads of International Research organizations (HIROs), including the
European Commission, the Canadian Institutes of Health Research (CIHR), the French Institut National
de la Santé et de la Recherche Médicale (INSERM) and Institut Pasteur, the German Max Planck
Institute, the Medical Research Council/UK, the Swedish International Development Cooperation Agency
(SIDA), and the Wellcome Trust. World RePORT (http://worldreport.nih.gov) is seen as a public tool to
track funding and activities so one can analyze the complete landscape of research funding, identify
funding gaps and areas where there may be a duplication of efforts, seek problems where collaboration
would be useful, and improve efforts to work more effectively and synergize investments in research.
Current economic pressures are a significant driver to share data at the project level with other donors
and the public and make that data public. The intention is to identify areas of synergy, platforms and
networks to build additional research funding and models for successful distribution and coordination of
the distribution of these funds.
There is a need (and a language barrier) to work with Lusophone and Francophone countries. The
interactive database provides descriptors of the research collaborations and funding by all NIH Institutes
and Centers and the 8 organizations noted above that with institutions in sub-Saharan Africa. The data
on World RePORT links to each research activity, funding institution, local principal investigator, and
local research institution web site and contact information. Data can be exported to an Excel
spreadsheet for further analysis. Populating this database requires a manual upload at present, but
automated input is anticipated in the future. Currently the map and database is limited to sub-Sahara
Africa. Expansion to other world regions is dependent on additional resources for design and
maintenance of the tool.
Page 10 of 16
8. William Cullerne Brown and Gerard Ralphs, Research Africa
Research Africa aims to support capacity to conduct research within Africa by providing details on
funding opportunities from more than 5,000 funders. They are working to characterize research funding
with a team of 20 full time staff based in South Africa. They have over 100,000 researchers registered
and using their subscription based database. The database is a spin out from a research policy journal
(Research Research group) and has a strong emphasis on providing quality information. At an aggregate
level the dataset can be used to understand the policy and funding landscape, identifying funding
opportunities and making informed decisions about what to apply for. It enables benchmarking (e.g. for
governments, universities and research groups) as to who applies for what or where grants are
awarded.
Page 11 of 16
9. Ian Vinney, Health Research Classification System
A decade ago the UK established the UK Clinical Research Collaboration (UKCRC) to bring together
the main UK health research agencies to work on common standards and approaches. One area
was to create a common health research classification system (HRCS). This common language is
intended to enable a comparison of like for like research to provide a map of health research that is
funded in the UK. There is a comprehensive website that takes you through the methodology and
the policy tool. The classification system has two dimensions: type of research and type of disease
and there was a concerted effort to prevent double counting. Two surveys have been undertaken
using data in 2005 and 2009. The initiative created a big debate on coordination and the
comparison across the 21 agencies sparked discussion on the relatively limited funds for prevention
research and translational research. The outcome was that new initiatives in preventative research
and translational research were launched with joint funding. The data was also used to negotiate
improved settlements for government funding.
The exercise revealed the significant challenges in this area not least of which is the huge manual
effort required to re-code the different funders data into the single classification system.
Subsequent investigations with automated approaches using computer generated translation are
being investigated. Coverage is still mainly confined to the public sector. For example health
relevant research in the UK was about 8BN GBP and HRCS allowed categorizing about 1.6BN GBP.
New uses for the data are being developed such as a categorization analysis of outputs, outcomes and
impacts of research. This early work is presented at (www.Researchfish.com). The web portal connects
funding organizations, researchers and universities - linking up individual research outcomes with
multiple funder contributions. This enables instant and thorough reporting, so that funders can track the
progress of research and ensure value for money, whilst saving time and money on complex
administration. This is in the early stage and data from the private sector is missing. Research fish has
had the uptake of 18 funding organizations allowing to have most of the portfolio data for their funding
in one place.
Page 12 of 16
10. Theresa Wicklin Gillespie and Elizabeth Ponder, Global Health Primer
The Global Health primer – developed by Bio Ventures for Global Health – is now housed at Emory
University. Emory is being re-structured integrating various disciplines into the research component,
such as the schools for management and business. Emory works with 25 neglected diseases including 8
of the WHO NTDs. The Global Health Primer: Tracks and analyzes progress in global health research and
development; Provides an evidence base to support decision making, policy change, and action; and
Brings new innovators to the table to address the most pressing medical needs of the poor.
The Primer is organized to allow allows users to: Search by disease; Search by drug targets (e.g.
kinases); Search by technology (e.g. vaccines); Diagnostics; Pipeline for each disease, target, technology;
Updated via data collection directly from R&D sources, media, publications
Afternoon session: opportunities to develop a more coordinated approach to R&D resource flow
mapping at a global level.
The afternoon session focused on identifying the challenges and opportunities that exist in developing a
more coordinated approach to R&D resource flow mapping at a global level. Three groups worked on
these issues and reported back separately. These findings have been collated below under xx headings
that summarizes the general recommendations of this group.
Summary
1) Build on what is there
In designing any new mechanism or even going as far as establishing an observatory for health R&D at a
global level, the approach should be to build on what is already there. This includes the experience of
the organizations that presented at this meeting but also spend more effort at identifying other R&D
Page 13 of 16
relevant initiatives and perhaps data collection and analysis outside of the R&D sectors e.g. Amazon,
Google, meteorological work. Try to adapt or scale existing survey and use methods that are already
familiar e.g. the Frascati Manual. Identify the key standards that are needed and advocate for their use
e.g. unique research IDs (ORCID) or a standard number for grant classification. Be conscious of good
practice in data management and ensure the necessary safeguards on privacy and confidentiality are
respected.
In principle any global observatory should be an aggregate of what is available at regional, national or
sub-national level. The global system is a high level analysis the detailed granular data can be found at
the contributing sites.
2) Build Small – think big
Try to identify the essential key data points that describe the research in enough detail to be useful but
not so much detail as to be a burden to collect. Identify what is the minimum that could be collected
that would add value. This might be as small as 5 or more data points. Avoid collecting unnecessary
data as it adds to the burden – if the contributors have to work too hard to generate new data it is
unlikely to happen.
Note the achievements of the International Clinical trial Registry Platform (ICTRP)2 which has 21 meta-
data points even though clinical trials have far more in their protocols and at country level registries. At
first all data collection will require substantial manual input to re-code and quality check as there is no
automated system out there ready to be used. If a minimum data set can be defined, again see ICTRP
then automated data aggregation becomes feasible. In time this core set of data points could be
expanded but it is better to get any global mechanism populated with data. Benchmarking (described
here as the power of ‘envy and embarrassment’) is a powerful tool in getting others to comply with data
reporting. No one likes to be seen as the group with ‘data not available’ next to their name. Try getting
the big donors involved first but not loose site of the fact this should be an iniative that is first and
foremost responding to the needs of countries.
3) Identify value added – at all levels
What is in it for the contributors? How do stakeholders benefit? There is a need to undertake
stakeholder analysis to identify what users and contributors want – and what they will support (See G-
FINDER example above). At a country level can the incentive and ‘reward’ for collecting and supplying to
be identified: can the data improve research management, will it improve the success of attracting
internal and external funding to research, will it empower recipients to set priorities and be successful in
negotiating with donors to fund those priorities.
4) Get going – grow and adapt
Balance the need for planning and partnership with many stakeholders with getting pilot studies up and
running and making data available as it is easier for people to react to what is there than come at this
2 http://www.who.int/ictrp/en/
Page 14 of 16
from a blank perspective. If value can be shown, and trust and credibility built in the mechanism, then
information is more likely to flow.
Conclusions and next steps
The discussion points and recommendations presented here will be used by WHO as it moves forward
developing the concept of a mechanism to improve monitoring of health R&D at a global level and the
potential for establishing a global health R&D observatory.
The Member States of the World Health Organization will debate this issue at the next World Health
Assembly 20-28th May 2013.
For further information contact:
Robert Terry
Programme Manager - Research, Development and Policy
Public Health, Innovation and Intellectual Property (HIS/PHI)
World Health Organization
Avenue Appia
CH-1211, Geneva 27
Switzerland
T: +41 (0)22 791 2632
M: +41 (0)79 244 6091
Skype: rterry_who
http://www.who.int/phi
Page 15 of 16
Agenda
WHO informal workshop - monitoring R&D resource flows
from 08.30 Coffee available Speaker
09:00 Introductions All
09:15 Context - objectives for the day - background to the global health R&D Observatory
Rob Terry
Overview of current/recent R&D monitoring initiatives (max 5 slides/ max 10 minutes each)
1.Global Forum for Health Research Stephen Matlin/Alison Young
2. OECD Alison Young
3. UNESCO Rohan Pathirage
4. RIYCT / PAHO Rodolfo Barrere /Analia Porras
10:00 - 12:00 5. G-Finder Javier Guzman
6. HRWeb David Abreu/Luis Gabriel Cuervo
7. World RePORT James Herrington/Sandeep Somaiya
8. Research Africa William Cullerne Brown /Gerard Ralphs
9. Health Research Classification System Ian Vinney
10. Global Health Primer Theresa Wicklin Gillespie/Elizabeth Ponder
12:00 - 12:30 Summary discussion All
12:30 - 14:00 Lunch
14:00 - 16:00 Challenges and Opportunities: -political -economic -social -technical -legal -ethical/environmental
All
16:00 - 17:00 Summary - potential next steps Rob Terry
Page 16 of 16
Participants
WHO informal workshop - monitoring R&D resource flows
Name Organization Contact
Elizabeth Ponder Berkeley University - Ex Bio Ventures for Global Health
Jennifer Dent Bio Ventures for Global Health [email protected]
David Abreu COHRED - HRWeb [email protected]
Theresa Wicklin Gillespie
Emory University School of Medicine - Global Health Primer
James Herrington Fogarty International Center - World RePORT [email protected]
Anna Wang Bill & Melinda Gates Foundation [email protected]
Stephen Matlin Institute of Global Health Innovation, Imperial - ex-Director Global Forum for Health Research
Ian Viney Medical Research Council - Health Research Classification System
Sandeep Somaiya NET ESolutions - World RePORT [email protected]
Alison Young On behalf of OECD [email protected]
Javier Guzman Policy Cures - G FINDER [email protected]
William Cullerne Brown
Chair, Research Africa [email protected]
Gerard Ralphs Research Africa [email protected]
Rodolfo Barrere RICYT [email protected]
Rohan Pathirage UNESCO Institute for Statistics Science, Technology and Innovation Unit
Val Snewin Wellcome Trust [email protected]
Liz Allen Wellcome Trust [email protected]
Rob Terry WHO [email protected]
Beatrice Halpaap WHO/TDR [email protected]
Anilia Porras WHO/PAHO [email protected]
Luis Gabriel Cuervo
WHO/PAHO [email protected]