measuring r&d in developing countries: annex to the frascati manual
DESCRIPTION
Measuring R&D in Developing Countries: Annex to the Frascati Manual. West Africa Regional Science, Technology and Innovation Policy Reviews and Statistics Workshop Bamako, Mali 10-13 May 2010. Outline. The problem The process - PowerPoint PPT PresentationTRANSCRIPT
www.uis.unesco.org
Measuring R&D in Developing Countries: Annex to the Frascati Manual
West Africa Regional Science, Technology and Innovation Policy Reviews and Statistics WorkshopBamako, Mali
10-13 May 2010
www.uis.unesco.org
Outline
The problem
The process
Contents of the Technical Guide “Challenges Facing the Measurement of R&D in Developing Countries’
Thinking ahead
www.uis.unesco.org
The problem
Recognition, meeting targets, evidence-based S&T policy, but:
• lack of interest at the level of policy makers (low policy-relevance?)
• S&T is still not properly represented in economic/social public policies. Lack of resources devoted to statistics in S&T
• lack of technical knowledge for the production of cross-nationally comparable R&D statistics
• difficulties in applying FM concepts and methods
• weak statistical institutions
www.uis.unesco.org
The process (1)
Experience acquired through the UIS work, in particular through direct contact with S&T statisticians in numerous workshops and other meetings around the developing world.
Advisory Meeting to the UIS S&T Statistics Programme held in Montreal, Canada, December 2007.
Papers commissioned by UIS to Jacques Gaillard (IRD, Paris), Michael Kahn et al (HSRC, South Africa), and Gustavo Arber et al (RICYT, Argentina).
Proposal for an annex to the Frascati Manual on measuring R&D in developing countries was presented at the OECD 2008 and 2009 NESTI meeting.
www.uis.unesco.org
The process (2)
Expert Meeting on Measuring R&D in Developing Countries in Windhoek, Namibia, 14 to 16 September 2009.
Consultant drafted:• Working paper on Measuring R&D in Developing
Countries
• Proposed Annex to the Frascati Manual
Both to be released in 2010
Some of the issues might also present measurement challenges for a future revision of the Frascati Manual.
www.uis.unesco.org
Main outcomes of the Namibia meeting
Developing countries a very heterogeneous concept
Problems not unique to developing countries
Stay within boundaries of FM: additional areas may be addressed within FM framework
Most recommendations stood up
Much additional work needed
www.uis.unesco.org
Contents of the Technical Guide
1. Introduction
2. The growing importance of R&D in developing countries
3. Information on R&D expenditure
4. The R&D workforce, internal and international mobility
5. Specific fields of R&D activity
6. Foreign and internationally controlled entities
7. Strengthening R&D statistics systems
8. Thinking ahead
www.uis.unesco.org
2. R&D systems in developing countries
Particular characteristics of R&D activities to be taken into account:
• R&D performers function within the specific context of a national, cultural, political, financial and economic system, frequently carrying with them the legacies of colonial, post colonial and other forms of governance.
• different structures in terms of state/government, research/innovation system, higher education system, statistical system.
• particular ‘culture of information’
• Users of R&D stat: Gov, analysts. + international donor agencies
However, international comparability is foremost.
www.uis.unesco.org
Nature of R&D activities
There is often more ‘R’ than ‘D’ in developing countries.
Strong presence of the government and higher education sectors in the performance of R&D.
Degree of informality is common. Informal R&D is difficult to capture and therefore it is usually considered beyond the scope of R&D surveys.
S&T indicators need to be adapted to particular policy needs, and need to provide answers to actual policy questions of the developing countries.
www.uis.unesco.org
Heterogeneity and concentration
R&D activities and their institutional framework present distinctive characteristics structures needs to be properly understood.
Developing countries are a heterogeneous group:• Group A: countries with consolidated R&D systems and developed
S&T statistics systems no major difficulties in applying Frascati Manual concepts.
• Group B: countries with consolidated R&D systems and less developed S&T statistics systems need specific guidance on how to establish and consolidate sound R&D statistics systems.
• Group C: countries with incipient R&D systems need specific guidelines on how to start creating a regular R&D statistical collection.
High degree of concentration (in group of countries, in particular institutions, in major projects, etc) lead to volatility and inconsistencies in statistics.
www.uis.unesco.org
Other issues
Informal R&D:
• Occasional R&D
• R&D in the informal sector / Informally organized R&D
difficult to measure / not peculiar to developing countries.
www.uis.unesco.org
3.Patterns in research funding and budgeting
R&D used to be largely funded by the government, but now new sources of funds are emerging (Foundations, NGOs, foreign organizations, private business) use of public sector budgetary information (widely used in developing countries as GERD proxy) is no longer valid.
Funding in developing countries provide support to individuals and groups rather than institutions remain unaccounted for, seldom declared.
Discrepancy between voted and allocated budgets; confusions between S&T and R&D budget; difficulties in identifying R&D components in the national budget; lacking separate research budget; budget commitments are not frequently followed up raises problems with their use for GERD estimates; lead to an over- or underestimation; use of different classification limit the availability of key data; source of funds accounted for budget data create incompatibilities with FM classifications; use of combination of budget data and information from annual reports from performing units lead to double counting; capital expenditure frequently unaccounted.
www.uis.unesco.org
4. Researchers and research profession
Underestimation of researchers
Unpaid research
Informal research
Research outside of the normal work setting with external funding
Multiple part time positions not taken into account or undercounted (“Taxi –professors”)
Master’s research
www.uis.unesco.org
Researchers and research profession
Overestimation of researchers
Counting the contract instead of the real effort (research-professors or enseignant-chercheur)
Multiple full-time research positions
Special cases
FTE calculation >1 and FTE>HC
R&D in times of crisis
Visiting researchers
Brain circulation
www.uis.unesco.org
Counting researchers
Recommendations
Peer interviews of researchers
Include a module on barriers
Use secondary sources• Publication databases, both national and international
• STMIS and other databases of researchers
• Databases and registers of clinical trials
• Databases and registers of the main foreign donors involved in funding R&D in the countries
• University accreditation databases
www.uis.unesco.org
5. Dealing with special types of R&D - Traditional knowledge
Traditional knowledge
A cumulative body of knowledge, know-how, practices and representations maintained and developed by peoples with extended histories of interaction with the natural environment.
These sophisticated sets of understandings, interpretations and meanings are part and parcel of a cultural complex that encompasses language, naming and classification systems, resource use practices, ritual, spirituality and worldview.
Dichotomy between traditional and scientific knowledge systems
• substantive grounds – because of differences in the subject matter and characteristics of traditional and scientific knowledge.
• methodological and epistemological grounds – because the two forms of knowledge employ different methods to investigate reality.
• contextual grounds – because traditional knowledge is more deeply rooted in its environment.
www.uis.unesco.org
Special types of R&D - Traditional knowledge
Links between traditional and scientific knowledge systems
Traditional knowledge (in general) as an object of scientific study (ethno-botany, ethno-pedology, ethno-forestry, ethno-veterinary medicine, ethno-ecology, etc).
Application of scientific methods to traditional knowledge, converting it into a source of scientific information (in biodiversity science or nature conservation).
Application of science to unlocking the potential of traditional knowledge (research on traditional medicinal practices, traditional pharmacopeia, etc).
Interaction between scientists and communities in participatory technology development using the traditional practices.
www.uis.unesco.org
Special types of R&D - Traditional knowledge
Measurement issues and recommendations
Establish the boundaries of R&D for the purposes of the Frascati Manual related to traditional knowledge.
The activities establishing an interface between traditional knowledge and R&D (also the cases where R&D component can be appropriately measured), are to be counted as R&D activities, The production, storage and communication of traditional knowledge, in traditional ways, should not be counted as R&D.
Need to consider particular scientific disciplines currently not explicitly incorporated into the classification of Fields of Sciences. Some of these fields are trans-disciplinary (e.g. ethno-botany), making them extremely difficult to map into the current classification’s structure.
www.uis.unesco.org
Special types of R&D - Clinical trials
Clinical trials
(Can) involve a significant amount of R&D
Growth area for developing countries (outsourcing of R&D, decentralization of the laboratories, activities of pharmaceutical companies, need to conduct clinical trials among a wide population of potential users).
www.uis.unesco.org
Special types of R&D - Clinical trials
Measurement of clinical trials
Registers of clinical trials available, e.g. WHO but also national level
Funding often from abroad (headquarters of the pharmaceutical companies involved)
Different types of performing units:• a local branch of the foreign main sponsor
• universities and university hospitals
• individual researchers
• local medical clinics
• locally registered PNPs
• international PNPs
www.uis.unesco.org
Special types of R&D - Clinical trials
Measurement issues and recommendations
Occupation category of local staff
• Medical doctors and other professionals with at least ISCED 5A degrees should be considered as researchers
• Nurses and other staff with qualifications below ISCED 5A should be accounted for as technicians
FTE calculation is important (often part-time)
Attribution of sector of performance must be done with care to avoid double counting.
www.uis.unesco.org
Special types of R&D - Industrial activities
Reverse engineering: understanding the structure and functioning of an object (in order to make a new device or program creates a similar object in a different way), copying it, or improving it.
Recommendation: If reverse engineering is carried out in the framework of an R&D project to develop a new (and different) product, it should be considered as R&D.
This is dealt within FM measurements.
Community development and other social projects should be considered R&D only as long as they are in a development and testing phase, in which case they should be counted as experimental development, most probably in the field of social sciences
This falls within Social science R&D activities.
In some developing countries, religious research has a particular importance. In principle, religious research is a part of humanities, and institutions performing it should be included in R&D surveys.
This falls within Humanities R&D activities.
www.uis.unesco.org
6. The foreign institutions sector
Recommendation
Create a “foreign institutions” (FI) sector as a separate sector of performance to make the resulting data more policy relevant.
Funding flowing from this sector to other sectors should be considered from “Abroad” as stated in the main body of the Frascati Manual
What is included?
Foreign antennas
International organizations
Foreign company’s R&D labs (remains in the BE sector)
Foreign universities (remains in the HE sector)
www.uis.unesco.org
Foreign research centres
“Foreign antennas”, or foreign research centres, are based in the country, but have foreign researchers and foreign funding direct impact on R&D measurement and on the use of R&D stats in policy making; strongly distort the countries’ R&D indicators.
Foreign research labs/MNCs set up by foreign companies may cater to the R&D needs of their headquarters, with decision making taking place outside the host country little involvement of the local innovation system; need to distinguish such foreign-owned institutions.
International organizations with R&D activities, involving local staff and addressing local issues significant weight in the total GERD and R&D personnel.
Foreign universities based and conducting R&D in campuses set up in the country. pose particular problems, not accounted for in the FM.
www.uis.unesco.org
The foreign institutions sector
The principal sector sub-classification
Business enterprises
Government
Higher Education
Private non-profit
International organizations
Practical consequences of introducing the Foreign Institution sector
FI sector should be treated at the same level as other sectors of performance.
Specific questionnaire for the FI sector should be designed, addressing the particular characteristics of these institutions (demographic characteristics of researchers, nationality and country of birth, parameters related to the internationalization of R&D, linkages between these institutions and the national innovation system).
Resulting data for the FI sector should be published separately from other sectors.
www.uis.unesco.org
7. Strengthening S&T statistics systems in developing countries
Institutionalizing S&T statistics
Establishing registers
Structural issues in the private sector and the private not-for-profit sector
User-producer networks
Science & Technology Management Information Systems and other secondary sources
Survey procedures and estimation
www.uis.unesco.org
Institutionalization of S&T statistics
Political support
Infrastructure and sustained staff training/capacity building
Involvement of NSOs: “Official statistics” status for R&D surveys.
Adequate legal framework
www.uis.unesco.org
Establishing registers
R&D in developing countries tends to be very much the purview of public bodies
Recommendations:
Establishing a database of public sector R&D projects
• include human and financial resources; align with national policies.
• design could reflect the R&D statistical reporting/definitions.
• source for evaluation of such projects.
Establishing STMIS
• provide overview of research system.
• framework for establishing complete registers as sample frames for R&D surveys.
www.uis.unesco.org
Establishing registers
Other sources
• associations (trade, academic).
• learned societies.
• registers or databases of scientists and engineers.
• database of research grants.
• databases of scientific publications.
• patents and other IP documents.
• business registers.
www.uis.unesco.org
Structural issues in the private sector and the PNP sector
Publicly-owned businesses play a major role in R&D in some developing countries
Recommendations:
• should consider issuing data for ‘publicly-owned businesses’ separately from the ‘fully private enterprise sector’.
• private enterprises could also be disaggregated by ownership, in particular the various degrees of foreign ownership.
www.uis.unesco.org
Structural issues in the private sector and the PNP sector
Business enterprise R&D is presumed to be generally weak in developing countries when compared to industrial countries.
Recommendations:• this fact needs to be taken into account when conducting sample
surveys, perhaps by over-sampling, especially amongst larger companies.
• big companies should not be missed out as it might imply significant error.
• countries should invest some time in interviewing key firms to understand their R&D function and obtain a clear picture of their activity.
High-tech companies: R&D has more added value.
Recommendations: • need for careful identification of potential R&D actors.
Private-non-profit sector: make a significant contribution to R&D in developing countries, but the sector tends to be very volatile.
www.uis.unesco.org
User-producer networks
Recommendations: • user-producer networks and other forms of stakeholder
consultation should be instituted.
• establishing national S&T statistics groups.
• involve multiple actors.
• coordinating/networking among institutions/databases.
• partnering with business associations.
• conducting face-to-face visits by statisticians and project leaders.
• exploit pre-existing personnel ties.
• get NSO involved; to deal with privacy of information.
• training of interviewers/primary data producers.
www.uis.unesco.org
Science and Technology Management Information Systems and other secondary sources
STMIS (such as CV-LAC, database of scientists, research grants, etc): frequent source for the production of R&D statistics.
Recommendations: • need close integration between the statistical system and the
STMIS.• need adjustments to produce comparable statistics, taking into
account issues of definitions and coverage.• need a balanced approach using both STMIS and surveys.• need different approach to Private sector organizations as they are
frequently not covered by these systems.
Combined R&D and innovation surveys
Recommendations: • the relative rarity of occurrence of R&D in businesses needs to be
taken into account.
www.uis.unesco.org
Survey procedure and estimation
Recommendations:• attention needs to be paid to questionnaire design.
• frequency of survey.
• prioritize area of work; accompanied by step-by-step approach.
• use of survey questionnaires of other countries for inspiration: need adaptations to local situation.
• get expertise from the NSO, in conducting survey, in sampling ….
• different questionnaires might be designed for different sectors based on stakeholder consultations; “one size does not fit all”.
• procedures need to be developed for estimating missing data.
www.uis.unesco.org
8. Thinking ahead: Other products – beyond R&D
Redefine the concepts of scientific and technological education and training at broadly the third level (STET), Scientific and technological services (STS) and S&T activities (STA)
Better integrate education statistics with R&D statistics
Hands on guidance
Metadata
Model questionnaire
