world bank document · venture capital (vc)industry by establishing a legal and regulatory...

Document of The World Bank

Report No:ICR0000322

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-44950)

ON A

LOAN

IN THE AMOUNT OF US$155.0 MILLION

TO THE

REPUBLIC OF TURKEY

FOR AN

INDUSTRIAL TECHNOLOGY PROJECT

December 19, 2006

Private and Financial Sector Department and Turkey Country Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

( Exchange Rate Effective 12/01/2006 )

Currency Unit= Turkish Lira

Turkish Lira 1.00= US$ 0.78

US$ 1.00= Turkish Lira 1.32

Fiscal Year

January 1- December 31

ABBREVIATIONS AND ACRONYMS

CAS Country Assistance Strategy

CPIU Central Project Implementation Unit

ECU European Customs Union

EDP Economic Development Program

FMS Financial Management System

ICR Implementation Completion Report

IPR Industrial Property Rights

KOSGEB Small and Medium Enterprise Development Organization

MAM Marmara Research Centre

MSTQ Measurement, Standards, Testing and Quality

NGO Non-Governmental Organization

OECD Organization for Economic Cooperation and Development

PAD Project Appraisal Document

PCT Patent Cooperation Treaty

PIP Project Implementation Plan

PIU Project Implementation Unit

PPF Project Preparation Facility

R&D Research & Development

SFA Subsidiary Finance Agreement

SIS State Institute of Statistics

SME Small and Medium Enterprise

TDF Technology Development Finance

TDP- I Technology Development Project -I

TP Technopark

TPE Turkish Patent Institute

TSC Technology Services Center

TSE Turkish Standards Institute

TSS Technology Support Services

TTGV Technology Development Foundation of Turkey

TUBITAK Scientific and Technological Research Council of Turkey

TURKAK Turkish Accreditation Agency

UFT Undersecretariat of Foreign Trade

UME National Metrology Institute

VC Venture Capital

VCF Venture Capital Fund

WTO World Trade Organization

Vice President: Shigeo Katsu

Country Director: Andras Horvai

Sector Manager: Gerardo M. Corrochano

Project Team Leader:Ahmet Gurhan Ozdora

Turkey Industrial Technology Project

CONTENTS

1. Basic Information........................................................................................................ 12. Key Dates .................................................................................................................... 13. Ratings Summary ........................................................................................................ 14. Sector and Theme Codes............................................................................................. 25. Bank Staff.................................................................................................................... 26. Project Context, Development Objectives and Design ............................................... 37. Key Factors Affecting Implementation and Outcomes............................................... 78. Assessment of Outcomes ............................................................................................ 99. Assessment of Risk to Development Outcome......................................................... 1610. Assessment of Bank and Borrower Performance.................................................... 1611. Lessons Learned...................................................................................................... 2112. Comments on Issues Raised by Borrower/Implementing Agencies/Partners......... 23Annex 1. Results Framework Analysis ......................................................................... 24Annex 2. Restructuring (if any)..................................................................................... 28Annex 3. Project Costs and Financing .......................................................................... 29Annex 4. Outputs by Component.................................................................................. 31Annex 5. Economic and Financial Analysis (including assumptions in the analysis).. 35Annex 6. Bank Lending and Implementation Support/Supervision Processes............. 47Annex 7. Detailed Ratings of Bank and Borrower Performance .................................. 50Annex 8. Beneficiary Survey Results (if any)............................................................... 51Annex 9. Stakeholder Workshop Report and Results (if any) ...................................... 58Annex 10. Summary of Borrower’s ICR and/or Comments on Draft ICR ................... 59Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders ..................... 60Annex 12. List of Supporting Documents..................................................................... 61MAP .............................................................................................................................. 62

1

1. Basic Information

Country: Turkey Project Name: Industrial Technology Project

Project ID: P009073 L/C/TF Number(s): IBRD-44950

ICR Date: 12/19/2006 ICR Type: Intensive Learning ICR(ILI)

Lending Instrument: SIL Borrower: GOT Original Total Commitment:

USD 155.0M Disbursed Amount: USD 152.0M

Environmental Category:CImplementing Agencies

Marmara Research Center National Metrology Institute Turkish Patent Institute Turkish Technology Development Foundation Cofinanciers and Other External Partners

2. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 04/20/1998 Effectiveness: 10/18/1999 10/18/1999 Appraisal: 09/26/1998 Restructuring(s): Approval: 06/17/1999 Mid-term Review: 07/15/2001

Closing: 12/31/2003 04/30/2006

3. Ratings Summary 3.1 Performance Rating by ICR Outcomes: Highly Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory

2

3.2 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating:

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA): None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA): None

DO rating before Closing/Inactive status:

Highly Satisfactory

4. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing) Law and justice 11 General industry and trade sector 89 100

Original Priority Actual Priority

Theme Code (Primary/Secondary) Personal and property rights Primary Secondary Small and medium enterprise support Primary Secondary Other financial and private sector development Primary Secondary Export development and competitiveness Primary Primary Technology diffusion Secondary Primary

5. Bank Staff Positions At ICR At Approval

Vice President: Shigeo Katsu Johannes F. Linn Country Director: Andras Horvai Ajay Chhibber

Sector Manager: Gerardo M. Corrochano

Lajos Bokros

Project Team Leader: Ahmet Gurhan Ozdora

Vinod K. Goel

ICR Team Leader: Vinod K. Goel ICR Primary Author: Vinod K. Goel

3

6. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative)

6.1 Context at Appraisal (brief summary of country macroeconomic and structural/sector background, rationale for Bank assistance)

The macroeconomic conditions in Turkey in the 1990s were not conducive for R&D. The inflation rate remained very high (above 50 percent), and highly variable. The fragile banking system, whose primary function was to finance public deficit, and volatile capital flows and exchange rates increased economic risk and uncertainty. The economy went through boom-and-bust cycles: although the average growth rates were moderate, manufacturing achieved high growth rates for a few years followed by sharp contractions (the average annual growth rate of manufacturing output was 5.5 percent for the period 1995-2005) . Turkey started liberalizing its foreign trade policies in the 1980s and initiated a number of changes in other policy areas (including export subsidies, intellectual property rights and competition policy) in the early 1990s, mainly as a result of international agreements (GATT/WTO) and preparations for the European Customs Union (ECU). In January 1996, a custom union agreement was signed between Turkey and EU, allowing most industrial goods to pass freely between the partners. While access to ECU offered Turkey a unique opportunity to accelerate its development through freer and better access to markets, it also made Turkish firms more vulnerable to international competition. In order to take advantage of trade opportunities created by the ECU and respond effectively to greater competition in the domestic market, Turkish industry needed to upgrade from low-quality, labor-intensive products towards the production of higher value-added goods and services. Although the Scientific and Technological Research Council of Turkey (TUBITAK) was established in 1963, Turkey did not mount a comprehensive technology policy until the early 1990s. In 1993, Turkish Government approved a comprehensive policy document, Turkish Science and Technology Policy: 1993-2003. This document set four targets: increasing the number of researchers per 10,000 people from 7 to 15; raising the GERD (gross expenditure on R&D) to GDPratio from 0.3 percent to 1.0 percent; moving up in the rank of scientific publications from 40th to 30th position; and increasing the share of business in total GERD from 18 percent to 30 percent. The Government recognized its crucial role in providinga conducive environment for technology development. In the 1990s, various laws regulating industrial property rights were passed and several institutions were established or reorganized: TUBITAK, Technology Development Foundation of Turkey (TTGV), the National Metrology Institute (UME), Small and Medium-sized Industry Development Organization (KOSGEB), R&D support schemes, Turkish Patent Institute (TPE), Turkish Accreditation Agency (TURKAK), technology development zones/technoparks, etc. In order to further encourage innovation among Turkish industry, the Government in 1995 started a scheme known as TIDEB (currently renamed as TEYDEB) to provide grants to

4

private firms for R&D activities. This scheme was administered by TUBITAK in collaboration with TTGV. The government also granted public research institutions some degree of autonomy to become more market-orientedand reduce their dependence on government funding. The Government’s efforts to improve technology infrastructure and services were supported under the World Bank’s Technology Development Project (TDP-I, a US$100 million Bank Loan in 1991-1997) aimed at: (i) bringing the MSTQ system in Turkey to OECD standards, (ii) supporting private sector investment in industrial technology development by providing seed capital and subsidized grants/loans, and (iii) developing a venture capital (VC)industry by establishing a legal and regulatory framework, rationalizing the tax treatment of venture capital funds (VCFs), and financing through the IFC a role model VCF and management company. The Government, to build upon the success of TDP-I, requested the Bank support for a follow-up project and the Industrial Technology Project (ITP) was designed on the ground work laid down by the TDP-I. It aimed at both further strengthening successful components of TDP-I (metrology services and R&D funding), and expanding into new activities (public R&D institutions and IPRs). Although both the TDP-I and ITP projects aimed at supporting technological, legal and regulatory framework for technological activity, promoting private sector R&D, and linking technological activities in the private and public sectors, there were two significant differences between TDP-I and ITP in terms of their approach. TDP-I was mainly involved inthe establishment of three organizations (TTGV, National Accreditation Council and the VCF/VCM), and required legal and regulatory work for their existence. ITP also aimed at the creation of new organizations (VCF, technoparks, etc.) and laws (on technology parks, IPRs, etc.), but these were only to form a part of the participating organizations’ activities, and the Project’s success was not critically dependent on their realization. As such, the ex anterisks involved in ITP were lower, and ITP was successful in realizing its developmental objectives.

6.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The major objectives of the Project, as identified in the Project Appraisal Document (PAD) were to:

• Assist in the harmonization of Turkish technology infrastructure with European Customs Union standards; and

• Assist firms in upgrading their technological capabilities in order to improve the competitiveness of Turkish industry, both in domestic as well as foreign markets.

6.3 Revised PDO and Key Indicators (as approved by original approving authority), and reasons/justification The Project Development Objectives were not revised.

5

6.4 Main Beneficiaries, original and revised (briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project)

The main beneficiaries of the project were:

• Exporters, to have access to technologies that were previously unavailable or inadequate, giving them better access to new markets;

• Domestic firms, to be able to increase their competitiveness through greater awareness, adoption, and modification of technologies; and

• The Government, to have less strained budget due to the increasing capability of some of the institutions to earn a greater portion of their incomes through the sale of their services.

Other beneficiaries include: Implementing Agencies: Technology Development Foundation of Turkey, National Metrology Institute, Marmara Research Center, and Turkish Patent Institute 6.5 Original Components (as approved) The Project components were related to achieving the Project Objectives. The objectives were planned to be achieved through the following four components: A. Strengthening of Industrial Property Rights Services; B. Strengthening of Metrology Services; C. Restructuring of R&D Institutions; and D. Supporting Technology Upgrading by Firms The Bank funds were lent to the Treasury to be passed on to Project Agencies in the form of soft loans and grants, under Subsidiary Finance Agreements (SFA). The Turkish Patent Institute was responsible for the strengthening of IPR services component; the National Metrology Institute for the metrology services component; the Marmara Research Center (MAM), the leading public research organization in Turkey, under TUBITAK, for the restructuring of R&D institutions component; and the Technology Development Foundation of Turkey for supporting the technology upgrading activities of Turkish firms. Component A: Strengthening of Industrial Property Rights (IPR) Services

Component was aimed at supporting activities necessary to bring Turkish IPR system in conformance with ECU requirements. It was to be done by: (i) helping the TPE in improving its operation and procedures in line with international agreements; (ii) helping to reorganize the workflows of TPE staff (training, upgrading data systems, etc; (iii)

6

increasing the general awareness on the IPR regime; (iv) establishing dedicated information centers for documentation and dissemination of information, and (v) supporting TPE in improving the IPR enforcement through improved training for judicial officials. Component B: Strengthening of Metrology Services

This component was intended to support the expansion of UME facilities to serve a larger portion of Turkey’s metrology needs, including chemical and medical metrology. The plan was to cover 90 percent of physical measurement standards and service needs by the end of the Project. UME was to transfer new metrology technologies to industry by consultancy services, training, publications, etc. The Project was to provide financing for laboratories (new buildings), equipment, consultancy and training. Component C: Restructuring of R&D Institutions

This component was to support the restructuring of MAM research institutes. The tasks were designed to: (i) restructure the Center and institutes so as to decentralize management responsibility, and strengthen management by introducing strategic management, change management, business development, annual business plans andbudgetary procedures, and full cost accounting principles; (ii) establish profit centers for each institute; (iii) develop human resources; (iv) upgrade its infrastructure and laboratory facilities, IT network and office equipment; and (v) invest in a Technopark facility, if possible. Component D: Supporting Technology Upgrading by Firms

This component was aimed at continuing the TTGV support for technological upgrading activities (R&D funding). This was intended to be done through: (i) supporting TTGV institution building efforts for a successful transformation to a greater risk-taking and a more hands-on approach; (ii) providing funding for new initiatives where TTGV would have a catalyzing role (technology service centers TSC), VCFs for high-tech industries and Technopark investment); (iii) offering technical and managerial support to SMEs; and (iv) assisting in the establishment of a National Accreditation Council (TURKAK). 6.6 Revised Components The Project Components were not revised. 6.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations)

No significant changes

7

7. Key Factors Affecting Implementation and Outcomes 7.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable)

The Project was intended to respond to the growing realization within the country that weak technological activity and the absence of a research culture, with heavy and continuous dependence on imported technologies, would hinder its growth prospects in a liberalized global economy. The Project was introduced at a very appropriate time. The Customs Union agreement with the European Union increased the awareness of the need for stronger local R&D capabilities in Turkey. It became important for enterprises not only to improve production efficiency and quality but also to invest in technology and introduce new products. Thus, the Bank’s funding and assistance were timely in supporting and providing impetus for the Government’s reforms. Project objectives were consistent with the objectives of the Country Assistance Strategy (CAS). They clearly reflected government priorities, and complied with applicable Bank safeguard policies. Project design incorporated valuable experiences and lessons learned from other Bank-financed projects and it was built on the successes of the TDP-I, reflecting important lessons learnt from that Project on institutional development matters. The PAD documented the Project and its background in sufficient detail. Duringthe process of project preparation, a participatory approach was adopted: numerous meetings were held with Turkish counterparts and stakeholders to determine the appropriate interventions for each component as well as to assess the potential of risk financing. For UME, for example, project objectives and design were based on a feasibility study performed by the National Physical Laboratory of the United Kingdom, where conclusions of the study fed into the design of the metrology component of ITP. Key project stakeholders participated in appraisal and loan negotiations and the proposed implementation arrangements were considered adequate. There were several innovations in ITP as compared to TDP-I. First, a Central Project Implementation Unit (CPIU) was tobe established at the TTGV to coordinate common aspects of the Project. Second, four special accounts would be opened, one for eachProject Agency (there was one joint account for all organizations in TDP-I which did not work well). Third, ITP envisaged an independent monitoring and evaluation (M&E) mechanism with three sets of indicators to monitor Project progress and achievements: input, output, and outcome indicators. 7.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable)

The macroeconomic crisis of 2001 caused some inevitable delays on some of the Project activities. Apart from creating problems in budgeting and counterpart funding, the crisis led to a period ofinactivity within the country in various sectors, which slowed down the progress in some components, such as MAM and TTGV services. After consultation

8

with the Bank, the realization of the initial MAM target self-sufficiency rate (70 percent) was postponed for two years. Nevertheless, this target encouraged MAM to reorganize its research institutes, where, textiles and genetic research institutes were separated from MAM in 2000, thereby improving its focus on applied research and overall effectiveness. 7.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The ITP had a strong independent monitoring and evaluation mechanism. The monitoring and evaluation indicators were chosen in consultation with the participating organizations onthe basis of the respective Project inputs, conceptual relevance to Project objectives, ease of calculation and parsimony. Three sets of indicators were used to monitor Project progress and achievements:

• Input indicators to keep track of the extent to which implementation met stated expectations and schedule;

• Output indicators to track the immediate results of implementation in accordance with project plans;

• Outcome indicators to evaluate the extent to which the Project has achieved its objective of improving technology development in Turkish industry.

As a part of the monitoring process, the implementing institutions regularly collected data on input and output indicators. The development impact was measured by analysis of survey data collected with thehelp of the State Institute of Statistics (SIS). Three surveys were planned: at the beginning of the Project to benchmark firm performance, mid-term for monitoring and feedback, and towards the end of the Project for final evaluation. The first survey (and the evaluation study) was completed in 2000, and the second one in 2003. An interim review was conducted in 2004, and in 2005, interviews were conducted with firms that received technology development financing from TTGV to get information about long term (growth) effects of the R&D support program with the final monitoring and evaluation report prepared in May 2006. 7.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable)

The Project complied with all applicable Bank safeguard policies. 7.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutionalcapacity, and next phase/follow-up operation, if applicable)

The sustainability of the Project is very likely. Although the Project was relatively modest in its financial implications, its catalytic effects on industrial competitiveness

9

and innovation are likely to be significant and to extend over the long-term. To the extent that a strong technology infrastructure and intellectual property rights regime,combined with an active innovation culture in industry, are vital to the growth and competitiveness of the productive sector (the impact will spill over into modern services), such projects can help countries transform their productive structures and cope with the competitive rigors of globalization. The evidence suggests that ITP raised technologicaleffort and export competitiveness, both of which are of direct relevance to all developing countries. The Project upgraded and strengthened the technology infrastructure institutions which have led to lower costs for Turkish firms, closer links between firms and public R&D institutions and greater attention to IPRs. Also, the Project raised the awareness of the importance of R&D and showed firms how to design andmanage R&D projects, disciplined their R&D teams, and put them in close contact with academics in the relevant disciplines. It is important to note, that, as evidenced by the TDP-I and the Industrial Technology Development Project in India (1991-1997), the full outcomes of the Project are likely to become visible in the medium-term, i.e. in three to fiveyears after project closing. The Turkish Government recognizes the importance of innovation infrastructure for enhancing its competitiveness, generating more and better quality jobs, and fully aligning with the EU standards. The issue has becomeparticularly important in the context of the ongoing discussions for EU accession. The 2003 EU Innovation Communication identified for the first time specific challenges that candidate countries need to address in the knowledge areas in order to improve the performance of the enlarged EU, including improving countries’ institutional capacity to foster innovation, embedding innovation in several policy areas and strengthening the role of the private sector in innovation.

8. Assessment of Outcomes

10

with the ECU and the World Trade Organization (WTO), the Project was well timed. The Project was responsive to the needs of the Borrower. It was designed to improve the productivity in the technology area. Specifically, the Project addressed two important market failures in the area of technology-services. First, there was the demand-side failure: many Turkish entrepreneurs had underestimated the need for technology improvements and were reluctant to spend scarce funds on new technologies. Second there was a supply-side failure: if leftto the market, technology services would be underprovided on account of their public good nature. The Project addressed both these market failures by supporting the provision of technology services where they did not exist and involving the private sectorin design, delivery and cost sharing of technology services. The Project also had a positive fiscal impact, since it enabled the public institutions (TPE, UME, MAM) to generate a much higher level of earnings from their own services and thereby reduced burden on the public resources. First, the net impact of the Project on the private provision of technology services was positive, and second, the self-financing ratio of the participating institutions increased as a result of the Project for example, in 2005, MAM covered over 55 percent of its expenses from its service revenues compared to about 10-15 percent before the Project. The Project enabled the Government to take advantage of the lessons learned from previous and ongoing Bank-financed projects, theOED evaluation of industrial technology projects, as well as the current literature on industrial technology development. The Project was designed on the successes of TDP-I and took into account an important lesson learnt from that Project. In particular,it recognized the need for focusing on institutional strengthening from the start through training, study tours, technical assistance, and greater private sector participation in setting priorities for participating institutions. Also, given the nature of technology development efforts and the risk associated with some of these efforts, the flexibility designed into the Project was adequate to allow for certain initiatives to be piloted if the environment was right (technoparks, venture capital), rather than committing large funds up front. The Project envisioned several benefits to Turkey, including increased economic and financial returns for entrepreneurs, higher productivity, new product and processes, and as a result, increased domestic and export sales and profitability. In addition, the Project had a substantial catalytic effect on other public sector investments in the provision of technology services. Given its focus on improving the incentives and management practices of the participating public technology support institutions, the Project influenced the overall government technology policy and expenditure in the direction of commercialization, accountability and transparency. 8.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 4)

11

The Project fully met its development objectives, as described below: 1. Assist in the harmonization of Turkish technology infrastructure with European Customs Union standards.

• Through the reforms at the TPE, Turkey’s Industrial Property Rights Regime is now mostly in compliance with WTO and ECU Standards. TPE has modernized its systems and processes and has improved performance in several key areas: patent and trademark applications have increased significantly, average patent processing time has been reduced, and the number of patent and trademark attorneys and IPR personnel in the country has more than doubled. TPE has also made significant progress on its data automation and in reducing its backlog of applications processing and paper files.

• The UME is also aligned with ECU standards and covers 90 percent of Turkish

Industry needs. UME has expanded the number of services offered to industry with a reduction in costs and time for these services, and also has significantly increased its capacity: a number of calibrations performed have increased almost 75 percent in four years from 406 in 2001 to 707 in 2005. Large proportion of these calibrations (80 percent in 2005) is comparable to international best practices.

2. Assist firms in upgrading their technological capabilities in order to improve the competitiveness of Turkish industry, both in domestic as well as foreign markets.

• The TTGV has become animportant agency for R&D financing in Turkey and has supported about 1,400 SMEs with matching TA grants and 260 R&D projects through matching foreign currency loans with a commercialization rate of 90 percent for all R&D projects. The TTGV has attracted about US$150 million of R&D financing, mostly from the private sector and a majority of its R&D projects are being put into commercial production and synergy between the industry and academic/research community hasincreased. The TTGV has supported two Technoparks which currently house over 100 technology-based companies engaging over 2,000R&D personnel. In addition, it is a partner in two venture capital funds in collaboration with private sector and international investors as well as in one Start-Up Fund.The TTGV has facilitated collaborative links between industrial companies and universities/R&D institutes via the project evaluation and monitoring process. The number of universities and RDIs involved in the evaluation process reached 29 in 2005, and thenumber of researchers/scientists who participated in the evaluation process increased to 545 in 2005. However, most importantly, TTGV has supported the creation of an R&D financing culture in Turkey which did not exist before.

• The MAM has been successful in focusing its energies on market and increasing

contractual research and services and has significantly improved the quality of its R&D, gaining international recognition. MAM has made excellent progress in

12

restructuring and reorganizing its institutions and expanding its industrial projects and services. MAM and all its laboratories have obtained ISO 9000 certification and many of its laboratory analyses are internationally accredited and others are in the process of obtaining such accreditation. MAM has increased its private sector applied research income and its self-sufficiency ratio has risen from 14 percent in 1999 to over 55 percent in 2005. It has significantly expanded its client base from 87 in 1999 to 348 in 2005 and established a Technopark toincrease synergies with the industry and its scientists.

The overall impact of the Project has been significant as evidenced by the:

• Modernized Infrastructure (UME, TPE). UME is now capable of covering almost 90 percent of Turkish industry’s metrologyneeds as compared to about 30 percent before the Project and TPE has improved its service standards by almost two times since start of the Project;

• Restructured R&D Institutions (MAM has achieved a self-sufficiency ratio of more than 55 percent as compared to 10-15 percent before the Project);

• Increased Productivity (firms supported by the Project are more productive by 11 percent on average);

• Increased Output of firms supported by the Project (output has increased by 6-10 percent, and 80 percent of firms have developed new products or processes);

• Increased R&D Expenditures by firms(average growth rate 34 percent, 40 percent for SMEs that were supported by the Project);

• Increased Exports (annual growth rate 20 percent in manufacturing industries); • Improved Industry and Academic/Research Community Synergies; and • Enhanced Skills Base in firms.

Also, through the reforms supported under ITP, there has been an increase in the Turkish firms developing, adapting, and commercializing new technologies, and the impact is visible in the improved productivity, output, R&D expenditures and exports of Turkish firms. The R&D expenditures in Turkey have almost doubled over the past decade to 0.66 percent of GDP in 2002, with private sector share now making up over 40 percentof these expenditures, compared to from 24 percent in 1999. In addition, as evidenced by the Growth Competitiveness Index (GCI) and the Business Competitiveness Index (BCI), Turkey has improved its ranking on several innovation and technology development indexes (see Table 1 below). Table 1: Selected Innovation and Technology Development Indicators. Turkey Ranking

Rank in 2003 Rank in 2004 Innovation 68 56 Technology Transfer 39 30 Government prioritization of ICT 90 81 Government success in ICT promotion 87 79 University/industry research collaboration 67 62 Laws relating to ICT 86 77 Technological readiness/sophistication 59 54 quality of the national business 55 53 company operations and strategy 56 43

13

company operations and strategy 56 43 Source: World Economic Forum - The Global Competitiveness Report ( 2005-2006)

It should be noted that the above positive improvements/results reflect intermediate outcomes because, as mentioned earlier, the full impact and complete outcomes of technology development projects are realized only in three to fiveyears after the project completion.

8.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return)

Not applicable. 8.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency)

Rating: Highly Satisfactory

The most tangible success of the Project was upgrading of the technology infrastructure institutions (MAM and UME) and the strengthening of TPE. This has led, in turn, to lower costs (and faster service) for Turkish firms (of calibration services), closer links between firms and public R&D institutions and greater attention to IPR issues. An equally important long-term achievement, whose results will become fully evident only in medium-term, was the stimulation of an R&D culture in Turkish industry. While international competition was probably the more important stimulant here, the Project helped to raise the awareness of the importance of R&D by the launching of TTGV programsand the scheme for matching loans for technology development. The Project taught firms how to manage R&D projects, disciplined their R&D teams, and put them in close contact with academics/scientistsin the relevant disciplines. They acquired great prestige and the winning of TTGV projects became a symbol of technological competence. This element of the Project has thus stimulated R&D among other firms and has led to stronger linkages with academia and research community. It is not easy to quantify how widespread the effect is (with currently available data), but it is likely to be one of the factors behind the observed growth in industrial R&D in Turkey. In terms of restructuring research institutions the Project showed that public institutions can be reformed and play a valuable role in technology development, if the institutional culture is changed. Also, the institutions that were assisted in this Project served as excellent role models, and spillovers to other institutions, not participating in the program, has brought the industry-oriented culture also with them. The Project succeeded in all objectives. As a result of the Projects (ITP as well as TDP-I), the main elements of Turkish National Innovation System (NIS) are in place with key players including TUBITAK, TTGV, TSE, TPE, UME, TURKAK, KOSGEB,

14

universities, R&D institutions, incubators, technoparks and other organizations. The TUBITAK is now playing an increasingly important role in promoting innovation in the Turkish economy with increased level of resources and responsibilities and introducing a number of new programs. The TPE has been modernized to bring Turkey’s IPR regime in compliance with WTO and ECU requirements and improve its services to industrial clients. The TTGV has now become an important institution providing a range of innovation financing services to Turkish enterprises. The MAM has been restructured and now provides a wide range of R&D services to the public and private sector earning more than half of its expenses, and UME is now capable of providing a wide array of metrology services as well as earning money from industrial clients. It is fully aligned with EU standards and can cover almost 90 percent of Turkey’s metrologyneeds at lower costs and faster response time. The TSE (supported under TDP-I)has modern standards and testing facilities to serve the Turkish industry. Turkish firms are now more aware of the need for innovation and are starting to develop, adapt, and use new technologies for improved productivity and exports. In addition, the venture capital industry is beginning to take root and synergy between universities, R&D institutions and industry has begun to improve with the establishment of a number of technoparks and incubators.

The Project is a good example of private-public partnership. It was designed with strong private sector participation, not only in the demand for these services but also in the design and cost sharing of their demand. This private sector focus promoted sustainability of technology support services and resources in Turkey. Individual institutions that were restructured and assisted become more industry-oriented and revenue-generating and self-sustainable to a large extent in the long run. The Project is part of other valuable technology projects that the Bank has undertaken. The Turkish projects rank with the Indian project in their catalytic impact on industrial and technological development. They were undertaken at the right time andwon the support of all stakeholders. They provided valuable learning experience for the Bank and for the host country. They should be repeated in other countries, with appropriate adaptations to local needs and institutions. The lessons of these projects are of value to the Bank in its future work on technology development. They are also of great value to developing countries. Most countries, confronted with an intensely competitive world economy, are seeking to raise their technological and innovative capabilities. The Turkish projects show how a country with a large industrial sector and a weak innovation system can catalyze a technology culture and upgrade its technology infrastructure effectively. While the available results on impact are positive, the real economic effects of technology development projects like these will appear only in the long term (though they may be difficult to distinguish from all other influences on technological activity). This does not detract from their significance to Turkish development; on the contrary, it is the interaction of technology development with the other primary engines of sustained growth and competitiveness that is important.

15

8.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above)

(a) Poverty Impacts, Gender Aspects, and Social Development

Not applicable.

(b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development)

The Project had a substantial impact on institutional development. The technology infrastructure institutions were significantly upgraded and strengthened. This, in turn, has led to lower costs for Turkish firms, closer links between firms and public R&D institutions and greater attention to IPR issues. In addition, the Project helped raise the awareness of the importance of R&D in the country, stimulated R&D among other firms thus having the spill-over effect and led to stronger linkages with academia.

(c) Other Unintended Outcomes and Impacts (positive or negative, if any)

Not applicable 8.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes)

The development impact of the Project was measured by analysis of survey data collected with the help of the State Institute of Statistics. A large number of interviews were conducted by independent experts to get "qualitative" information about its impact. In addition to extensive interviews conducted as a part of Interim Report in 2004 and the assessment of TDP-I and ITP projects in 2005, a set of structured interviews with TTGV clients (211 firms) were conducted by an independent company in January 2006 to get detailed information on client firms. Moreover, 102 firms whose applications for technology development funding were rejected by TTGV during the 1999-2005 period were also interviewed to check if these projects were conducted without the TTGV support. Finally, 11 calibration laboratories were interviewed to get information about the UME impact on secondary level laboratories. Overall assessment of ITP services by client firms shows that MAM and UME services were received favorably. The TPE services were graded lowerthan the "normal" level initially, but the degree of satisfaction with TPE services increased in 2005 as TPE’s modernization was completed. Technological aspects of MAM and UME services were valued favorably, but their costs were considered high. The evidence suggests that ITP had a positive impact on Turkish industry in terms of: • Technological effort and R&D spending,

16

• Productivity and profitability,

• Competitiveness (export intensity), and

• Employment generation The program achieved to a large extent the intended outcomes in the Turkish industry.Technological activities are expected to have long term impact on firms’ performance. Since the program was just completed, only its short term impact was evaluated, and, in this sense, its real impact will extend over the long term only. Moreover, ITP clients have closer links with their suppliers and help them to raise their quality levels. Thus, ITP’s impact is likely to be diffused in the economy through user-supplier linkages and other forms of spillovers as well.

9. Assessment of Risk to Development Outcome Rating: Moderate

The technology upgrading component constituted the largest part of the ITP. This component relied mainly on strengthening TTGV core business: R&D funding. Since TUBITAK-TIDEB started to provide R&D grants in 1995, several interviewees suggested that it would be better for TTGV to extend financing into upstream (start-up support, "angel capital" etc.) and downstream activities (commercialization of R&D) that would complement and strengthen its core business of R&D financing. The issue of self-sufficiency targets set for MAM: it was mentioned in the Project Implementation Plan (PIP) that the "main measure of MAM’s success will be its performance in increasing its industrial contractual income and its move to self sufficiency", and the self-sufficiency ratio was expected to increase from 15 percent in 1999 to 70 percent in 2004. Although the MAM managers who participated in project planning maintained that the 70 percent target was realistic, if not too low, some interviewees stated that it was ambitious and even detrimental in realizing MAM’s mission that includes basic and strategic research (as mentioned earlier, the timing of MAM’s self-sufficiency ratio target was revised later). Despite this, the assessment of objectives suggests that the Project was introduced at the right time, and in the right direction (of providing the missing components and strengthening the key components of the NIS).

10. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

17

into account the Project’s consistency with the government’s development priorities and the Bank’s CAS. The Bank team had relevant skill mix (supplemented by external experts) and brought appropriate expertise into project design, as well as consulted extensively a wide spectrum of stakeholders, including the clients of TDP-I. During preparation and appraisal, the Bank took into account the adequacy of the project design and assured consistency with the Bank’s safeguard policies. In addition, the Bank had agood working relationship with the Borrower and Project Agencies during preparation and appraisal. The Bank team developed a comprehensive independent monitoring and evaluation mechanism in which three sets of indicators were used to monitor project progress and achievements: input indicators, output indicators, and outcome indicators. As a part of the monitoring process, the implementing institutions were required to regularly collect and provide data on input and output indicators. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Highly Satisfactory The Bank’s performance during the implementation of the Project was highly satisfactory. Bank supervision of the Project was thorough and professional. Also, the skill-mix and continuity of the team were satisfactory. The Bank’s team had intensive missions and discussions with each Project Agency and monitored progress through annual business plans, quarterly project management reports and site visits. In addition,constant feedback through the comprehensive monitoring and evaluation system and performance indicators included in the Project allowed each institution to assess its achievements and impact on a periodic basis. The Bank’s Project team kept close contact with the Project Agencies through frequent electronic mail communications, video conferences, as well as telephone conversations which were very effective in resolving urgent issues. The Bank’s client relationship was cordial and productive. The Bank took a firm and dynamic role in the supervision of the project implementation. Aide Memoires were regularly prepared and transmitted, which alerted the Government and Project Agencies to problems with project execution and suggested remedies in a timely manner, in conformity with the Bank procedures and client needs. The Implementation Supervision Reports (ISRs) realistically rated the performance of the Project both in terms of achievement of development objectives and implementation. Whenever delays in implementation occurred, the Bank team was able to propose concrete steps in consultation with Project Agencies and a timetable for putting the program on track. During the Project implementation, significant changes in the management of Project Agencies caused delays (and sometimes confusion) in Project activities and adversely affected implementation. However, continuity in the Bank team and its proactive and timely interventions helped alleviate this negative impact to a large extent. (c) Justification of Rating for Overall Bank Performance

18

Rating: Satisfactory The overall Bank performance is rated as satisfactory, alternating between intensive supervision and flexibility depending on the needs of the Project under changing realities on the ground. 10.2 Borrower (a) Government Performance Rating: Satisfactory The Borrower’s performance (represented by Treasury) in the preparation of the Project was satisfactory. During the preparation stage, the Borrower displayed high level of commitment to the objectives of the Project and all major aspects such as technical, financial, economic, institutional, environmental and sociological factors, including stakeholder commitment. The government officials and staff of the Project Agencies both worked closely with the Bank’s team on a continual basis. Counterpart funding was an issue at UME and MAM and caused some Project implementation delays. While the Project design included flexibility in resources depending on project needs, the actual processof reallocation of these resources from the "Unallocated" category to UME was delayed by almost a yearbecause of government procedures. These contributed todelays in procurement processes and a slower commitment and disbursements. The Government consistently maintained its strong commitment throughout the implementation and was able to overcome many issues encountered during project implementation. Owing to the 2001 macroeconomic crisis, the Government faced counterpart funding problems which were addressed subsequently although with delays which had some adverse impact on certain project activities. Further imposition of nation wide austerity measures including restrictions on external travel and hiring of new staff, had adverse impact on the project implementation, especially in the case of UME and MAM. (b) Implementing Agency or Agencies Performance Rating: Satisfactory Implementing Agency

Performance

MAM did a good job in restructuring itself from mostly a basic research institution to a customer oriented applied research organization. In the early years of the Project, one of the main issues was its self-sufficiency ratio, used as the main criterion for market-orientation. The self-sufficiency ratio did not reach the levels envisaged in the original project plan as the economic crisis in 2001 reduced demand for MAM services. After consultation with the Bank, the realization of the target self-sufficiency ratio (70 percent) was

19

postponed for two years. Nevertheless, such target forced MAM to reorganize its research institutes and focus on market demand as well as its internal costs structure- two institutes (textiles and genetic research institutes) were separated from MAM in 2000 because their income from industrial research and services was too low-- genetic research institute was not doing applied research and textile institute did not have adequate demand for its services.

The above achievements notwithstanding, the reorientation of MAM towards the market may have gone too far, detracting from its equally important function of conducting basic research and stimulating world-class publications and patents. There were two changes in MAM’s top management during the Project. The previous MAM management while restructuring focused too much on overall income probably at the cost of quality of its income. It had placed higher emphasis on income from public sector sources (which was easier in comparison to private sector income). Further, a large part of this income was coming from testing services rather that high level applied research, patents and R&D commercialization. It is important in any project dealing with public R&D to preserve and enhance the national science base and the ability of institutions to provide the public goods of basic research. Not only is this important for the applied research base, it is also vital to attract and retain first-class scientists.

MAM also faced difficulties in installing its MIS and cost accounting systems and implementing recommendations of an external study focusing on its central organization, business services and cost structure. More importantly, it has not been able to implement many of the recommendations of its 2005 International Review Study which was conducted to enhance the quality of its research and improve overall effectiveness and image in the market. The recommendations of this Study included a strategy to increase the quantity and quality of its research outputs, their protection (patenting) and commercialization, increase in private sector contracts, joint projects with international R&D organizations, etc.

National

UME leadership (at the time of appraisal) was dynamic and knowledgeable about metrology aspects but lacked experience in handling large construction contracts (and despite Bank team advice did not want to hire its in-house civil works experts. UME experienced difficulties with the civil works contractor and delays in construction of laboratories- this had an adverse impact on other aspects of Project. On the operational side, UME did a good job in putting together state of the art laboratory facilities and extending international cooperation. It expanded calibration (and training) services to industry but was not able to expand that much the primary calibration due to lack of private

20

sector secondary labs in the country. However, due to changes in TUBITAK and its own management in 2004, UME experienced many problems in project implementation and procurement was delayed significantly. Further, UME experienced staff shortages due to restrictions on the hiring of new staff by the Prime Minister’s Office. Despite the Bank’s best efforts with TUBITAK and Treasury, this issue could only be partly addressed by TUBITAK, which resulted in under-utilization of its modern laboratory infrastructure and lost opportunity.

Turkish Patent Institute

TPE implemented the project under difficult circumstances with changing management and decreasing autonomy. There were some delays in TPE activities mainly due to frequent changes in top management and their limited focus on the Project and Bank procedures, and their reluctance to delegate to PIU staff enough responsibility. Initially, the understanding of the Project among the TPE staff was weak which hampered the preparation of Annual Operating Plans and their implementation especially the training plans. A new TPE President was appointed in mid-1999 (and again in 2003) who took over the project management responsibility himself causing disruptions. A new PIU was assigned and the PIU structure was reorganized in June 2000 and again in 2003. Initially, construction activities were also delayed due to managerial problems-- the award of the construction contract was delayed by almost one year due to top management inaction. The amendment of the TPE Law was significantly delayed and it is still not fully in line with international standards and EU requirements. Because of restrictions in the TPE Law, TPE experienced a severe shortage of permanent staff and had to manage its activities by hiring temporary staff.

The current TPE management took timely steps to address problems and improve project implementation. As a result, things improved significantly and project implementation moved fairly smoothly.

TTGV established in 1991 under the Bank’s first Technology Development Project has done a good job in promoting R&D culture among Turkish firms and enhancing synergies between enterprises and academic/research community. It has also introduced many innovative approaches in its services. However, despite the Bank’s two projects and considerable institution building efforts, its management and organizational culture remained conservative in relation to its mandate and role. For example, it was slow to respond to new approaches and changing market needs. It preferred a lower risk approach in its financing practices- for example, contrary to its agreement with the

21

Bank; it asked its TDF clients to provide guarantees when they took R&D loans. It also expected a lower rate of failure from R&D projects and much higher rates of repayments (over 90-95 percent). This in turn resulted in TTGV avoiding support for higher risk but high reward R&D projects, thus lost opportunity for the country. Further it was very slow to promote seed capital fund and innovation center initiatives due to its rigid attitudes. More importantly, after the unsuccessful efforts on the Bank’s Knowledge Economy Project (which did not materialize) TTGV has reduced its TDF financing and is taking a more conservative approach partly as a result of the current Budget Law which restricts public support to NGOs. However, the Bank team feels that TTGV which has received over US$100 million in public funding (mostly through Bank projects), TTGV has a responsibility and an obligation to be more proactive (and innovative) and play a larger role in the technology development arena by leveraging its skills and financial strength by liaising with private sector initiatives and institutions.

(c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory The Agency PIUs were always responsive to suggestions and took remedial actions, the CPIU in TTGV was less active. Required reports were prepared and submitted to the Bank mostly on schedule. This led to successful monitoring of the programs as well as providing assistance to the supervision missions of the Bank in assessing the implementation progress against the set milestones. However, taking into account the shortfalls described above in Section C, the overall Borrower performance is rated Satisfactory.

11. Lessons Learned (both project-specific and of wide general application)

22

2. Flexibility in project design: Given the complex nature of technology development activities, it is critical that flexibility is incorporated into the project design, to enhance its effectiveness and to reduce the bureaucracy involved in implementation and to permit adaptation of the project to changes in the institutional and economic environment. This flexibility in project design significantly contributed to the success of ITP-- through the incorporation of unallocated funds for use in priority activities that demonstrated high performance, annual updates and review of business plans to periodically assess institutional activities and objectives; performance indicators to act as a feedback loop, etc. 3. Comprehensive monitoring and evaluation system:ITP introduced a rigorous and independent M&E system that included indicators chosen in consultation with implementing institutions on the basis of the respective project inputs, conceptual relevance to project objectives, ease of calculation and practical considerations of collecting relevant data. 4. Intense implementation supervision by Bank and Borrower:Continuity in the skills and teams at both the Bank side, Borrower and Project Agenciessides in managing these projects is critical to ensure the transfer of knowledge as well as benefit from worldwide experience. Treasury, despite changes in its management, consistently provided its strong support and guidance to the Project and was proactive in helping to address problems as much as it could, given its limited mandate. The State Planning Organization (SPO) also was very supportive of the Project and took appropriate steps to address the counterpart funding problems. In ITP, significant changes in the management of implementing institutions (such as UME, MAM, TPE, TTGV as well as TUBITAK ) caused delays in project activities and affected implementation. However, continuity in the Bank team (and its proactive approach) helped alleviate this negative impact to a significant extent. The role of a strong local champion is also crucial for following through with the tough initial reforms as well as to ensure integrity of the project design. 5. Sustainability of R&D institutions: In order that research institutions move away from being wholly government-funded and develop self-financing capacity, the government should provide strong incentives for external revenue generating activities and allow the institutions to retain and accumulate their earnings, and determine the use of these earnings with greater flexibility and freedom.But this should be done in a proper contest and by ensuring that R&D institutions maintain their excellence in basic research while expanding applied research. They should not put too much focus on the quantity of revenues but rather should pay more attention to the quality of revenues, which should be coming primarily from the private sector and mainly by providing applied research, patents and commercialization of R&D outputs rather than from public sector contracts and by providing testing services. 6. Government interventions:The market deficiencies that hold back technological effort can be overcome by carefulmarket friendly interventions, especially when timed with a liberalization effort that creates the incentives for undertaking such efforts.

23

12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies

Some Project Agencies raised one key concern regarding Bank’s procedures especially the procurement process. While keeping in mind the Bank’s fiduciary responsibilities, the Bank team was proactive and flexible in responding to client needs and allowedmany changes and modifications as required in procurement and other procedures. For example, in the case of UME, the Bank allowed a lot of flexibility in packaging its equipment purchases and even agreed to finance parts for the equipment which UME wanted to fabricate itself. The TTGV and MAM did not have many difficulties in following Bank procurement. But UME experienced difficulties after the change in management in 2004 when it made changes in its virtual PIU and did not want to use the expertise of staff who had worked earlier on the procurement matters. Similarly, in the case of TPE, it experienced problems with the procurement when it changed it’s PIU team in 2003 and failed to engage an experienced procurement expert. Overall, the Project Agencies should have continuity in key staff especially, procurement staff and at the same time, the Bank should show maximum flexibility in implementing the project, especially procurement. Another point raised by some Project Agencies related to the continuity of the Bank team. However, in the Bank team, the key person (who was Task Manager at appraisal and largely throughout implementation) remained with the Project from preparation/appraisal until Project closing. Even though the Bank team had some small changes (once in procurement staff and local project coordinator), this did not have any adverse impact on the Project. The Bank team also did not agree with the specific changes in one paragraph section 10.2 b suggested by MAM (regarding its International Review Study and private sector income) and TTGV (regarding its conservative culture) as in Bank team’s judgment such suggestions were not commensurate with reality.

(b) Cofinanciers

Not Applicable

(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

Not Applicable

24

Annex 1. Results Framework Analysis Project Development Objectives (from Project Appraisal Document)

The major objectives of the Project, as identified in the Project Appraisal Document (PAD) were to:

• Assist in the harmonization of Turkish technology infrastructure with European Customs Union standards; and

• Assist firms in upgrading their technological capabilities in order to improve the competitiveness of Turkish industry, both in domestic as well as foreign markets.

Revised Project Development Objectives (as approved by original approving authority)

The Project Development Objectives were not revised. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from approval

documents)

Formally Revised Target

Values

Actual Value Achieved at Completion or

Target Years Indicator 1 : TPE - Reduction in processing time of IPR applications. Value (quantitative or Qualitative)

19.8 months 10 months

Date achieved 12/01/1999 12/31/2005 Comments (incl. % achievement)

Indicator 2 : TPE - Increase in foreign and domestic applicants for IPR protection (patents + trademarks + industrial designs)

Value (quantitative or Qualitative)

28,000 32,000


Indicator 3 : UME - Increased income from Metrology Services Value (quantitative or Qualitative)

US$0.15 million

US$2 million


25

Indicator 4 : UME - Increase in Coverage of Metrology needs to Turkish industry Value (quantitative or Qualitative)

30% 85%


Indicator 5 : UME - Increase in number of types of calibrations performed Value (quantitative or Qualitative)

100 600


Indicator 6 : TTGV - Number of Beneficiary Firms of Technology Financing (R&D projects supported by TTGV)


103 300


Indicator 7 : TTGV - Share of SMEs/Startups in TTGV Portfolio Value (quantitative or Qualitative)

70% 80%


Indicator 8 : TTGV - Repayment Ratio of Technology Financing Value (quantitative or Qualitative)

90% 90%


Indicator 9 : Number of Technoparks established (TTGV + MAM) Value (quantitative or Qualitative)

0 3

Date achieved12/01/1999 12/30/2005

26

Comments (incl. % achievement)

Indicator 10 : TTGV - Number of companies supported through VCFs Value (quantitative or Qualitative)

0 10


Indicator 11 : TTGV - Share of R&D projects supported that are commercialized Value (quantitative or Qualitative)

58% 80%


Indicator 12 : TTGV - Number of projects (firms) benefited from Technical Assistance - matching grants


0 600


Indicator 13 : MAM - Reduced Dependence on Government Budget (Industrial Income Generated)


$4.4 million $13.5 million


Indicator 14 : MAM - Increase in Income from Contractual Research (MAM Self-sufficiency ratio)


14% 55%


Indicator 15 : MAM - Number of Industrial Projects

27


120 200


Indicator 16 : MAM- Number of Publications Value (quantitative or Qualitative)

238 180


(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from approval

documents)

Formally Revised Target

Values

Actual Value Achieved at Completion or Target

Years

Indicator 1 : The project is making excellent progress in meeting project development objectives.


Date achieved Comments (incl. % achievement)

28

Annex 2. Restructuring (if any)

Not Applicable

29

Annex 3. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD M)

Actual/Latest Estimate (USD M)

Percentage of Appraisal

STRENGTHENING OF INDUSTRIAL PROPERTY RIGHTS

15.00 14.63 97.53

STRENGTHENING OF METROLOGY SERVICES

33.00 44.56 135.03

RESTRUCTURING OF R&D INSTITUTIONS

33.00 32.27 97.79

SUPPORTING TECHNOLOGY UPGRADING BY FIRMS

60.00 58.95 98.25

UNALLOCATED 12.45 0.00 .00 Total Baseline Cost 153.45 150.41

Physical Contingencies 0.00 Price Contingencies 0.00

Total Project Costs 153.45 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 1.55 1.55 1.55

Total Financing Required 155.00 151.96

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate (USD M)

Actual/Latest Estimate (USD

M)

Percentage of Appraisal

Borrower 45.86 42.60 92.89 INTERNATIONAL BANK FOR

RECONSTRUCTION AND DEVELOPMENT

156.92 151.86 96.78

(c) Disbursement Profile

31

Annex 4. Outputs by Component Component A: Strengthening of Industrial Property Rights (IPR) Services

The main objective of the IPR component was to strengthen TPE by modernizingthe facilities and infrastructure essential for effective IPR services. As a result of the Project funding, TPE has now world-class facilities; and its new building also houses specialized courts on IPR. Also, TPE staff has benefited from training and upgrading its information infrastructure, and TPE has become a party to all major international IPR agreements. As a result of changes in the intellectual property rights regime, patent and trademark applications in Turkey have increased continuously since 1995. The economic crisis of 2001 negatively affected the growth in patent applications but applications for trademarks and industrial designs continued to increase. However, the number of patent applications recorded a significant jump again in 2004 and 2005. TPE has carried out training activities andinvested in upgrading its information infrastructure. The average patent application processing time has fallen significantly from 19.8 months in 1999 to 4.7 months in 2005. There has been a similar reduction for applications for utility models (from 19.1months in 1999 to 10.7 months in 2005) and trademarks (14.3 months in 1999 and 5.0 months in 2005). Industrial design applications are processed rapidly (1-2 months). The new legal framework for IPRs in Turkey has led to a surge in the number of patentand trademark attorneys. There were only 75 patent attorneys and 100 trademark attorneys in 1996. Their numbers increased to 267 and 428 in 1999, and 720 and 1020 in 2005, respectively. As a result, the ratio of patent applications to patent attorneys remained almost constant in 1996-1999, but declined sharply since 1999, from 12.5 to 4.9 in 2005. The number of appeals against rejections has gradually increased over time, from 76 in 1999 to 606 in 2005 (or, from 0.6 percent of rejected applications to 8.5 percent). This trend is likely to increase the administrative burden for TPE. TPE’s income from IPR services has increased substantially to US$26.5 million in 2005. Its expenditures have declined to 31 percent in 2005. The number of employees increased from 234 in 1999 to 391 in 2005 (67 percent), mainly because of employees on short-term contracts: the share of short-term employees has increased from 36 percent in 1999 to 47 percent in 2005. Component B: Strengthening of Metrology Services

At the end of 1990sUME was capable of meeting about 30 percent of the industry’s needs for metrology. By the end of the Project it had reached the 90 percent level. UME has won international recognition, and hassigned mutual recognition agreements with many national institutes and international organizations so as to satisfy the requirements for international traceability of its metrology standards.

32

UME emphasized R&D in measurement technologies and developing new standards and equipment. This emphasis helped UME to improve its metrology capabilities and provide better services to the industry. UME provided calibration and testing services to industry and assisted in the establishment of new calibration laboratories. It signed a protocol for cooperation with the Turkish Accreditation Agency in 2002, and provided strong support for the development of other components of the MSTQ system in Turkey.

UME has been successful in enhancing the range and quality of calibrations in Turkey. The number (type) of calibrations UME could perform increased almost 75 percent in four years (from 406 in 2001 to 707 in 2005), the actual number of calibrations performed for customers exceeded 4,000 in 2005. A very large proportion of these calibrations (almost 80) are comparable to those provided by leading metrology institutes in the world. As a result of R&D activities conducted by UME, the range of uncertainty of 10 percent of calibrations has been reduced every year. The increase in the quality of calibration services has been complemented by a reduction in the duration of providing these services; also the cost of local calibration is considerably lower than in Europe, to where Turkish firms had to go before. UME has stimulated the growth of the private sector calibration industry (there are now 18 private calibration laboratories). Existing calibration laboratories have a very favorable view on UME’s activities and its role as a custodian of primary standards in Turkey. It deliberately sets high fees for calibration services to encourage private calibration laboratories, and is specialized increasingly in primarycalibration and metrology R&D. It has also raised awareness of the importance of calibration for export competitiveness in Turkish industry. UME provides training services to the Turkish industry on a wide range of technical issues: the types of training provided have increased from 44 in 2001 to 78 in 2005. Also, UME has been very active in international cooperation. UME staff joined almost 50 international committees and the number of international collaborations has increased substantially. Calibration services account for about 90 percent of UME’s industrial income that reached US$3.1 million in 2005 (up from US$1 million in 2000). A self-sufficiency ratio has increased to 24 percent from 15 percent in 2000. UME has invested almost US$53 million in six years (2000-2005) in its new laboratories, building, consulting services, and staff training. A customer survey conducted in 2004 revealed that UME enjoys a high degree of customer satisfaction about the quality of its services. Component C: Restructuring of R&D Institutions

The objective of restructuring and strengthening MAM was largely achieved. MAM’s infrastructure has benefited from investment in new scientific equipment, its IT network was established, and new management methods (strategic management, change management, business development, annual business planning andbudgetary procedures, and cost accounting) were introduced. MAM was re-oriented towards

33

conducting more contract research, also a technopark and technology free zone (TEKSEB) was established, attracting substantial interest from R&D-intensive companies. MAM was successful in expanding its client base. The number of clients rose from 87 in 1999 to 348 in 2005, a 4-fold increase in six years. MAM has also been able to keep the share of repeat clients almost constant throughout the period. The number of industrial services performed (analyses, tests, etc.) has increased steadily in this period (from 14,458 in 2000 to 50,580 in 2005); and economic crisis of 2001 had littleinterruption. The impact of economic crisis, however, was seen in average income from industrial services: earning per industrial service was US$79 in 2000, dropped to US$26 in 2001 and 2002 due to the sharp real devaluation of TL and increased to its pre-crisis level (US$78) in 2005. Also, the number of industrial projects conducted by MAM has increased from 43 in 2001 to 94 in 2003. The number of private projects reached 73 in 2005. MAM and all its laboratories have obtained ISO 9000 certification. Many of its laboratory analyses are internationally accredited and others are in the process of obtaining such accreditation. The self-sufficiency ratio has risen continuously because of the re-orientation of MAM towards the industrial sector, from 14.2 percent in 1999 to about 58 percent in 2005. MAM has significantly reduced its non-research personnel, bringing the share of researchers in total staffing from 51 percent in 1999 to 65 percent in 2005. Component D: Supporting Technology Upgrading by Firms

TTGV was the largest component of the Project in terms of funding. It was established by the participation of public and private sectors. TTGV received additional funding for technology development financing from the Undersecretariat of Foreign Trade (UFT) and also managed the Montreal Protocol Multiparty Fund in Turkey to assist industry in moving towards non-ozone depleting technologies. The expert review system for project applications has been very effective, and has brought academics and scientistsin contact with the industry. In addition to technology development financing (TDF), TTGV supported the establishment of VCFs, and provided technology services and training programs. Most importantly, TTGV made valuable contributions to nurturing a "technology culture" and awareness in Turkey, through activities like active campaigning and technology awards launched in 1999. TTGV received, on average, 71 applications annually for technology development financing during 1993-1998. The number of annual applications increased to 150 during ITP (1999-2005). TTGV supported about one-third of these projects. The share of SMEs and start-ups in TDF clients remained about 80 percent. TTGV provides US Dollar loans for R&D activities. TTGV was successful in its core business of technology development funding. The repayment ratio has been very high for such risky activity, at above 94 percent for ITP projects which could be an indication that TTGV did not fund riskier R&D Projects. TTGV has participated in two venture capital funds (VCFs), Is Risk and Turkven, to facilitate the development of the venture capital industry in

34

Turkey. The two VCFs have been active in seeking out venture investment opportunities and had funded nine projects by the end of 2005. In addition to its TDF function, TTGV has also provided training services and technology support services (to almost 1,400 SMEs). The four technology service centers (TSCs) established by TDP-I havecontinued to assist a large number of (small) companies. Although the main function of TDF is to provide R&D funding, it has also facilitated collaborative links between industrial companies and universities/R&D institutes (RDIs) via the project evaluation and monitoring process. The cumulative number of universities and RDIs involved in the evaluation process reached 29 in 2005, and the number of researchers/scientists who participated in the evaluation process increased to 545 in 2005. Technology financing is TTGV’s core activity. Since TTGV provides US dollar denominated loans, its income and expenditures were not greatly affected by the 2001 economic crisis. Its average annual income has been around US$15 million and its expenditures around US$4 million in 1999-2005 without any major variations. Its personnel have increased steadily and by end 2005 amounted to 48 (24 in 1999).

35

Annex 5. Economic and Financial Analysis (including assumptions in the analysis)

Four outcomes of ITP were assessed: technological effort and R&D, labor productivity, competitiveness, and employment. Technological effort and R&D

The main objective of ITP was to stimulate R&D and technological activities in the Turkish industry, especially in the manufacturing sector. It was assumed that business enterprises under invest in R&D, and total R&D expenditures could be increased by providing subsidized loans and assisting firms in their technological activities. An important criterion, in this respect, is the measurement of the so-called "additionality"; that is, additional investment in R&D as a result of the Project. The support provided by ITP was aimed to generate additional R&D. In order to assess the impact of R&D support program implemented by TTGV, three complementary approaches were used. First, those firms that received R&D support were asked what they would do, had they not received R&D support. Second, those firms whose applications were rejected were asked if they implemented the project without TTGV support, and if they did, if there was any change in the project budget. Third, econometric analyses were performed to check the impact of the program. Some211 firms who completed at least one R&D project supported by TTGV were asked what they would do without the support. 12 percent of large corporations (LCs) and 27 percent of SMEs stated that they would cancel the project (22 percent on average) (Table 2). Almost half of the firms who claimed to conduct R&D project without TTGV support stated that they would reduce the project budget by 40 percent. A few firms stated that they would increase the project budget by 20 percent. Thus, Large Companiesand SMEs would spend 20 percent and 40 percent less, respectively, on R&D without TTGV support (on average 34 percent). These findings suggest that TTGV’s R&D support program has a substantial additionality effect, especially on SMEs.

36

The Survey providedinformation about 110 R&D projects of 102 firms rejected by TTGV. More than 40 percent of these projects have not been conducted because of the lack of TTGV support. A large proportion of projects conducted without the support have been scaled down. Thus, R&D expenditures have been reduced by 50 percent when they did not get TTGV support.

There are various methods that are used to assess the impact of R&D support programs on R&D activities. Matching methods (treatment effect models) have been extensively used in the literature on the evaluation of labor market policies. These methods have also been applied to assess the effects of R&D support programs in recent studies.

Matching methods are based on a comparison of the outcomes that would have been observed for supported ("treated") firms had they not benefited from the program. Let RD1

be the outcome conditional on R&D support ("treatment") and RD0 the outcome that would have been observed if the same firm received no support ("no treatment"). The impact of the program is

∆ = RD1 – RD0

37

Since only RD1 or RD0 is observed for each firm, ∆ is not observable. Although it is not possible to identify the individual effect of the support program, the average effect can be estimated under certain assumptions.The main parameter estimated in empirical studies is the mean "impact of treatment on the treated", and it is defined by

TT = E (∆| D = 1) = E (RD1| D = 1) – E (RD0| D = 1)

where D = 1 denotes the group of support-receiving firms (D = 0 the group of firms that do not participate in the program), and E (.) the expectation operator.

The counterfactual E (RD0| D = 1) needs to be estimated because it is not observed. There are three estimators available: before-after (uses the data on support-receiving firms prior to participation in the program), cross-section (uses the data on non-participants), and difference-in-differences (DID, uses both types of data). The cross-section (CS) and DID estimators are defined as follows:

CS = E (RD1it | Dit = 1) – E (RD1

it| Dit = 0)

DID = E (RD1it – RD1

it-1| Dit = 1, Dit-1 = 0) – E (RD0it – RD0

it-1| Dit = 0, Dit-1 = 0)

The CS estimator compares the difference in average values of supported and non-supported firms whereas the DID estimator compares the average change in the outcome variable in supported and non-supported firms conditional on not having received the support at time t-1 (Dit−1 = 0). Due to its panel data feature, the DID estimator is superior to other (before-after and cross-section) estimators, because it allows for firm-specific and time-specific unobserved effects, that are eliminated by "same-firm" and "same-period" differences, respectively.

The cross-section and DID estimator require the construction of a control group that is "similar" to the support-receiving group. In this report, we use nearest neighbor matching to construct the control group where each firm receiving support was matched with its closest non-participant neighbor. Firms are matched on the propensity score (the probability to receive TTGV support) that is estimated by a probit model (we used the psmatch2 program written by Leuven and Sianesi, 2003). This method ensures that the groups of support receiving and non-participant firms are statistically not different in X, the set of variables used in the estimation of propensity scores.

38

Estimated average effects of TTGV support program on total R&D intensity (including TTGV support) and own R&D intensity are shown in Table above The DID parameter was estimated for two cases: Case 1 includes all observations whereas Case 2 includes only those establishments that performed R&D in both years under consideration (t-1 and t). The propensity scores are used to construct the matched control group. t-tests show that, although the group of support receiving establishments is different from the unmatched group (non-participants) in almost all respects, the groups of support receiving and matched non-participant establishments are statistically not different in the set of variables used for matching. In other words, the treated (supported) and matched control groups have, on average, similar characteristics. The DID estimates for R&D and own R&D are quite substantial and statistically different from zero. For example, TTGV clients increased their R&D intensity by 5.22 percentage points and own R&D intensity by 3.79 percentage points from t-1 to t, i.e., after receiving the support (the difference between R&D intensity and own R&D intensity is the value of the supported projects), whereas there is almost no change in the R&D intensity of non-participants that have similar characteristics in the same time period. We observe a similar change in Case 2 that includes only R&D performers. TTGV clients experienced 3.67 percentage points increase in R&D intensity and 2.35 percentage points increase in own R&D intensity, whereas the matched control group raised R&D intensity only by 0.06 percentage points. These results indicate that there could be an "acceleration effect" because an average firm increases its own R&D spending if it receives any R&D support.

We have also estimated R&D investment models by using panel data on R&D intensity. The findings of various econometric estimates confirm the results of average effects estimates. TTGV support is found to have a (statistically and economically) significant impact on R&D and technological effort. In addition to the econometric evidence, whichis inevitably subject to caveats in discerning causal relations, it is important to consider qualitative evidence obtained from interviews. This also indicates clearly that the projects had beneficial effects. Firms benefiting from ITP services often changed the way they conducted R&D. The main benefits claimed were: the time discipline and tight R&D

39

process management introduced by TTGV, the advice provided by the supervisors (academic and industrial) and the prestige attached to winning TTGV projects. The screening and evaluation of R&D projects by TTGV was felt to be very useful, and one firm claimed to have "saved a lot of money" when TTGV rejected one of its proposals. Most firms claimed that the access gained to leading academic researchers in the relevant fields was a major benefit – and would not have been possible without the project even if they were to offer money to the experts. Firms interacted more closely with universities and academic research staff. Most of the institutions participating in the ITP projects were held in high regard by the enterprises using their services and appear to have made a positive contribution to technological upgrading and competitiveness in Turkey. The contribution, however, is still relatively modest – perhaps it would be unreasonable to expect more at this early stage, given the long tradition of minimal technological activity in the country. More generally, a "technology culture" is now developing in Turkish industry, with enterprises realizing the value of technological effort and the contribution that technology institutions and universities can make. Exposure to international competition is clearly an important stimulus, but the projects have helped build an awareness of R&D methodologies and linkages. Most firms did not know how to plan an R&D budget, develop an R&D plan or manage R&D projects. The fact that TTGV required very tight schedules and demands "invoices for everything" was considered to be very useful. Some suggestions to improve the technology development support program received from survey were to advertise more and inform more firms of the available facilities, get more resources and staff, and set up sector-specific departments with the relevant expertise. In addition to TTGV, other ITP components had an impact on R&D and technological effort as well. For example, TPE clients were all highly complimentary about its services and appreciated its improved facilities and staff skills. It was a 'transparent' organization, with well-trained and helpful staff. Its provision of training to IPR staff was considered very valuable and its awareness raising campaign effective. However, there was insufficient experience of foreign patent rules, and problems arose when there were differences with foreign patent authorities on technical details of particular patents. The expansion of UME has benefited Turkish industry. Earlier, most metrology work had to be carried out in Europe , which was slow and expensive. With the development of UME capabilities in most relevant areas and the promotion of private calibration laboratories, costs have fallen sharply. MAM clients generally speak highly of its capabilities and services. Its equipment is modern and the staff well-qualified. The cost of its services is regarded as modest, decided by the number of man-days, cost of materials and an overhead (one large firm suggested that it could easily raise its charges). The staff is responsive to private sector needs and pressures. Clients intend to continue their collaboration and to extend the scope of their

40

contracts. However, there is a risk that UMEwill do too much industrial work and not enough basic research and primary calibration. However basic research provides the long-term foundation for its contribution to the economy, and justifies its existence as a publicinstitute.

Productivity

ITP clients are expected to be more productive than non-participant firms as a result of technological effort supported by ITP services. In order to test if there is a difference between clients and non-participants, average effects of ITP participation were estimated for low-technology manufacturing, medium and high-technology manufacturing, services

41

and the software sector because industry-specific factors can also affect productivity. Table below summarizes main findings on productivity. There is a significant difference between ITP clients and non-clients in terms of labor productivity (measured as output per employee), especially in manufacturing industries. Clients for all ITP components are more productive than non-clients in manufacturing. This is also valid in most of the cases in services and software sector where there are a "sufficient" number of clients (more than 10 clients). However, the difference between clients and non-clients could be caused by other exogenous factors that are not determined by ITP program participation. For example,as seen in Table above, client firms are on average larger than non-clients. Thus, if large firms are more productive than small firms, than we could observe productivity differential in favor of client firms. Therefore, we established "control" groups for each ITP component by using propensity score matching as explained in the preceding sub-section, and tested if the productivity difference between clients and non-clients is statistically significant. (As may be expected, the average productivity of control groups is higher than the average of all non-clients.) Our findings indicate that ITP clients are more productive than those non-clients in the control group, and the differences are statistically significant for MAM and UME clients in low- and medium/high-tech manufacturing, and TTGV clients in low-tech manufacturing. The differences are not statistically significant for any ITP component in services and software sector. Since the number of client firms is rather small in services and software, the standard errors of estimates becomes quite large, and the null hypothesis of no difference cannot be rejected. The findings suggest that the average effects of ITP participation on labor productivity is, on average, positive in manufacturing industries. The evidence for services and the software sector is not conclusive. Competitiveness

The average effects of ITP participation on competitiveness were estimated similarly. "Competitiveness" is defined by "export intensity", i.e., the share of exports in total sales, because it is assumed that competition in foreign firms is tougher, and those firms who are able to sell in foreign markets are likely to be more competitive. A simple comparison between average export intensity reveals that ITP clients are,on average, more competitive than non-clients in control groups. In 12 out of 16 comparisons (4 ITP components × 4 sectors), clients have higher average export intensity than non-clients in control groups. The difference is statistically significant in six cases in favor of ITP clients. MAM and UME services seem to be more beneficial for competitiveness and services is the sector that takes the best advantage of ITP services in raising competitiveness.

42

Employment

The HTP Survey provided accurate information on employment generation by TTGV clients. There are 195 firms in the survey that were established before 2000. Total numberof employees in these firms, classified by sectors, and employment growth rates are shown in Table above. Chemicals industry is by far the largest sector in terms of employment among all R&D support receiving sectors. There were about 27 thousands peopleemployed by TTGV clients firms in the chemicals industry in 2005. Total number of employees of clients reached 75 thousands in 2005. TTGV clients who received R&D support before 2005 achieved a substantial increase in

43

employment from 2000 to 2005. The 5-year employment growth rate of client firms is 26.3 percent. Firms in automotive, machinery and software sectors achieved remarkable employment growth rates (66.0, 53.7 and 43.3 percent, respectively). When only those firms that received support during the ITP project (1999-2005) are taken into account (157 firms), the growth rate becomes even higher (46.4 percent). Firms were asked why they applied for TTGV support. About 80 percent of firms indicated that they applied TTGV support for financial reasons(there is not any significant difference between large and small firms in this regard). Employment growth rates for those firms that applied for R&D support for financial reasons were also calculated to test if financial constraints affect growth rates. It seems that financial constraint for R&D does not make any difference for employment growth. Finally, six very large firms (those who employed more than 1000 people either in 2000 or 2005) are excluded from the sample. The 5-year average employment growth rate declined to 32.1 percent because most of these very large firms increased their employment from 2000 to 2005.

44

Average employment growth rate of TTGV clients has to be compared with other groups of firms to get a better view of the impact of R&D support program. For this purpose, we selected two groups of firms: first, we calculated employment growth rates for those firms whose applications were rejected in the same period (67 firms). These firms are expected to be technologically closer to TTGV clients because they all conduct R&D. The second group includes all those firms surveyed by the SIS to calculate the Manufacturing Industry Production Workers Indices (more than 3,000 firms). Table 6 provides the data for annualized growth rates of employment for these groups of firms. TTGV clients are divided into two groups depending on when they received the support. Manufacturing industry experienced a decline in total employment (production workers) by almost one percent annually since 2000. Employmentin the private sector remained almost constant whereas the public sector had a significant loss. Average annual employment growth in TTGV clients who received R&D support before 2000 is almost equal to the industry average, whereas those who received support had a very high growth rate (9.3 percent). Those firms whose R&D projects were not financed by TTGV achieved quite a significant growth in employment. However, it is apparent that those rejected firms who anyway continued with their R&D project had almost the same growth rate as those of TTGV clients, but those that cancelled the R&D project rejected by TTGV had a much lower average annual employment growth. These findings indicate that technological effort is what matters for employment generation. Its source of funding may have a secondary impact. The data used here doesnot provide additional information about employment generation by skill categories. The qualitative evidence obtained through the Industrial Technology Services Survey indicates that R&D support receiving firms experienced an increase in the number of skilled workers and researchers. Moreover, a recent econometric study found that TTGV clients, controlling for the endogeneity of program participation, increase the demand for researchers.

45

Performance dynamics

The Industrial Technology Services Survey conducted in 2005 includes the data on various performance measures for the last three years, i.e., 2002-2004. These measures are used to assess changes in the performance of client firms. Since changes in performance are compared, only those firms that provided the data for all three years were selected for the analysis. Firms in services and software sectors are grouped together because of the insufficient number of observations for some categories. Figures 5a, 5b and 5c provide the data on changes in a number of performance measures for low-technology manufacturing, high-technology manufacturing and services (including software companies), respectively. The data are normalized for 2002 (2002 = 1) to allow for comparisons over time. All clients firms except TPE clients achieved faster output growth rate than the rest of other firms in low-technology manufacturing. Note that "TPE client" is defined rather broadly; it includes any firm that benefited any services provided by TPE. Therefore, almost half of the firms in manufacturing are "TPE clients". (The proportion is much lower in the case of services, about 20 percent.) The growth of TTGV clients is particularly striking. Clients’ employment performance is either better than or at par with non-clients. TPE clients perform slightly worse than others. Thus, there is no significant difference between clients and others in terms of labor productivity growth rate. TTGV clients constitute an exception. Due to their superior output growth performance, their labor productivity increased at a very high rate (almost 50 percent in real terms from 2002 to 2004).TTGV clients in low-technology manufacturing are also successful in raising wages.The highest increase in wages is observed among TTGV clients who were able to increase labor productivity at a very high rate. MAM, UME and TPE clients raised their wages slightly more than non-clients. Low-technology firms increased exports to a very large extent in 2004. UME clients performed even better than the average whereas MAM, TTGV and TPE clients’ export performance is somewhat lower. Innovativeness is, of course, the main determinant of competitiveness in the long-term. The data indicate that client firms performed much better than others in the 2002-2004 period in terms of innovativeness. The proportion of innovative firms (in terms of both product and process innovations) is higher in client firms. As may be expected, TTGV clients who received support for their R&D projects are, on average, more innovative than other firms. Moreover, it seems that SMEs benefited more than large corporations (LCs) from the ITP with regard to innovativeness. The innovativeness performance may indicate that clients’ performance is likely to be much better than non-clients in the future. There are two major differences between low-technology and high-technology manufacturing in terms of the relative change in the performance of client firms. First, the client firms perform relatively better in low-technology manufacturing. Second, TPE clients perform relatively better in high-technology manufacturing than in low-technology manufacturing. This finding may indicate that IPRs could be more important in high-technology manufacturing where technological dynamism is higher. Thus, in terms of

46

output growth, MAM and TPE clients perform better whereas UME and especially TTGV clients lag behind. In terms of employment generation, MAM clients are also in a better position. All categories of clients with the exception of TTGV clients achieved faster labor productivity growth rates than non-client firms in high-technology manufacturing. As a result of better productivity performance, all but MAM clients increased wages more thannon-client firms. These finding may indicate that clients increase the quality of labor they employ which may generate further benefits in the long-term. Export performance of MAM and TTGV clients are much better than non-clients, and UME and TPE clients perform almost at the same level that non-clients do. Although firms in high-technology manufacturing are quite innovative (the proportion of innovative firms is more than 50 percent), the ITP clients perform even better. In services, client firms (except TTGV clients, most of them software companies) achieved faster output and employment growth than non-client did. However, their labor productivity growth rates remained quite close to the average growth rate. As in low-and high-technology manufacturing,TTGV clients in services had the highest growth rate in wages. The innovative performance of small TTGV clients in services is outstanding. All client categories had much higher growth in exports and were more innovative. These findings together imply that client firms’ performance in services is also quite remarkable, and we would expect more benefits in the future. Outcome assessment

The qualitative and quantitative evidence suggests that ITP program had a positive impact on Turkish industry in terms of: • Technological effort and R&D spending,

• Productivity and profitability,

• Competitiveness (export intensity), and

• Employment generation

The program has achieved, to a large extent, the intended outcomes in the Turkish industry although its share in total expenditures on technological activities was quite small. Technological activities are expected to have long term impact on firms’ performance. Since the program was just completed, this study can evaluate only its short term impact, and, in this sense, should be considered as a partial assessment of its impact that will extend over the long term. Moreover, ITP clients have closer links with their suppliers and help them to raise their quality levels. Thus, ITP’s impact is likely to be diffused in the economy through user-supplier linkages and other forms of spillovers as well.

47

Annex 6. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/SpecialtyLending Elmas Arisoy Sr Procurement Spec. EAPCO Furuzan Bilir Operations Officer ECCU6 Vinod K. Goel Consultant ECSPF

Supervision/ICR

Ayse Seda Aroymak Sr Financial Management Specia

ECSPS

Mohini Bhatia Consultant EASRD Ian Cooper Consultant ECSPF Vinod K. Goel Consultant ECSPF Hiran Herat Consultant AFTNL Salih Kemal KalyoncuProcurement Spec. ECSPS Ekaterina N. KoryukinProjects Officer COCPO Zeynep Lalik Mete E T Consultant ECSPS Sanjaya Lall Consultant AFTP1 Ahmet Gurhan OzdoraSr Operations Off. ECSIE Ibrahim Sirer Sr Procurement Spec. ECSPS

(b) Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD M)1 06/28/1999 Satisfactory Satisfactory 0.00 2 09/30/1999 Satisfactory Satisfactory 0.00 3 12/29/1999 Satisfactory Satisfactory 5.80 4 04/10/2000 Satisfactory Satisfactory 6.14 5 07/17/2000 Satisfactory Satisfactory 7.67 6 12/08/2000 Satisfactory Satisfactory 12.66 7 04/12/2001 Satisfactory Satisfactory 19.57 8 06/18/2001 Satisfactory Satisfactory 23.58 9 09/05/2001 Satisfactory Satisfactory 30.50 10 12/21/2001 Satisfactory Satisfactory 33.70 11 04/09/2002 Satisfactory Satisfactory 40.13 12 06/10/2002 Satisfactory Satisfactory 42.05 13 10/07/2002 Satisfactory Satisfactory 54.77 14 06/17/2003 Satisfactory Satisfactory 76.16 15 09/16/2003 Satisfactory Satisfactory 82.66 16 03/12/2004 Satisfactory Satisfactory 103.61

48

17 03/29/2004 Satisfactory Satisfactory 103.61 18 09/28/2004 Satisfactory Satisfactory 121.29 19 12/10/2004 Highly Satisfactory Satisfactory 123.47 20 04/13/2005 Highly Satisfactory Satisfactory 133.01 21 12/13/2005 Highly Satisfactory Satisfactory 144.48

(c) Staff Time and Cost Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands

LendingFY93 7.06 FY94 11.55 FY95 3.63 FY96 22.31 FY97 38.99 FY98 64.77 FY99 257.22 FY00 2 1.71 FY01 0.00 FY02 0.00 FY03 0.63 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00

Total: 2 407.87

Supervision/ICRFY93 0.00 FY94 0.30 FY95 0.40 FY96 0.00 FY97 0.00 FY98 0.00 FY99 0.00 FY00 59 157.37 FY01 30 141.89 FY02 25 120.54 FY03 26 138.76 FY04 30 127.39 FY05 19 125.70 FY06 87.29 FY07 8.46

49

FY07 8.46 Total: 189 908.10

50

Annex 7. Detailed Ratings of Bank and Borrower Performance

Bank Ratings Borrower Ratings Ensuring Quality at Entry:

Satisfactory Government: Satisfactory

Quality of Supervision:Highly Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Satisfactory

51

Annex 8. Beneficiary Survey Results (if any) A large number of interviews were conducted with the client firms in 2003-2004 to get a feedback on their perceptions about institutional effectiveness. Moreover, three surveys were conducted with 1332, 2376 and 3014 firms in 1999, 2002 and 2005, respectively. The first two surveys covered only manufacturing industries whereas the last one included services and software sectors as well. The surveys included questions regarding firms’ knowledge about ITP services, their use and perceptions of institutional effectiveness.

The impact of ITP depends on the diffusion of the information about these services. If firms are not aware of the existence of these services, they would not get any benefit even if they need them. It seems that four out of five firms in Turkish manufacturing industries do not have any information about the services provided by MAM and UME. Firms are informed the least about the TTGV services: Only 8-10 percent of firms had an idea about the TTGV’s R&D support program in 1999-2005. However, software companies are quite knowledgeable about TTGV’s support, due to the fact that a large number of software companies are located in the so-called "science parks" that have close connections with R&D support programs. Firms are more informed about the TPE services that may have a direct commercial effect (trademarks, etc.). Almost half of firms know something about the TPE services. The proportion of firms who were informed about ITP services increased to by about 20 percent since 1999. Program participants tend to be well informed about other programs. This is especially the case for MAM and UME clients. A MAM (UME) client is very likely to be more knowledgeable about the UME (MAM) services. Moreover, all ITP clients tend to be well-informed about IPR issues (the TPE services).

52

It is interesting to observe that, among all R&D support schemes, TTGV program is the least known possibly because of three reasons: TUBITAK-TIDEB support is more generous, and may attract more attention. KOSGEB, that has an extensive network among small and medium sized establishments in Turkey, has the best known program. The R&D support schemes implemented by the Treasury and the Ministry of Financeare known by more than 10 percent of manufacturing firms. These two programs are likely to be known by firms because they are codified in the tax regulations. Software companies are much more knowledgeable about R&D support programs than firms operating in other sectors because many software companies are located in science parks, and benefit from information externalities.

Although the number of firms knowledgeable about MAM and UME services increased in 2005, the proportion of firms who actually used these services among knowledgeable ones was low. About 60-70 percent of firms who claimed to have information about MAM and

53

UME services did not benefit from these services. Manufacturing firms tend to use MAM and TPE services more intensively than those in services and software sectors. There is no discernible change over time in use of ITP services. When asked why they did not benefit from any ITP service, most of the firms claimed that they do not need these services. Moreover, an insignificant proportion of firms stated that they did not use UME services because they were expensive. The share of those firms that did not use MAM,TPE and UME services because of quality concerns has been extremely low in all time periods.

54

There were, on average, more than40 firms per year who received R&D support from TTGV from 1999 to 2005. TTGV provided about US$31 million loans for R&D activities in 1999-2002 whereas during this period the business sector spent US$1,573 million for R&D. Thus, the share of R&D loans in business expenditures on R&D was about 2 percent. (In the last three years (2003-2005), TTGV provided US$40 million loans.) Although TTGV’s share in total R&D is low, it has a significant contribution to those who receive support. Table below presents the data on the number of R&D projects conducted by TTGV clients that completed at least one TTGV-sponsored R&D project until the end of 2005 (there are 207 firms whose project data are available). Those firms who benefited TTGV support until 2005 received the support for only 14 percent of their project. In other words, 86 percent of R&D project conducted by TTGV clients did not receive any support from TTGV. Since firms tend to apply for TTGV for their large projects, the share of TTGV-supported projects in R&D expenditures reached 46 percent. The share of TTGV-supported project is higher in automotive and components industry and lower in electronics/informatics and machinery industries.

TTGV clients, almost at equal proportions, stated that high loan costs (31.5 percent), sufficient own resources (27.4 percent), and inconvenience of getting TTGV support (26.2

55

percent) were the main reasons for not applying for TTGV for other R&D projects. Overall assessment of ITP services by client firms are shown in Table above. MAM and especially UME services were received favorably but the overall level of satisfaction declined somewhat from 1999 to 2005, and TPE services were graded lower than the "normal" level, but the degree of satisfaction with TPE services increasedslightly in 2005. Technological aspects of MAM and UME services were valued favorably, but their costs were considered to be quite high as well. Interestingly, the software sector had negative view for all aspects of TPE activities. There are more than50 firms who received TTGV support at least for two projects (Table 9). These firms were asked to assess changes in the quality of TTGV’s activities (from the first project to the last one). More than half of firms suggested that paperwork needed to complete project applications got more complicated (difficult/time consuming), and almost the same number of firms stated the response time has become shorter. A great majority of firms were pleased that the quality of evaluation and monitoring improved over time.

56

When asked about alternative sources for MAM and UME services, a significant number of firms acknowledged that MAM and UME services are cheaper than alternative sources. For example, 41.6 percent of manufacturing firms in 1999 stated that MAM services were cheaper than its alternatives, whereas only 17.7 percent of manufacturing firms claimed the opposite. The same ratios for UME services were 41.3 percent (UME cheaper) and 24.5 percent (UME expensive) in 1999. However, it seems that MAM and UME services were getting more expensive relative to their alternatives in the last five years. Thus, the proportion of manufacturing firms that considered MAM (UME) cheaper declined to 32.1 (23.9) percent in 2005, whereas those who stated that MAM (UME) was more expensive increased to 38.8 (32.1) percent in the same year. Service sector firms had a more favorable view for MAM and UME costs in 2005.

58

Annex 9. Stakeholder Workshop Report and Results (if any) Not applicable.

59

Annex 10. Summary of Borrower’s ICR and/or Comments on Draft ICR

60

Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable.

61

Annex 12. List of Supporting Documents

Project Appraisal Documents of may 5, 1999

Loan Agreement of August 2, 1999

ITP ISRs: 1999-2006

Borrower Inputs to the ICR

Turkey CASs

An Assessment of the Industrial Technology Project by E. Taymaz – Reports 2000, 2003, April 2004 and May 2006

Project Implementation Plan

Project Agency Business Plans and Annual Operating Plans

Project Management Reports, Annual Audit Reports

Feasibility Studies – MAM, UME, TTGV

Project Agency Feedback Survey Reports

Special Reports- Reviews, Accreditation and Others

MAM International Review November 2005- R.A. Mashelkar, et al

Mid Term Reports May-July 2001

A Comparative Assessment of India- Industrial Technology Project and Turkey –TDP-I & ITP Projects- May 2005- Taymaz, L all & Hari

Innovation Systems- World Bank Support of Science and Technology Development- April 2004-Vinod K. Goel, et al

62

MAP

TTuzuzGGööllüü

HoyranHoyranGGööllüü

BaysehirBaysehirGGööllüü

AksehirAksehirGGööllüü

Prut

ÇÇoorruuhh

MMuurraatt

KKuurraa

FFiirraatt

SSaakkaarryyaa

DDeevvrreezz

KKiizziill

CCeekkeerreekk

KKiizzii

ll

SSeeyyhh

aann

GGöökkssuu

CCeeyyhhaann

KKeellkkiitt

KKuuzzeeyy AAnnaaddoolluu DDaaggllaarr ii

TToorrooss DDaaggllaarr iiKaramanKaraman

KonyaKonya

AksarayAksaray

NigdeNigde

NevsehirNevsehir KayseriKayseri

MalatyaMalatya

AdiyamanAdiyaman

ElazigElazig

TTunceliunceli

ErzincanErzincan

BingBingööllMusMus

BitlisBitlis

DiyarbakirDiyarbakir BatmanBatman SiirSiirtt

SirSirnaknakMardinMardin

HakkariHakkari

VVanan

AgriAgriIgdirIgdir

ErzurumErzurum

KarsKars

ArArtvintvin

RizeRize

GGüümmüüshaneshane¸̧BayburBayburtt

GiresunGiresun

TTokatokat

SivasSivas

AmasyaAmasya

KastamonouKastamonou

ÇÇankiriankiri

KarabKarabüükk

BoluBoluSakarSakaryaya(Adapazari)(Adapazari)

KocaeliKocaeli(Izmit)(Izmit)

BilecikBilecik

EskisehirEskisehir

KKüütahyatahya

BursaBursa

YYalovaalova

IstanbulIstanbulTTekirdagekirdag

EdirEdirneneKirklareliKirklareli

ÇÇanakkaleanakkale

BalikesirBalikesir

ManisaManisaIzmirIzmir UsakUsak

AAydinydin

DenizliDenizliBurdurBurdur

IsparIspartata

AfyonAfyon

AntalyaAntalya

MuglaMugla

KirikkaleKirikkale

ÇÇorumorum

YYozgatozgat

KirsehirKirsehir

AdanaAdana

OsmaniyeOsmaniye

KahramanKahraman Maras Maras

GaziantepGaziantep

KilisKilis

SanliurSanliurfafa

ArdahanArdahan

ANKARAANKARA

AArraass

DDüüzcezce

SYRIAN ARABREPUBLIC

IRAQ

ARMENIA

GEORGIA

BULGARIARUSSIAN FEDERATION

AZER-BAIJAN

AZER.

To Batumi

To Kirovakan

ToMaku

ToSalmas

ToOroumieh

ToDahukTo Damir

KabuTo

Al Hasakah

ToAleppo

ToBurgasTo

Kurdzhali

Kuzey Anadolu Daglar i

To ros Daglar iKaraman

Konya

Aksaray

Nigde

Nevsehir Kayseri

Malatya

Adiyaman

Elazig

Tunceli

Erzincan

BingölMus

Bitlis

Diyarbakir Batman Siirt

SirnakMardin

Hakkari

Van

AgriIgdir

Erzurum

Kars

Artvin

RizeTrabzon

Gümüshane¸Bayburt

Giresun

Ordu

Tokat

Sivas

Amasya

Samsun

Sinop

Kastamonou

Çankiri

Karabük

BartinZonguldak

Bolu

Düzce

Sakarya(Adapazari)

Kocaeli(Izmit)

Bilecik

Eskisehir

Kütahya

Bursa

Yalova

IstanbulTekirdag

EdirneKirklareli

Çanakkale

Balikesir

ManisaIzmir Usak

Aydin

DenizliBurdur

Isparta

Afyon

Antalya

Mugla

Kirikkale

Çorum

Yozgat

Kirsehir

Icel(Mersin)

Adana

Hatay (Antakya)

Osmaniye

Kahraman Maras

Gaziantep

Kilis

Sanliurfa

Ardahan

ANKARA

SYRIAN ARABREPUBLIC

IRAQ

ISLAMICREP. OFIRAN

ARMENIA

GEORGIA

BULGARIARUSSIAN FEDERATION

AZER-BAIJAN

AZER.

GRE

ECE

GREECE

TuzGölü

HoyranGölü

BaysehirGölü

AksehirGölü

Lake Van

Çoruh

Murat

Kura

Aras

Firat

Tigris

Euphrates

Sakarya

Devrez

Kizil

Cekerek

Kizi

l

Seyh

an

Göksu

Ceyhan

Kelkit

B lack Sea

Medi terranean Sea

Sea ofMarmara

Gulf ofAntalya

Bosporus

Dardanelles

To Batumi

To Kirovakan

ToMaku

ToSalmas

ToOroumieh

ToDahukTo Damir

KabuTo

Al Hasakah

ToAleppo

To Ladhiqiyah

ToBurgasTo

Kurdzhali

ToKomatini

Kuzey Anadolu Daglar i

To ros Daglar i

Mt. Ararat(5166 m)

26°E 28°E 30°E 32°E 34°E 36°E 38°E

42°E 44°E30°E28°E 32°E 34°E

36°N

38°N

40°N

42°N

40°N

42°N

TURKEY

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

0 50 150100

0 50 100 150 Miles

200 Kilometers

IBRD 33501

JAN

UA

RY 2005

TURKEYPROVINCE CAPITALS*

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

PROVINCE BOUNDARIES*

INTERNATIONAL BOUNDARIES

*Province names are the same as their capitals.