evaluation of the clean air agenda ...€¦ · web view1 introduction 1 2 background 2 2.1 profile...

150
Evaluation of the International Actions Theme of the Clean Air Agenda November 2010

Upload: others

Post on 08-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Evaluationof the

International Actions Theme of the

Clean Air Agenda

November 2010

Page 2: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Report clearance steps

Planning phase completed March 2009Report sent for management response May 2010Management response received June 2010Report completed June 2010Report approved by Departmental Evaluation Committee November 2010

Acronyms used in the report

ADM Assistant Deputy MinisterAPEC Asia–Pacific Economic Cooperation APP Asia–Pacific Partnership on Clean Development and ClimateARAF Accountability, Risk and Audit FrameworkAWG Ad Hoc Working GroupCAA Clean Air AgendaCAA–RMS Clean Air Agenda Results Management SecretariatCARA Clean Air Regulatory AgendaCCME Canadian Council of Ministers of the EnvironmentCCS Carbon capture and storageCED Clean Energy DialogueCESD Commissioner of the Environment and Sustainable DevelopmentCOP Conference of the PartiesCSLF Carbon Sequestration Leadership Forum DFAIT Foreign Affairs and International Trade CanadaDG Director GeneralDGMC Director General Management CommitteeDGTLCC Director General Theme Leads Coordinating CommitteeDM Deputy MinisterEC Environment CanadaEGTT Expert Group on Technology TransferG&C Grants and contributionsGHG Greenhouse gasHMARF Horizontal Management and Accountability Reporting FrameworkIEA International Energy AgencyIC Industry CanadaIPCC Intergovernmental Panel on Climate ChangeITL International Transaction LogLDCs Least Developed CountriesLEG Least Developed Countries Expert GroupLULUCF Land Use, Land Use Change and ForestryM2M Methane to MarketsMEF Major Economies Forum on Energy and ClimateNGO Non-governmental organizationNOx Nitrogen oxidesNRCan Natural Resources CanadaOECD Organisation for Economic Co-operation and DevelopmentPEMA Pollution Emission Management AreaPIC Policy and Implementation CommitteePM Particulate matter

i

Page 3: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

REEEP Renewable Energy and Energy Efficiency PartnershipSIDS Small Island Developing StatesSO2 Sulphur dioxideUNFCCC United Nations Framework Convention on Climate ChangeEPA Environmental Protection Agency

Acknowledgements

The Evaluation Project Team was led by Linda Lee, under the direction of the Environment Canada Evaluation Director, Shelley Borys, and included Dorota Geissel and Heather Low from Foreign Affairs and International Trade Canada; Alan Amey and Annette Hollas from Natural Resources Canada; and Rosemary Clarke from Environment Canada. The Team would also like to thank all individuals who provided assistance to the project, including:

all interviewees who provided their insights on the International Actions Theme; the expert focus group members, who provided critical analysis and opinions on the

Clean Air Agenda’s international accomplishments and challenges; and all departmental personnel who responded to our inquiries and requests for

documentary evidence, and provided detailed comments crucial to the development of this report and the associated appendices.

This report was prepared for Environment Canada by Stratos, Inc., in collaboration with Marbek and with Environmental and Management Consultants.

ii

Page 4: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

TABLE OF CONTENTS

EXECUTIVE SUMMARY........................................................................................................V1 INTRODUCTION..............................................................................................................12 BACKGROUND...............................................................................................................2

2.1 Profile – International Actions Theme Program Components......................................22.2 Governance Structure for the International Actions Theme.........................................72.3 Resource Allocation for Program Components............................................................72.4 Program Logic Model...................................................................................................9

3 EVALUATION DESIGN.................................................................................................113.1 Purpose, Objectives and Scope of the Evaluation.....................................................113.2 Evaluation Approach and Methodology.....................................................................133.3 Limitations..................................................................................................................13

4 FINDINGS BY EVALUATION QUESTION....................................................................144.1 Rating of Findings......................................................................................................144.2 Evaluation Issue: Relevance – Continued Need.......................................................154.3 Evaluation Issue: Relevance – Alignment with Government Priorities......................194.4 Evaluation Issue: Relevance – Alignment with Federal Roles and Responsibilities..264.5 Evaluation Issue: Performance – Achievement of Expected Outcomes....................294.6 Evaluation Issue: Performance – Demonstrated Efficiency and Economy................474.7 Findings for DFAIT’s Class of Contributions in Support of International Environmental Programs 57

5 CONCLUSIONS.............................................................................................................645.1 Relevance..................................................................................................................645.2 Performance..............................................................................................................65

6 RECOMMENDATIONS..................................................................................................697 MANAGEMENT RESPONSE........................................................................................72ANNEX 1 EVALUATION FRAMEWORK.............................................................................79ANNEX 2 Summary of Findings...........................................................................................90

LIST OF TABLES

Table 1: Resource Allocations by Department and Program Component...............................8Table 2: Grants and Contributions Allocations by Departments and Program Component....9Table 3: Ratings Legend........................................................................................................14Table 4: Departmental Priorities............................................................................................23Table 5: PM Annex Planned Spending vs. Actual Spending, 2007–08 to 2008–09..............52Table 6: DFAIT Grants and Contributions, 2007-–2008 –to 2009–2010...............................60

LIST OF FIGURES

Figure 1: International Actions Theme Logic Model..............................................................10

iii

Page 5: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

iv

Page 6: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Executive Summary

Scope and Objectives of the Evaluation

This evaluation was conducted as part of Treasury Board Secretariat requirements to evaluate the program components grouped under the Clean Air Agenda (CAA) International Actions Theme. The objective of this evaluation was to assess the relevance and performance of the Theme, focusing on the fiscal years 2007–08 to 2009–10. The evaluation covered all program components within the Theme over that period, including Foreign Affairs and International Trade Canada’s (DFAIT’s) Class of Contributions in Support of International Environmental Programs. Given the timing of the evaluation, which occurred before the end of the implementation phase of this theme (ending 2010–11), the focus was on program outputs and movement toward the achievement of early outcomes.

Theme Components

The International Actions Theme contributes to achieving the ultimate objectives of the CAA by addressing air quality–related emissions whose sources originate from outside Canada, and by ensuring that international actions on climate change and air pollutants are in line with Canada’s interests. The Theme contributes to these objectives under four program components: (1) the Particulate Matter (PM) Annex; (2) International Partnerships and Negotiations; (3) International Financial and Other Obligations; and (4) the Asia–Pacific Partnership on Clean Development and Climate (APP). These components are implemented, with varying levels of responsibility, by four federal departments: Environment Canada (EC), DFAIT, Natural Resources Canada (NRCan) and Industry Canada (IC). The components involve activities intended to advance Canada’s international actions to improve Canadian air quality and reduce greenhouse gas (GHG) emissions, and that are currently funded through a $50-million funding envelope over the four-year period from 2007–08 through 2010–11.

Evaluation Methodology and Findings

The evaluation addressed five key issues: (1) the continued need for the program components within the International Actions Theme; (2) alignment with Government priorities; (3) alignment with federal roles and responsibilities; (4) achievement of expected outcomes; and (5) demonstrated efficiency and economy. To examine these issues and the specific evaluation questions related to each, the evaluation employed three main lines of inquiry: (1) document review; (2) key-informant interviews; and (3) a focus group of experts. The evaluation scope solely addressed Canada’s performance at the international level and did not include its ability to meet Kyoto Protocol targets. In addition, the evaluation was limited in its ability to consult with other countries’ negotiators and obtain their unique perspective on Canada’s influence internationally.

Findings and Conclusions

Listed below are the five key evaluation issues, and, under each of those, the evaluation’s findings and conclusions (numbered) are summarized.

v

Page 7: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Issue 1: Continued Need for the International Actions Theme Programs

1. The program components and activities under the CAA International Actions Theme are relevant and continue to be needed to achieve expected outcomes.

Issue 2: Alignment with Government Priorities

2. Activities are in direct alignment with federal government priorities. 3. Activities are aligned with the participating departmental priorities.

Issue 3: Alignment with Federal Roles and Responsibilities

4. Activities are aligned with federal roles and responsibilities.

Issue 4: Achievement of Expected Outcomes

5. With some delays, progress has been made in the achievement of outputs in component areas. The following expected outputs have been produced:

• Assessed contributions have been delivered to international bodies.• Canada’s Kyoto Protocol National Registry for carbon dioxide (CO2) emission units is

in place.• Negotiations for a new post 2012 climate regime have been carried out under the

United Nations Framework Convention on Climate Change (UNFCCC). • Policies, negotiating positions and advice have been produced on climate change,

although only initial research and early negotiating positions have been completed for air quality components.

• There has been participation in a variety of international bodies and initiatives outside of the UNFCCC, and support has been provided to foster partnerships. For example, a range of initiatives such as the Expert Group on Technology Transfer (EGTT) and Least Developed Countries Expert Group (LEG), as well as contribution programs, have built capacity, increased understanding in areas such as adaptation, advanced discussions on key issues in negotiations, and advanced clean technologies.

• Formal and informal information sessions have taken place with other levels of government, Aboriginal groups and domestic stakeholders to support climate change activities.

However, there were significant delays in achieving certain activities and outputs:• The National Registry for CO2 emission units was approximately six months late.• Many activities under the APP were not initiated in the first two years. • Work on a PM Annex to the Canada–United States Air Quality Agreement has not

proceeded as planned.6. Some progress has been made in the achievement of immediate outcomes, with

achievement of the following expected immediate outcomes:• Canada has complied with its financial and other obligations under international

climate change treaties and agreements.• Canada’s positions with respect to the post-2012 international climate change

framework have been clearly and consistently communicated (within the context of the UNFCCC) in a range of other multilateral and bilateral fora, and with strategic allies such as members of the Umbrella Group (a loose coalition of non–European Union developed countries). In addition, a number of these positions are reflected in

vi

Page 8: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

the Copenhagen Accord, although key gaps remain. However, it is difficult to attribute the advancement of these interests directly to Canadian international activities under this theme, particularly given the multiple factors involved in international negotiations.

• There has been improved information sharing and more formal engagement mechanisms with provinces and territories, but enhanced consultation activities have not necessarily resulted in increased consideration of stakeholder interests in Canada’s positions. For the PM Annex, this evaluation found no evidence that stakeholders were consulted directly and it was not clear how Canada’s position was developed or how the views of domestic partners and stakeholders have had an influence or been meaningfully reflected.

• There has been a small increase in the awareness of trade-related opportunities pertaining to climate-friendly technologies.

• While bilateral and multilateral relationships have been advanced through voluntary (i.e., non-assessed) contributions, and contributions have contributed to a shared understanding of issues and advancing Canada’s position abroad, there is no evidence yet of a return on investment in terms of GHG reductions.

7. No significant unintended outcomes were identified, but opportunities are emerging from the UNFCCC process to enhance Canada’s export opportunities and partnerships for clean technology.

8. External factors have influenced success and delayed the achievement of outcomes. Key factors include: changes in Canadian policy direction and positions; higher than anticipated demands regarding international climate change; challenges within the UNFCCC decision-making process; and staff turnover and Ministerial changes.

Issue 5: Demonstrated Efficiency and Economy

9. There are a limited number of more economic and efficient means to achieve objectives, and such means have not been the focus of program implementation. However, there are options in terms of how Canada uses its voluntary contributions and how to best manage the APP Secretariat.

10. Some options to improve efficiency have been implemented. Particularly beneficial has been the establishment of a Chief Negotiator and Ambassador for Climate Change in the federal government.

11. There has been progress in program implementation as planned, but there have been significant delays in some areas. Initial lack of progress for the APP and negotiating the PM Annex led to under-spending during the first two years. However, there have been higher than expected demands in the International Partnerships and Negotiations component.

12. The Horizontal Management and Accountability Reporting Framework (HMARF) is in place and is facilitating government-wide reporting of financial performance, but is not yet supporting decision making at the program level.

13. Performance data are collected and reported, but not fully across all program components; and such data are not used by managers.

vii

Page 9: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Recommendations

The following recommendations were developed based on evaluation findings and conclusions. These recommendations are directed to the responsible Assistant Deputy Ministers of the International Affairs Branch at EC, the Global Issues Branch at DFAIT, the Energy Sector at NRCan, and the Industry Sector at IC, and the Associate ADM of the Environmental Stewardship Branch at EC.

The following recommendations were developed to continue improving and advancing the CAA International Actions Theme.

Recommendation 1: EC, in consultation with other participating departments, should review and implement options for enhancing coordination and management of program components in order to improve the strategic direction and effectiveness for the Theme overall.

Although there is some overall coordination across departments, each program component within the International Actions Theme is essentially implemented and managed separately by the department(s) responsible. This theme is not managed or implemented as a “theme” as originally articulated. Therefore, areas for improving possible synergies and connections between the goals of program components and departmental mandates should be explored, and formal mechanisms should be put in place (e.g., regular senior-level meetings across participating departments) to ensure that shared objectives for this theme and the CAA overall are met.

Recommendation 2: EC should consider the costs and benefits of integrating PM Annex program activities with Clean Air Regulatory Agenda (CARA) activities.

Streamlining all air quality activities together within the Department can lead to a more coordinated and coherent approach to planning, completing and reporting on the results for initiatives related to PM. Under the existing CAA structure, the majority of air quality activities are funded by sources within CARA. Since PM Annex activities are more closely aligned with the domestic objectives for CARA than to international climate change activities, EC should explore the appropriateness of integrating PM Annex resources and activities with those of CARA. This integration should facilitate the coordination of all work related to PM at EC.

Recommendation 3: EC should improve the effectiveness of its engagement with provinces, territories and other domestic stakeholders for the PM Annex, by considering the most appropriate formal mechanisms for consultation that will provide strategic direction for the future. There was little evidence that the provinces, territories and other domestic stakeholders were engaged as part of the PM Annex’s negotiations process. Given the role that these groups play in implementing clean air and clean energy programs and technologies, regional partners and other levels of governments should be engaged to enhance opportunities and articulate region-specific issues and context to inform priorities and negotiating positions for the Annex. Because the Annex experienced some delays in implementation, it is imperative to develop effective processes that will enable the Annex to move forward efficiently toward achieving its expected outcomes in the future. EC should therefore review existing processes for consultations and consider formal mechanisms for

viii

Page 10: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

implementation of consultations that will provide strategic direction for the Annex’s negotiating positions.

Recommendation 4: EC, DFAIT and NRCan should develop an overarching strategic framework to guide the selection of international climate change voluntary contributions to international bodies and programs.

Within the International Actions Theme, there are multiple and sometimes conflicting goals assigned to Canada’s voluntary (non-assessed) contributions to international bodies and programs, which makes it difficult to determine the cohesiveness of these contributions in meeting broader thematic objectives. While some program components allocate funds using specific selection criteria based on their specific objectives, some components allocate funding based on opportunity rather than proactively. Therefore, an overarching framework should be developed to guide the selection of the International Actions Theme’s voluntary contribution projects. This framework should articulate the overarching strategic priorities, criteria for selection, appropriate terms and conditions, and broad processes to review, select, approve and monitor/oversee contribution projects.

This recommendation’s aim is to ensure that all funds are allocated to those initiatives that can best achieve the Theme’s broad expected outcomes. This overarching strategic framework should apply to all voluntary contributions in the Theme, to enable a more integrated process for soliciting, reviewing and approving proposals based on common expected outcomes and priorities (e.g., GHG reduction potential, trade-development potential, or value in developing partnerships, or some combination of these or other objectives). The intention is to have a Theme-level framework and a set of priorities that govern all voluntary grants and contributions programs under the Theme. However, this framework is intended to guide, not replace, the component-specific processes that are already in place for program components.

To support international partnerships and negotiations, this contributions strategy should also consider the goal of developing strategic alliances with developing countries, particularly with Least Developed Countries (LDCs) and Small Island Developing States (SIDS), to better understand the potential adverse effects associated with climate change (e.g., adaptation in SIDS and LDCs), thereby building on commitments to support the poorest and those most vulnerable to climate change. Engagement with other federal departments not included in the International Actions Theme (e.g., the Canadian International Development Agency) may be beneficial.

Recommendation 5: EC, NRCan, DFAIT and IC should consider how APP program activities and outputs can be used to provide trade opportunities for the development, diffusion and deployment of clean technologies.

The initial portfolio of APP projects focused less on technology deployment and more on capacity building and technology research. However, opportunities for Canadian businesses to develop and promote clean technologies internationally are emerging as large developing countries articulate their climate change actions and technology requirements to reduce GHG emissions. In order to maximize these opportunities (e.g., enabling technologies for smart-grid development in countries looking to improve energy efficiency), federal departments should explore how the International Actions Theme’s voluntary contributions can be used to increase support for these emerging trade opportunities for Canadian businesses.

ix

Page 11: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Recommendation 6: EC, DFAIT, NRCan and IC should develop and implement mechanisms that will ensure the ongoing collection of meaningful and accurate performance measurement data.

Given that this evaluation occurred early in program implementation, the achievement of immediate outcomes was assessed at this time. Ongoing performance measurement data are required to assess progress toward achievement of intermediate and final outcomes in subsequent evaluations. However, some program components under this theme do not have a clear strategy for collecting ongoing performance information. As well, the performance measurement data collected as part of the HMARF are not used by, or seen as useful for, program managers. Therefore, EC, in consultation with participating departments and working with the Clean Air Agenda Results Management Secretariat (CAA–RMS), should enhance the performance measurement strategy for this theme, including the development of specific, measureable performance indicators. A clear plan should then be developed for the ongoing collection of performance measurement data, and mechanisms should be established in all participating departments to maintain the accuracy and usefulness of these data for each of the program components under this theme.

For example, performance indicators should be developed and a mechanism should be implemented to collect and maintain these performance measurement data for APP-funded projects. While it is too early to assess project success, there should be a system in place that will capture outcomes for funded projects so that these outcomes can be assessed against established APP objectives in the future. Because of the dynamic nature of the policy directions and context of some activities under the International Actions Theme, it is challenging to develop accurate and useful ongoing performance indicators. Flexible strategies for program-component-level performance measurement and reporting may be considered to account for changing policy directions, but a clear plan should be developed that describes how these data will be recorded and managed.

Management ResponseThe Assistant Deputy Ministers (ADMs) of the International Affairs Branch at Environment Canada, the Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), the Energy Sector at Natural Resources Canada (NRCan), the Industry Sector at Industry Canada and the Associate Assistant Deputy Minister (AADM) of the Environmental Stewardship Branch at Environment Canada, accept the evaluation and all of its recommendations, and have provided a plan to implement the following management actions in response to the evaluation recommendations.

Recommendation 1: Environment Canada, in consultation with other participating departments, should review and implement options for enhancing coordination and management of program components to improve the strategic direction and effectiveness for the theme overall.

The ADM of International Affairs Branch at Environment Canada (EC) agrees with this recommendation.Environment Canada will work with the Department of Foreign Affairs and International Trade (DFAIT), Natural Resources Canada (NRCan) and Industry Canada to review and implement options for enhancing coordination and management of program components to

x

Page 12: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

improve the strategic direction and effectiveness for the theme overall. Meetings of all International Actions Theme Program Director Generals will be held every four months to facilitate communication across departments at the management level, provide strategic direction, discuss interconnections between programs, provide a forum for thematic issues and to review level of achievement towards the International Actions Theme goals

Timeline Deliverable(s) Responsible PartyMarch 31, 2011 Quarterly meetings of all

International Actions Theme Program Directors General (first meeting to be held in summer of 2010).

Director General, Climate Change International Directorate, EC

Recommendation 2: Environment Canada should consider the costs and benefits of integrating PM Annex program activities with Clean Air Regulatory Agenda (CARA) activities.

The ADM of the International Affairs Branch and the AADM of the Environmental Stewardship Branch agree with this recommendationParticulate Matter (PM) Annex activities are more closely aligned to the domestic objectives of the Clean Air Regulations Agenda (CARA) than to international climate change activities. Both the CARA and PM Annex programs’ existing funding will expire March 31, 2011. Senior management has considered the efficiencies gained from single-point reporting instead of reporting to two funding programs as well as the strong alignment with CARA-related domestic goals and concluded that work related to the PM Annex would be more effective and efficient if included in the CARA program. Initial consideration for future funding of CARA is underway and includes the PM Annex. The PM Annex will begin to officially integrate into the CARA program this fiscal year, while continuing to report through the International Actions Program.

Timeline Deliverable(s) Responsible PartyMarch 31, 2011 Integrate PM Annex

activities into CARA activities this year.

Director General, Strategic Priorities Directorate, Environmental Stewardship Branch, EC

Recommendation 3: Environment Canada should improve the effectiveness of their engagement with provinces, territories and other domestic stakeholders for the PM Annex by considering the most appropriate formal mechanisms for consultation that will provide strategic direction for the future.

The AADM of Environmental Stewardship Branch agrees with this recommendationThe content of the PM Annex will be based on the respective domestic policies to address air pollution in Canada and the U.S.

Provinces and territories were consulted formally on Canada’s proposed domestic approach in 2007 through the Environmental Planning and Protection Committee (EPPC) of the Canadian Council of Ministers of the Environment.

These consultations led to the formation of a multi-stakeholder process that included members of provincial and territorial governments, industry and environmental groups. A

xi

Page 13: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Canadian domestic approach to address air pollutants has been the subject of the work of this group over the last two years.

We will be working to further elaborate our approach (as will the U.S) and when both countries are in a position to engage in negotiations, an appropriate process for engagement, using existing modalities where possible, will be set up.

There is also provincial representation on the subcommittees set up under the Canada-US Air Quality Agreement that allows for direct participation in the process.

Timeline Deliverable(s) Responsible PartyTBD Domestic consultations

take place prior to further negotiations on the PM Annex under an appropriate process.

Director General, Strategic Priorities Directorate, Environmental Stewardship Branch, EC

Recommendation 4: Environment Canada, Foreign Affairs and International Trade Canada, and Natural Resources Canada should develop an overarching strategic framework to guide the selection of voluntary contributions to international bodies and programs.

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT) and Energy Sector of Natural Resources Canada (NRCan) agree with this recommendationEC will, in close consultation with DFAIT and NRCan, work to create an overarching strategic framework to guide the selection of voluntary (non-assessed) contributions to international bodies and programs.

A strategic framework was developed to ensure that projects funded under the Asia Pacific Partnership program met defined objectives. The experience of developing an APP strategic framework will serve as a useful guide to development of the overarching framework.

The implementation of the framework will guide the allocation of contribution funds to recipients that will best contribute to achieving the program objectives.

The framework will be developed in this final funded year to ensure the balance of funds is distributed in the most effective manner.

Should a new program of international actions be funded beyond the current financial year that includes similar voluntary contributions, this framework could be used as the basis for managing the new funds.

Timeline Deliverable(s) Responsible PartySeptember 30, 2010. Develop a strategic framework Lead:

Director General, Climate Change International Directorate, EC

Support:Director General, Energy

xii

Page 14: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

and Sustainable Development Directorate, DFAIT

Director General, Energy Policy Branch, NRCan

March 31, 2011 Test the framework using contribution funds for fiscal year 2010-11.

Lead:Director General, Climate Change International Directorate, EC

Support:Director General, Energy and Sustainable Development Directorate, DFAIT

Director General, Energy Policy Branch, NRCan

Recommendation 5: Environment Canada, NRCan, DFAIT and Industry Canada should consider how program activities and outputs can be used to provide trade opportunities for the development, diffusion and deployment of clean technologies

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry Canada (IC) agree with this recommendationThe clean energy technology program has worked with Canadian private industry, many of which are Small and Medium Sized Enterprises (SMEs), on baseline-setting and research projects with some deployment of clean technology projects. SMEs account for 45% of gross domestic product (GDP), much of the economy’s growth, 60% of all jobs in the economy, and 75% of net employment growth1 As we become aware, through the climate change negotiation sessions and other relevant international processes covered by the program, of emerging trade opportunities related to climate change, we agree that we need to increase SMEs awareness of these opportunities in order to increase deployment of clean technology.

The Departments will create a plan for better communication of clean technology trade opportunities to SMEs and will implement some activities prior to funding termination on March 31, 2011. The plan can be used as designed if the program is continued or as a foundation if the program is continued with changes.

In addition to ensuring information sharing with the Canadian SMEs, we can better inform the international community of Canada’s developing clean technology products to increase trade opportunities. For example, participation in the APP has resulted in sharing of

1 Public Works and Government Services Canada, “Importance of SMEs.” Accessed online on 21 May 2009 at http://www.tpsgc-pwgsc.gc.ca/app-acq/pme-sme/importance-eng.html.

xiii

Page 15: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

technology information and products among the APP countries thereby increasing world awareness of Canada’s clean technology.

Established in 2009, and not included in the scope of this Evaluation, EC leads the Canada-U.S. Clean Energy Dialogue (CED), with support from NRCan and DFAIT. The CED aims to enhance bilateral collaboration in advancing clean energy technology solutions to reduce greenhouse gas emissions and address climate change. The focus areas of the CED include: expanding clean energy research and development; advancing the development and deployment of clean energy technologies, with a focus on carbon capture and storage; and building a more efficient electricity grid based on clean and renewable generation.  EC, NRCan and DFAIT will continue to work together under the CED, and where appropriate, engage with other relevant departments, such as IC, to advance the development and deployment of clean energy technologies. A result of this program could be an increase in Canadian business trade opportunities in clean technologies, although this is not a specific goal of the program.

Beyond the International Actions Theme, NRCan leads a number of key partnerships with organizations such as the International Energy Agency, the Climate Technology Initiative and the Global Carbon Capture and Storage Institute (GCCSI), which contribute significantly to advancing private sector engagement and should be further leveraged in order to maximize support for Canadian business opportunities on clean technologies.

Also beyond the Theme, Industry Canada has been participating actively in the International Partnership for Hydrogen & Fuel Cell in the Economy (IPHE) and working with Canadian companies to build hydrogen and fuel cell industry worldwide. IC is also engaged in the Steel Committee of the Organization for Economic Cooperation and Development (OECD) and the North American Steel Trade Committee (NASTC). While neither the Steel Committee of the OECD nor the NASTC is a forum that is meant to deal with climate change issues, they are often raised by industry representatives and IC officers as being important. These issues will no doubt affect the sector's competitiveness going forward because of different environmental obligations for developed and developing countries

Timeline Deliverable(s) Responsible PartyOctober 31, 2010 NRCan, IC, DFAIT and

EC representatives will develop an action plan to increase the Canadian SMEs’ awareness of increased opportunities in clean technology

Lead:Director General, Climate Change International Directorate, EC

Support:Director, Resource Manufacturing and Value-Added, Industry Sector, IC Director, Partnerships, Climate Change International Directorate, EC

Director, International Affairs, NRCan

Director, Climate Change

xiv

Page 16: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

and Energy Division, DFAIT

March 31, 2011 Implement activities identified in the action plan.

Lead:Director General, Climate Change International Directorate, EC

Support:Director, Resource Manufacturing and Value-Added, Industry Sector, IC

Director, Partnerships, Climate Change International Directorate, EC

Director, International Affairs, NRCan

Director, Climate Change and Energy Division, DFAIT

Recommendation 6: Environment Canada, DFAIT, NRCan and Industry Canada should develop and implement mechanisms that will ensure the ongoing collection of meaningful and accurate performance measurement data.

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry Canada (IC) agree with this recommendationAs mentioned in the Evaluation Report, gathering meaningful performance measurement data for policy-related outcomes is extremely difficult and must be qualitative in nature rather then quantitative. We recognize that we need better measurement of outcomes for the policy-related components of the program and EC will, in close cooperation with DFAIT and NRCan, enhance the performance measurement framework to be more flexible regarding the policy-related components of the International Actions Theme and establish a method of tracking performance for use in decision making.

The APP and Methane to Markets components of the program provide an opportunity to track performance data against program goals (reductions in greenhouse gases, economic benefits for Canadians and accelerating development of clean technology). On project completion, the companies will be asked to report on the achievements of their projects against the three priorities.

Timeline Deliverable(s) Responsible PartySeptember, 2010 Establish a system to

track performance of policy-related activities

Lead:Director General, Climate Change International

xv

Page 17: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

under the program against qualitative performance measurement indicators.

Directorate, EC

Support:Director General, Environment, Energy and Sustainable Development Bureau, DFAIT;

Director General, Energy Policy Branch, NRCan

February, 2011 Establish a system to report level of achievement of meeting the three priorities of the APP program.

Director General, Climate Change International Directorate, EC

xvi

Page 18: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

1 Introduction

This evaluation was undertaken from December 2009 to April 2010 in order to evaluate the relevance and performance of the Clean Air Agenda’s (CAA’s) International Actions Theme. The intention of this evaluation is to meet the information requirements, related to international initiatives under the CAA, of senior management within Environment Canada (EC), Foreign Affairs and International Trade (DFAIT), Natural Resources Canada (NRCan), and Industry Canada (IC)—and to meet the information needs of the Treasury Board Secretariat. As well, international commitments were identified as a risk area in the 2008–09 Audit and Evaluation Plan for EC, which was approved by the Departmental Evaluation Committee on April 4, 2008.

For the International Actions Theme, EC’s Evaluation Division led the evaluation and represented IC’s interests. The CAA International Actions Theme Evaluation Committee performed an oversight role throughout the evaluation process, providing input and feedback to the consultant hired to conduct the evaluation, approving all deliverables outlined in the contract for evaluation, and coordinating the management response to evaluation recommendations in the evaluation report.

This report summarizes the evaluation process, findings, conclusions and recommendations, and comprises the following sections:

Background describes the CAA’s International Actions Theme and related program components, including the governance structure, allocated resources and the expected outcomes;

Evaluation Design describes the evaluation objectives, scope, issues, and approach taken;

Findings by Evaluation Question documents the evaluation findings across the four program components;

Conclusions; Recommendations; Management Response.

1

Page 19: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

2 Background

The International Actions Theme is one of eight themes under the CAA architecture, which includes 43 programs and eight federal departments and agencies. The other seven themes within the CAA are: Clean Air Regulations, Clean Energy, Clean Transportation, Indoor Air Quality, Adaptation, Partnerships, and Management and Accountability. Each lead department is responsible for conducting evaluations at the thematic level under the CAA. The evaluation results for all themes will be aggregated into a horizontal evaluation of the entire CAA. EC is the lead department on the overall CAA and is responsible for implementation of the horizontal evaluation.

This evaluation focuses only on the International Actions Theme. EC has the lead in coordinating program activities and the evaluation under this theme with the other involved departments: DFAIT, NRCan and IC. Program components under this theme involve activities intended to advance Canada’s international action, improve Canadian air quality and reduce greenhouse gas (GHG) emissions. Specific program outcomes of each of the four components captured under this theme collectively contribute to achieving the Theme’s broad results. In turn, the international theme contributes to achieving the ultimate objectives of the CAA by addressing air quality–related emissions whose sources originate from outside Canada and by ensuring that international actions on climate change and air pollutants are in line with Canada’s interests. The activities within the International Actions Theme have received funding for the period 2007–08 through 2010–11.

The purposes of the activities are to address the impacts of poor air quality and climate change, including respiratory-related hospital admissions and premature mortality due to particulate matter (PM); and climate change impacts on energy security, sustainable development, health, trade and industrial growth. These activities respond to the Speech from the Throne commitments in April 2006 related to “measures to achieve tangible improvements in our environment, including reductions in pollution and greenhouse gas emissions.” These international activities reflect the fact that Canada’s domestic air quality goals are influenced by international activities and that, to achieve progress, Canada needs to be aligned with international actions.

2.1 Profile – International Actions Theme Program Components

The CAA International Actions Theme comprises four program components:

1. Particulate Matter Annex 2. International Partnerships and Negotiations3. International Financial and Other Obligations4. Asia–Pacific Partnership on Clean Development and Climate

Activities within International Partnerships and Negotiations2 and International Financial and Other Obligations are related to the United Nations Framework Convention on Climate Change (UNFCCC) and the Kyoto Protocol,3 while the PM Annex and Asia–Pacific 2 Some activities under International Partnerships and Negotiations are not United Nations (UN)–related (e.g., participation at international fora such as the G8 and G12). 3 Activities within the International Actions Theme that are related to Canada’s obligations under the Kyoto Protocol involve the development of climate change policy, participation in negotiations at international fora, and meeting Canada’s financial obligations as a member of the Conference of the Parties of the United Nations Framework Convention on Climate Change. Other activities related to the Kyoto Protocol are funded under other sources and are not part of this evaluation.

2

Page 20: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Partnership on Clean Development and Climate (APP) are not. Specific program outcomes for each of these components collectively contribute to achieving the broad results of the International Actions Theme. The participating departments within this theme have varying levels of responsibility across these four components, as described below.

2.1.1 Particulate Matter Annex

PM comprises air-bound, microscopic pollutants that have negative impacts on air quality, the environment and human health. Led by EC, this component of the CAA International Actions Theme strives to establish and maintain negotiated commitments between Canada and the United States to reduce PM emissions and their flow between the two countries.

Historically, Canada and the United States have had a solid history of cooperation on air quality, dating from the signature of the bilateral Air Quality Agreement in 1991. An Ozone Annex to address transboundary ozone formation was negotiated and signed in 2000. Since then, further technical and scientific analysis culminated in a joint Canada-U.S. science assessment of transboundary PM in 2004. This assessment demonstrated that further reductions in transboundary pollution, including PM, would help both countries make continued progress in protecting human health and the environment. Canada’s Clean Air Regulatory Agenda (CARA) and, more particularly, the proposal to have a national regulatory framework in Canada to address various pollutants, including PM and its precursors, was welcomed by the United States.

Canada and the United States are interested in pursuing negotiations for a PM Annex. Stemming from the 1991 Air Quality Agreement, such an annex would establish commitments by the United States and Canada to reduce emissions of PM and precursors, thereby helping Canada to secure substantial improvements in air quality and associated health and environmental benefits. From a Canadian perspective, a PM Annex should provide for:

commitments by the United States to reduce sulphur dioxide (SO2) and nitrogen oxide (NOX) emissions, and to reduce emissions of primary PM, which will decrease transboundary flows of SO2 and NOx;

U.S. recognition, within the terms of the Agreement, that it is a source of transboundary PM pollution;

joint monitoring of PM levels in air and a review procedure to assess progress on these commitments against achieving the goals for air quality and acidification;

definition of a transboundary Pollution Emission Management Area (PEMA); and cooperative scientific and technical activities to further the understanding of PM, its

formation and transport, and its impacts on human health.

In addition, it is anticipated that the PM Annex could provide an opportunity to obtain a U.S. commitment to pursue the development of a cross-border emissions trading regime for SO2 and NOX.

EC’s Air Emissions Priorities Division4 (under whose auspices the Air Quality Agreement is managed) provides day-to-day administrative and coordination support related to the PM Annex.

4 Formerly the North American Smog Program.

3

Page 21: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

2.1.2 International Partnerships and Negotiations

This component focuses on Canada’s international climate change activities completed in international fora. Given the growing awareness in Canada of the increasing impacts of climate change and the need for a strategic, global approach to effectively address this issue, Canadians are looking to the Government of Canada to advance international action on climate change. Canada’s engagement in international climate change discussions and negotiations is supported by substantive policy development, research and analysis (using in-house resources), and bilateral outreach and advocacy. These activities are imperative for developing and preparing Canada’s negotiating positions, and for consultations with relevant stakeholders, in advance of Canada’s participation in multiple fora where climate change issues are addressed both within and outside the UN.

In order to deliver on expected results, EC, DFAIT and NRCan work together to provide analysis and policy input for the development and articulation of Canada’s positions and policies on international climate change. EC coordinates the Government of Canada’s participation within and outside the UN negotiation process, in close collaboration with DFAIT, NRCan and other interested departments, and leads and coordinates Canada’s efforts to shape and develop a future global approach to international climate change (post-2012). The Minister of the Environment is the Cabinet Minister responsible for the international climate change file, a role carried out in close collaboration with the Minister of Foreign Affairs.

A number of activities aim to ensure that Canadian environmental and economic interests are advanced through international negotiations and discussions. These include:

a) negotiation and development of policies, and participation in fora, relating to a wide range of climate change issues, including global mitigation actions, developing-country issues, impacts and adaptation aspects of climate change, economic diversification, market-based mechanisms, financing and investment flows, technology development, deployment, diffusion and transfer, forestry, capacity-building, research and systematic observation, and legal and administrative matters;

b) acting as Canada’s Designated National Authority for the Clean Development Mechanism;

c) development of a strategic approach for Canada’s role in a more effective and inclusive post-2012 agreement on climate change under the UNFCCC;

d) strategic bilateral engagement with other countries on international climate change in order to inform Canada’s position, identify areas of mutual cooperation, and influence the negotiating positions of key countries;

e) gathering input from and engaging domestic stakeholders—such as industry, environmental non-governmental organizations (NGOs), Aboriginal groups, and provinces/territories—on international climate change issues, to inform them and to garner support for Canada’s international negotiating positions; and

f) participation in non-UN processes (e.g., Major Economies Forum on Energy and Climate [MEF]) that support the negotiations of a future agreement on international climate change.

Additionally, DFAIT is responsible for the disbursement of additional grants and contributions (G&C) funds under the International Actions Theme. Projects funded under this program help Canada fulfill its international climate change commitments and further its

4

Page 22: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

international negotiating position. The contributions made under this program consist of those that are supplementary both to assessed contributions5 and non-assessed contributions.6

2.1.3 International Financial and Other Obligations

This component of the CAA International Actions Theme ensures that Canada meets its financial obligations under the UNFCCC7 and its Kyoto Protocol,8 and funds its share of the UN Climate Change Secretariat’s Program Budget. The UNFCCC and Kyoto Protocol are legally binding international treaties that serve as an overall framework for intergovernmental efforts to tackle the challenge posed by international climate change. Under these agreements, Canada is required to contribute its share of an assessed contribution to the UN Climate Change Secretariat’s Program Budget, which consists of a core operating budget (assessed) and supplementary budget for both the UNFCCC and Kyoto Protocol. Most recently, the assessed funding for the UNFCCC and Kyoto Protocol for 2008–09 was approximately $954,000, of which CAA funding covers half.9

This component also includes Canada’s commitments to establish a National Registry for Kyoto-recognized emission credits, a condition for Canada’s participation in the flexibility mechanisms of the Protocol (i.e., the Clean Development Mechanism, Joint Implementation, and the ability to trade national emissions allowances). National registries are required of all Annex 1–country signatories to the Kyoto Protocol, and are linked to an international, United Nations–based interactive database (the International Transaction Log [ITL]) that is used to monitor each country’s holdings and transfers of Kyoto-recognized emissions credits.

Within this component, Canada makes voluntary contributions to support initiatives in which Canada has a specific interest. These include the ITL, and funding to enable Canada to participate in and contribute to international technology initiatives outside the UNFCCC, such as Methane to Markets (M2M) and the Renewable Energy and Energy Efficiency Partnership (REEEP). The M2M partnership is an international initiative that advances cost-effective, near-term methane recovery and use as a clean energy source. REEEP is a capacity-building initiative that has completed over 130 renewable energy projects since 2004.

DFAIT acts as the focal point for overseeing financial contributions to the UNFCCC in order to ensure that assessed funding requirements are met, through providing administrative support and management coordination for the provision of assessed funding on behalf of Canada. EC serves as technical lead on developing the National Registry for emission credits, managing the connection to the interactive ITL, participating on behalf of Canada in 5 Assessed contributions refer to Canada’s assessed share of the costs of operating an international organization of which Canada is a member as a result of an act of Parliament, a Cabinet decision, an Order in Council or an international treaty.6 Non-assessed contributions refer to Canada’s voluntary contributions.7 The UNFCCC is an international treaty that sets an overall framework for intergovernmental efforts to address the challenges posed by climate change. The Convention, which entered into force on March 21, 1994, has near-universal membership with 191 countries and one regional organization (the European Economic Community) having ratified. Under the Convention, governments gather and share information on GHG emissions, national policies and best practices; launch national strategies for addressing GHG emissions and adapting to expected impacts, including the provision of financial and technological support to developing countries; and cooperate in preparing for adaptation to the impacts of climate change.8 The Kyoto Protocol is an international agreement linked to the UNFCCC. The defining feature of the Protocol is that it sets binding targets for 37 industrialized countries and the European community for reducing GHG emissions. The Protocol was adopted on December 11, 1997, and entered into force on February 16, 2005. A total of 184 parties to the Convention have ratified the Protocol. 9 The other half of assessed funding for the UNFCCC and the Kyoto Protocol is covered by DFAIT through other funding sources, and is not part of this evaluation.

5

Page 23: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

technology partnerships (e.g., REEEP), and leading Canada’s technology trade initiatives through M2M.10

2.1.4 Asia–Pacific Partnership on Clean Development and Climate

Led by EC, with input from NRCan and IC,11 the APP is a public-private partnership designed to accelerate the development, deployment and diffusion of clean energy technologies to address climate change. This partnership brings together seven countries (Australia, Canada, China, India, Japan, the Republic of Korea and the United States) in a voluntary initiative to address the issues of sustainable development, clean energy and climate change. These countries collectively represent approximately 50% of the world’s economy, population and energy use, and over 50% of the world’s GHG emissions. From a strategic perspective, the APP provides a collaborative forum for the engagement of major emitting sectors and countries that will need to realize significant reductions in business-as-usual emissions in order to stabilize global GHG concentrations at a level that will not dangerously interfere with the global climate system.

The APP is organized around eight task forces that address five energy-intensive sectors (aluminum, buildings and appliances, cement, coal mining, and steel) as well as three energy supply sectors (renewable energy and distributed generation, power generation and transmission, and cleaner fossil energy). Each task force is chaired and co-chaired by participating countries12 and oversees public sector–private sector collaborations for developing and implementing projects on climate-friendly technologies related to its particular sector. Project proposals are intended to be selected through a transparent, merit-based process.

The Policy and Implementation Committee (PIC) is an Assistant Deputy Minister(ADM)–level decision-making body within the APP that, in overseeing task force activities, has mandated responsibilities for policies, frameworks, technical advice, financing, public-private partnerships and procedures. PIC meetings are hosted annually by one of the participating countries. Canada hosted the October 2008 meeting in Vancouver, British Columbia. An Administrative Support Group currently hosted by the United States supports the PIC and APP more broadly.

Membership in the APP allows Canada to engage in joint public sector–private sector and bilateral or multilateral partnerships, and in projects related to climate-friendly technologies. Canadian delegations consisting of public and private sector representatives have been participants in the APP’s international activities through their involvement on various task forces and leadership and subject-matter expertise in the development of various clean technology projects.

Canada became the seventh member of the APP on October 15, 2007 at the 2nd Ministerial meeting in New Delhi. In June 2008, Canada made an initial commitment of C$20 million to fund Partnership activities and initiatives. $15.48 million of this contribution, managed by EC, allows Canada to partner with the Canadian private sector on strategic international clean technology projects in APP member countries. Given the amount of money available

10 Theme goals to advance the uptake of technologies and reduce GHG emissions are also pursued outside of formal UN negotiations through other fora such as the Expert Group on Technology Transfer, Carbon Sequestration Leadership Forum, and International Energy Agency.11 Industry Canada’s participation in the APP will be addressed through Environment Canada’s program evaluation activities.12 Canada was selected to the Co-Chair position on the Cement Task Force in May 2008.

6

Page 24: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

for these grants and contributions (approximately $3.8 million per year), it is anticipated that Canada’s funding may be focused on deployment and commercialization projects. It is expected that the shorter time-frame and tangible results of these types of projects should create the greatest and most immediate benefit for Canadian developers and exporters of clean technologies.

2.2 Governance Structure for the International Actions Theme

The governance of the International Actions Theme relies on existing structures within the CAA Horizontal Management and Accountability Reporting Framework (HMARF) structure that has been established to govern the CAA process. Four interdepartmental committees under the CAA HMARF accountability structure are responsible for providing strategic direction, coordination and managerial oversight of the CAA:

Deputy Minister (DM) Committee ADM Steering Committee Director General Theme Leads Coordinating Committee (DGTLCC) Director General Management Committees (DGMCs)

A DGMC governing body was established for the International Actions Theme. It includes the DGs from the Climate Change International Directorate at EC, the Environment, Energy and Sustainable Development Bureau at DFAIT, the Energy Policy Branch at NRCan, and the Service Industries and Consumer Products Branch at IC. The Committee convenes on an as-needed basis to discuss issues and activities of concern to the International Actions Theme.

The chair of the International Actions DGMC, the DG of the Climate Change International Directorate at EC, also sits on the DGTLCC of the CAA HMARF to represent the International Actions Theme. The DGTLCC is the primary operational committee responsible for implementing the elements of the CAA across all themes and for ensuring that participating departments provide a systematic review of the relevance and funding of programs—a review that is aligned with Canadians’ priorities and that effectively and efficiently delivers on the Government’s CAA responsibilities.

To support governance of the CAA, the Clean Air Agenda–Results Management Secretariat (CAA–RMS) was established to ensure that information is available to the DGTLCC, the ADM Steering Committee and the DM Committee within the CAA HMARF accountability structure. The CAA–RMS supports the oversight and monitoring of results, performance, resources and risks across the CAA themes, initiatives and programs.

2.3 Resource Allocation for Program Components

Funding of $50 million13 was allocated across four years from 2007–08 to 2010–11 for the International Actions Theme under the CAA. Table 1 describes the resource allocations for each program component by department by fiscal year.

13 Resource allocation figures presented in this report may differ from those published on the CAA horizontal initiative in Environment Canada’s Departmental Performance Reports due to differences in reporting periods.

7

Page 25: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Table 1: Resource Allocations by Department and Program Componenta

FISCAL YEAR

PM ANNEX INTERNATIONAL PARTNERSHIPS AND NEGOTIATIONS

INTERNATIONAL FINANCIAL AND OTHER OBLIGATIONS

APP TOTAL

EC $2,200,000 $11,400,000 $2,892,000 $18,882,700 $35,374,7002007–08 $550,000 $3,000,000 $1,023,000 $4,628,200 $9,201,2002008–09 $550,000 $2,900,000 $623,000 $4,738,200 $8,811,2002009–10 $550,000 $2,800,000 $623,000 $4,738,200 $8,711,2002010–11 $550,000 $2,700,000 $623,000 $4,778,200 $8,651,200

DFAIT $0 $6,400,000 $1,908,000 $0 $8,308,0002007–08 -- $1,750,000 $477,000 -- $2,227,0002008–09 -- $1,650,000 $477,000 -- $2,127,0002009–10 -- $1,550,000 $477,000 -- $2,027,0002010–11 -- $1,450,000 $477,000 -- $1,927,000

NRCan $0 $5,200,000 $0 $877,300b $6,077,3002007–08 -- $1,400,000 -- $271,800 $1,671,8002008–09 -- $1,300,000 -- $211,800 $1,511,8002009–10 -- $1,300,000 -- $211,800 $1,511,8002010–11 -- $1,200,000 -- $181,800 $1,381,800

IC $0 $0 $0 $240,000b $240,0002007–08 -- -- -- $100,000 $100,0002008–09 -- -- -- $50,000 $50,0002009–10 -- -- -- $50,000 $50,0002010–11 -- -- -- $40,000 $40,000TOTAL $2,200,000 $23,000,000 $4,800,000 $20,000,000 $50,000,000a Includes G&C expenditures. Refer to Table 2 for breakdown of G&C allocations.b Transfer of funds from EC to NRCan and IC, for operating expenditures related to Canada’s participation in the APP’s task forces. This includes Public Works and Government Services Canada accommodation costs.

EC’s G&C budget in the amount of $16.872 million is managed under the Department’s Terms and Conditions for the Class Contributions to Support Environmental Research and Development, Environmental and Sustainable Development Initiatives and Canada’s International Commitments. These Class Terms and Conditions address the range of funding activities, including research related to transboundary emissions sources (under the PM Annex), participation in environmental initiatives (under International Financial and Other Obligations), and supporting clean technology development and deployment (for the APP). As these were valid to March 31, 2010, new Terms and Conditions have been developed.

DFAIT’s $840,000 G&C program supports organizations in staging meetings and conferences, conducting studies, and undertaking activities related to climate change (under International Partnerships and Negotiations). This contribution is managed through DFAIT’s Terms and Conditions for a Class of Contributions in Support of International Environmental Programs, which was submitted to Treasury Board for approval in the fall of 2007. Evaluation of this contribution program has been incorporated into the evaluation of the International Actions Theme.

Grants and contributions of $1.908 million are realigned from EC to DFAIT to cover EC’s share of Canada’s assessed contribution to the UNFCCC and Kyoto Protocol (under International Financial and Other Obligations). This contribution is managed under DFAIT’s Terms and Conditions for Assessed Contributions to the United Nations Framework Convention on Climate Change and the Kyoto Protocol. International agreements such as the UNFCCC and Kyoto Protocol have governance mechanisms that include oversight services. DFAIT relies on oversight regimes to assist in monitoring the international

8

Page 26: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

organization’s accountability for achieving those objectives agreed to by its members for stewardship of its funds, contributed by member states including Canada.

NRCan’s $425,000 class contribution agreement supports key activities to help advance Canada’s objectives within the international climate change negotiations. Activities include support to the Climate Technology Initiative, the multilateral International Energy Agency (IEA), and the Expert Group on Technology Transfer (EGTT) under the UNFCCC.

Table 2 describes the G&C funds allocated for the International Actions Theme.

Table 2: Grants and Contributions Allocations by Department and Program Componenta

FISCAL YEAR

PM ANNEX INTERNATIONAL PARTNERSHIPS AND NEGOTIATIONS

INTERNATIONAL FINANCIAL AND OTHER OBLIGATIONS

APP TOTAL

EC $100,000 $0 $1,292,000 $15,480,000 $16,872,0002007–08 $25,000 -- $323,000 $3,770,000 $4,118,0002008–09 $25,000 -- $323,000 $3,870,000 $4,218,0002009–10 $25,000 -- $323,000 $3,870,000 $4,218,0002010–11 $25,000 -- $323,000 $3,970,000 $4,318,000

DFAIT $0 $840,000 $1,908,000 b $0 $2,748,0002007–08 -- $280,000 $477,000 -- $757,0002008–09 -- $230,000 $477,000 -- $707,0002009–10 -- $180,000 $477,000 -- $657,0002010–11 -- $150,000 $477,000 -- $627,000

NRCan $0 $425,000 $0 $0 $425,0002007–08 -- $125,000 -- -- $125,0002008–09 -- $100,000 -- -- $100,0002009–10 -- $100,000 -- -- $100,0002010–11 -- $100,000 -- -- $100,000TOTAL $100,000 $1,265,000 $3,200,000 $15,480,000 $20,045,000a No G&C payments were allocated to IC for the CAA International Actions Theme.b Transfer of $1.908 million ($0.477 million per year over four years) from EC to DFAIT to cover EC’s share of Canada’s assessed contribution to the UNFCCC and Kyoto Protocol.

2.4 Program Logic Model

The following Logic Model provides a visual representation of the CAA International Actions Theme,14 identifying the linkages between the activities and the achievement of its outcomes. Consistency has been maintained at the level of the intermediate and long-term outcomes of the International Actions Theme Logic Model and the higher-level outcomes of the CAA’s overarching Logic Model.

14 The key milestones in the development of this International Actions Theme Logic Model is presented under separate cover in a technical appendix.

9

Page 27: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Figure 1: International Actions Theme Logic Model

10

Page 28: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

3 Evaluation Design

3.1 Purpose, Objectives and Scope of the Evaluation

This evaluation was conducted as a part of the Treasury Board Secretariat’s requirement to evaluate funding for the program components of the CAA International Actions Theme. The objectives of this evaluation were to:

determine the ongoing relevance of the CAA International Actions Theme; assess the degree of implementation of the Theme; and assess the Theme programs’ performance.

This evaluation examined all planned activities for the International Actions Theme of the CAA, covering the four-year time frame from 2007–08 through 2010–11. The evaluation examined evidence provided on the program components from the start of the initiative in 2007–08 up to December 31, 2009. Given the timing of the implementation phase for this evaluation (the evaluation was conducted at this point to contribute to the overall evaluation of the entire CAA expected in October 2010), performance data for intermediate, final and even some immediate outcomes were not available. The focus of this evaluation, therefore, was on outputs and movement toward achieving early outcomes. Where processes are ongoing, the evaluation examined the extent to which the outcomes have been achieved and/or the likelihood that planned activities will achieve expected results. In addition, the evaluation examined how external factors have affected, or are likely to affect, the Theme’s ability to achieve its expected outcomes.

Of note, the International Actions Theme is linked to the UNFCCC and Kyoto Protocol through the International Financial and Other Obligations and the International Partnerships and Negotiations program components (refer to section 2.1 for the program profile), which are both within the scope of this evaluation. Activities directly related to Canada meeting Kyoto targets for emissions are not part of the International Actions Theme and are outside the scope of this evaluation.

Other initiatives that were considered in scoping this evaluation included the following:

DFAIT’s Class of Contributions in Support of International Environmental Programs, which was included in this evaluation. Originally, a formative program evaluation was planned for the end of fiscal year 2009–10 by DFAIT as part of the renewal of the Terms and Conditions for payment of contribution programs. However, that evaluation has been incorporated into this evaluation of the International Actions Theme. Indicators have been included in the evaluation matrix of the CAA International Actions Theme evaluation to capture its relevance and performance, and results will be included in the evaluation report.

The CED, which was not included in this evaluation. The CED was added to the CAA International Actions Theme in the fall of 2009, and is at the end of its first year of implementation. Therefore, even though it is part of the Theme, the CED is not part of the evaluation with the other program components. A separate evaluation is planned for the CED in 2010–2011, and results will be rolled up with results from the

11

Page 29: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

International Actions Theme evaluation in time to be aggregated to the CAA level in the fall of 2010.

Thirteen evaluation questions, grouped according to issue, were addressed in this evaluation to assess relevance and performance. (Evaluation Framework is provided in Annex 1.)

Relevance:

Issue 1: Continued Need for the Program1. Are program components within the International Actions Theme connected with key

international climate change and air quality needs?

Issue 2: Alignment with Government Priorities 2. Are program components within the International Actions Theme aligned with federal

government priorities? 3. Are program components within the International Actions Theme aligned with the

priorities of participating departments?

Issue 3: Alignment with Federal Roles and Responsibilities 4. Is there a federal role and responsibility for the Government of Canada to deliver

program components within the International Actions Theme?

Performance:

Issue 4: Achievement of Expected Outcomes 5. To what extent have intended outputs been achieved as a result of programs within

the International Actions Theme?6. To what extent have immediate outcomes been achieved as a result of programs

within the International Actions Theme?7. Have there been any unintended (positive or negative) outcomes that can be

attributed to the program areas within the International Actions Theme? Were any actions taken as a result?

8. Are there any external factors outside the International Actions Theme that influence the success of its programs?

Issue 5: Demonstrated Efficiency and Economy9. Are there more economic and efficient means of achieving objectives under the

International Actions Theme?10. How could the efficiency of programs under the International Actions Theme be

improved?11. To what extent have each of the International Actions Theme program areas been

implemented, or are they on track to being implemented as planned and on time?12. Is the management and accountability structure for the International Actions Theme

in place and functioning adequately to achieve expected results? 13. Is appropriate performance data being collected, captured and safeguarded? If so, is

this information used to inform senior management / decision makers?

12

Page 30: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

3.2 Evaluation Approach and Methodology

The three methods used to collect data for this evaluation are described below.15

1. Document Review :

An in-depth review of all relevant documents provided by the departments engaged in the International Actions Theme was conducted, and further documents were requested where gaps were identified or where additional information was needed to supplement the evidence base. The project team reviewed each document to understand the mandated requirements, goals and objectives, governance structure, authorities, activities, outputs and outcomes for each component. The overall adequacy of the document evidence base (e.g., availability of performance information) was also assessed. All evidence from the documentation was recorded against the defined evaluation issues. This data collection method addressed evaluation questions #1–13.

2. Key - Informant Interviews :

Interviews were conducted with key informants identified by the departments involved in the CAA International Actions Theme, with suggested additions by the project team (e.g., external contacts, international experts). Interviews were either conducted in person (e.g., senior management, executives in the private sector) or by phone. Interviews provided information on the program components’ relevance and performance. Additional interviews were conducted as the evaluation progressed based on feedback from interviewees and information learned through the process. A total of 43 interviews were conducted. This data collection method addressed evaluation questions #1–13.

3. Focus Group of Experts :

A two-hour focus group session was held with five Canadian experts on climate change and air quality issues.16 The purpose of the focus group was to:

share and test preliminary findings and key messages emerging from the evaluation; and

collect additional information and expert advice on Canada’s international performance related primarily to climate change, and also to air quality.

This data collection method addressed evaluation questions #5–8.

Preliminary findings were validated by the appropriate program officers from each of the four components of the International Actions Theme in order to correct for omissions and errors before completion of the evaluation report.

3.3 Limitations

The evaluation focused on the completion of program activities using allocated resources from the fiscal years 2007–08 to 2009–10. As such, the evaluation focused on the achievement of outputs and intended immediate outcomes only, since data collection took

15 The key methodological instruments used for data collection are presented under separate cover in a technical appendix.16 External experts did not fulfill an advisory or oversight function to the evaluation. 

13

Page 31: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

place approximately one year before the end of the funding (presented in the program Logic Model in Figure 1). The achievement of these higher-level outcomes may be assessed in future evaluations, depending on future funding for activities under this theme.

The confidentiality of the UNFCCC negotiations, and the direction provided by Cabinet to the Canadian delegation, constrained the evaluators’ ability to investigate the extent to which success in securing Canada’s stated interests in the Convention can be attributed directly or indirectly to efforts by the Government. In particular, interviews with other international negotiators were limited as were interviews with representatives from industry17

and the non-governmental sector. These interviews would have provided an external perspective on the Theme’s performance and relative impact on advancing Canada’s positions in the UNFCCC as well as in other multilateral negotiation processes in which Canada seeks to advance its positions. As a result, the evaluation was limited in providing an independent perspective on the Theme’s performance, particularly how its negotiation activities influence the international negotiation process within and outside of the UNFCCC.

Ongoing performance data were not available for all program components, particularly financial information, which limited the demonstration of achievement of expected outcomes.

4 Findings by Evaluation Question

This section provides evaluation findings for each of the defined evaluation issues and questions. Findings are presented by evaluation issue, under which there is a summary of integrated findings across all four program components, followed by detailed findings for each program component.

4.1 Rating of Findings

Ratings have been provided to indicate the degree to which this theme and each component have addressed the key evaluation criteria. The rating is assessed according to the chart below, provided by EC’s Evaluation Division. A summary table of these ratings/findings is provided in Annex 2.

Table 3: Ratings LegendRating SignificanceAchieved The intended outcomes or goals have been achieved or metProgress Made; Attention Needed

Considerable progress has been made to meet the intended outcomes or goals, but attention is still needed

Little Progress; Priority for Attention

Little progress has been made to meet the intended outcomes or goals and attention is needed on a priority basis

N/A A rating is not applicable~ Outcomes achievement ratings are based solely on subjective evidence

17 Industry representatives beyond those related to the APP.

14

Page 32: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Canada US Air Quality Agreement Progress Report 2008

Exposure to PM has been associated with premature mortality as well as indices of morbidity, including respiratory hospital admissions and emergency department visits, school absences, lost work days, restricted activity days, effects on lung function and symptoms, morphological changes, and altered host defence mechanisms.

Recent epidemiologic studies have continued to report associations between short-term exposures to fine particles and effects such as premature mortality, hospital admissions or emergency department visits for cardiopulmonary diseases, increased respiratory symptoms, decreased lung function, and physiological changes or biomarkers for cardiac changes.

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.2 Evaluation Issue: Relevance – Continued Need

Issue 1: Continued Need for the Program

Indicators Overall Rating

1. Are program components within the International Actions Theme connected with key international climate change and air quality needs?

Evidence that International Actions Theme programs adapt with the emergence of relevant scientific evidence on air quality and climate change

Evidence that International Actions Theme programs adapt to changes in government policy and priorities

Evidence that the contribution program supports international objectives

Achieved

SUMMARY:The evaluation found that, across all program components, there remains a continued need for the programs that are part of the International Actions Theme in order to continue meeting key international climate change and air quality needs. There is evidence that the programs have adapted to changes in government policy and priorities and that contribution programs support international objectives. However, program components need to continue adapting, given the emergence of relevant scientific evidence on air quality and climate change. Due to the continued need for the programs, the overall rating has been assessed as “achieved.”

1. Are program components within the International Actions Theme connected with key international climate change and air quality needs?

1a Particulate Matter Annex The PM Annex component of the International Actions Theme clearly responds to and connects with key air quality needs in Canada.

The understanding of environmental and human health impacts of PM has grown significantly in recent years. Many studies now link PM, particularly in its finer fractions (PM10

and PM2.5), to a host of health impacts—including aggravated respiratory diseases such as bronchitis and asthma. PM can also damage vegetation and negatively affect visibility.18

In recognition of the transboundary flow of PM across the Canada-U.S. border, the Canada–U.S. Subcommittee on Scientific Cooperation completed the Canada–U.S. Particulate Matter Science Assessment in 2004. The Committee was directed to better understand, through a scientific assessment, the current knowledge of transboundary transport of PM and PM precursors between Canada and the United States. The results of this assessment noted the following:

18 Canada–-U.S Subcommittee on Scientific Co-operation (2004). Canada–-United States Transboundary Particulate Matter Science Assessment. Available at: http://www.ec.gc.ca/Publications/default.asp?lang=En&xml=3BC8288E-E427-4E0E-A624-F67334D31BB0. This report was completed in support of the Canada–-U.S. Air Quality Agreement.

15

Page 33: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

PM is recognized as an important health concern; high ambient levels of PM and its precursors are observed in North America, and in

Canada the areas with high levels include southwestern Ontario and the Georgia Basin – Puget Sound Region;

precursors of PM generally contribute to the acidification of ecosystems and are a significant cause of visibility impairment;

PM and its precursors can be transported long distances and are transported between Canada and the United States; and

reductions in SO2 are likely to result in reductions in PM, acid deposition, and visibility impairment.

Based on the evidence provided by this assessment, Canada and the United States committed to negotiating a PM Annex to the Canada–U.S. Air Quality Agreement. As the Agreement is aimed at reducing the cross-border flow of air pollution and its impact on the health and ecosystems of Canadians and Americans, both countries are committed to develop and implement emission reduction programs to reduce fine particle concentrations. Such an Annex will establish commitments by the United States to reduce its emissions of PM and its precursors, and to help Canada secure improvements in air quality and associated health and environmental benefits. A PM Annex would aim to ensure that both Canada and the United States develop commitments to reduce emissions of SO2 and NOx as well as PM, leading to a substantive decrease in transboundary flows (including mobile sources of PM19) of these pollutants. A PM Annex would also aim to ensure that there is a shared responsibility to monitor PM levels, conduct science, and assess progress toward fulfilling PM commitments. It is further anticipated that a PM Annex could provide a starting point for developing a cross-border SO2 and NOx trading regime.

Negotiations for a PM Annex began in 2007, but have not progressed due to evolving policy context in both countries. There is therefore a continued need to work toward and finalize the PM Annex. Rating: Achieved

1b International Partnerships and Negotiations

The International Partnerships and Negotiations component of the International Actions Theme clearly responds and is connected to international climate change needs.

In particular, this component reflects the growing prominence of climate change in a range of multilateral and bilateral international fora. For example, climate change has been a fixture on the agenda of G8 Leaders since 2005. This focus has prompted several new climate change–related initiatives, such as the Gleneagles Plan of Action and the U.S.-led MEF. Bilateral fora such as the Canada–European Union Partnership and theCED provide strategic opportunities for Canada to define and advance its climate change activities. In addition to ad hoc working groups of the UNFCCC, multiple UN bodies have addressed climate change in recent years, including the UN General Assembly and the UN

19 In this context, mobile sources are air pollution generated by vehicles, engines, equipment that move between Canada and the United States.

16

Page 34: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Commission on Sustainable Development. International deliberations in each of these fora have highlighted the importance of sustained international action on climate change.

The program component also reflects scientific evidence on the need to address climate change, as evidenced most recently by the 4th Assessment Report published by the Intergovernmental Panel on Climate Change (IPCC) in 2007. Among its findings, this report noted that warming of the climate system is unequivocal and that global atmospheric concentrations of GHGs have increased markedly since the Industrial Revolution, now far exceeding pre-industrial values.

During the summer of 2009, and in recognition of the IPCC’s 4th Assessment Report, leaders from the G8 and MEF recognized the scientific view that the increase in global average temperature above pre-industrial levels should not exceed 2 degrees Celsius. G8 leaders also affirmed their commitment to reduce global emissions by at least 50% by 2050. Canada’s long-term targets are consistent with this vision.20

This component of the International Actions Theme is clearly relevant and responsive to international climate change.21

Rating: Achieved

1c International Financial and Other Obligations

Evidence from this evaluation indicates that Canada is meeting international climate change and air quality needs through its ongoing payment of international financial obligations.

Canada is a party to the UNFCCC, the legal agreement that provides the international response to global climate change. The UNFCCC Secretariat supports all institutions involved in the climate change process, particularly the UNFCCC Conferences of the Parties (COP), the subsidiary bodies and the Bureau. As a Party to the UNFCCC, Canada is obligated to contribute its share of dedicated assessed funding and, in this way, contributes financially to the UNFCCC Secretariat in order to address international climate change and air quality needs. Evidence from this evaluation demonstrates that Canada is meeting this obligation.

Beyond the assessed funding, other important obligations include Canada’s commitments to establish a National Registry for Kyoto-recognized emissions credits, a condition for Canada’s participation in the flexibility mechanisms of the Protocol (i.e., the Clean Development Mechanism, Joint Implementation, and the ability to trade national emissions allowances). National registries are required of all Annex 1 country signatories to the Kyoto Protocol and are linked to an international database (the ITL) that is used to monitor each country’s holdings and transfers of Kyoto-recognized emissions credits. In addition to establishing the Registry and the ITL connection, evaluation evidence indicates that Canada makes supplementary contributions to develop user guidance for the ITL. These obligations directly support international efforts to facilitate monitoring and accountability of emissions

20 http://www.climatechange.gc.ca/default.asp?lang=En&n=5497F282-1 21 It should be noted that, while Canada’s current performance and stated medium-term goals are not fully consistent with international climate change needs as articulated by the IPCC, setting Canada’s reduction targets is not under the purview of this theme and is outside the scope of this evaluation.

17

Page 35: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

trading, and contribute to establishing a level playing field through consistent technical-user guidance for the ITL.

Within this component, Canada also makes voluntary contributions to support initiatives in which Canada has a specific interest. These include funding to enable Canada to participate in and contribute to international technology initiatives outside the UNFCCC, such as M2M and REEEP.

Supplementary G&C funds are also provided to the Organisation for Economic Cooperation and Development (OECD) Climate Change Expert Working Group, which provides a forum to work out technical details related to implementation of the UNFCCC and Kyoto Protocol.

Rating: Achieved

1d Asia–Pacific Partnership

The APP supports key international climate change and air quality needs, as the main objectives of the program are to:

promote the development, deployment and diffusion of existing and emerging cleaner, more efficient technologies and practices;

complement the UN climate processes to address development, energy and climate issues; and

build on existing bilateral and multilateral initiatives to enhance cooperation in order to meet increasing energy needs and the associated challenges of air pollution, energy security, and GHG intensities.

The APP brings together key major-emitting and major-economy countries to collectively address the issues of sustainable development, clean energy and climate change. As climate change is a global issue, cooperation among these countries is essential for achieving the long-term GHG reductions needed for a sustainable future. The APP partners (Australia, Canada, China, India, Japan, Korea and the United States) focus on expanding investment and trade in cleaner-energy technologies, goods and services in key market sectors. The seven partner countries collectively account for more than half of the world’s economy, population and energy use, and they produce about 65% of the world’s coal, 62% of the world’s cement, 52% of the world’s aluminum, and more than 60% of the world’s steel.22

The clean-technology projects developed and supported by the APP help reduce GHGs, and improve air quality in the partner countries. Information sharing between countries is also an objective. This exchange of technologies and expertise between countries provides for a more level playing field for the industries involved, helping to address the issue of competitiveness resulting from the disparate climate change / air quality regulations of the partner countries.

The APP is seen by the partner countries as a complementary process to the UN climate change process, supporting but not replacing the Kyoto Protocol.

Rating: Achieved

22 Vision Statement of the APP, found at http://www.asiapacificpartnership.org/pdf/resources/vision.pdf .

18

Page 36: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

2007 Speech from the Throne“Our government will also bring forward the first ever national air pollution regulations. In so doing, our government will put Canada at the forefront of clean technologies to reduce air pollution and greenhouse gas emissions.”

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.3 Evaluation Issue: Relevance – Alignment with Government Priorities

Issue 2: Alignment with Government Priorities

Indicators Overall Rating

2. Are program components within the International Actions Theme aligned with federal government priorities?

3. Are program components within the International Actions Theme aligned with the priorities of participating departments?

Extent to which strategic directions within International Actions Theme match federal government priorities

Achieved

SUMMARY:The Theme’s program components are clearly aligned with federal government priorities and the priorities of participating departments. Due to this strong alignment, the overall rating has been assessed as “achieved.”

2. Are program components within the International Actions Theme aligned with federal government priorities?

2a Particulate Matter Annex

The PM Annex is clearly aligned with federal government priorities.

The 2006 and 2007 Speeches from the Throne highlight that Canada (through domestic action) will take action to reduce pollution by regulating GHG and air pollutant emissions. Similarly, the federal Turning the Corner plan to reduce GHGs and air pollution as well as the Regulatory Framework for Air Emissions laid out the broad design of the regulations for industrial emissions of GHGs and air pollutants. These initiatives focused on reducing emissions from industry, and outline regulatory measures to reduce emissions from the transportation sector as well as consumer and commercial products, and actions to improve air quality.

Furthermore, the Government of Canada has made a concentrated effort to work with the United States to develop a North American position on a number of environmental issues in key fora, such as the negotiations under the UNFCCC, the MEF, the APP and the CED.

The PM Annex is aligned with the long-standing history between Canada and the United States on addressing transboundary air quality issues, dating to the signature of the Air Quality Agreement23 in 1991. The Agreement was originally established to address shared concerns regarding transboundary air pollution (SO2 and NOx) that leads to acid rain.

23 Available at ? Available at http://www.ijc.org/rel/agree/air.html..

19

Page 37: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

This program component is consistent with an agreement signed in April 1997 between the two countries signalling their intent to develop a Joint Plan of Action to more specifically address the problem of transboundary air pollution and smog. Ground-level ozone was identified as the first priority under this plan, and in December 2000 the Ozone Annex was added to the Air Quality Agreement. The Joint Plan of Action called for the development of a PM science assessment, to then serve as the basis for a decision on whether to proceed with an additional annex on PM.

As such, negotiating the PM Annex is aligned with the Government of Canada’s position to address air quality issues in Canada generally, and more specifically to reduce cross-border air pollution (including PM and its precursors) affecting human health and the environment in Canada and the United States.

Rating: Achieved

2b International Partnerships and Negotiations

This program component is strongly aligned with federal government priorities.

For example, the October 2007 Speech from the Throne noted that climate change is a global issue requiring a global solution, and reiterated Canada’s commitment to “press for a new international agreement that cuts global emissions in half by 2050.” The November 2008 Speech from the Throne reiterated the Government’s intention to “continue its realistic, responsible approach to addressing the challenge of climate change,” by working with its international partners and by reducing Canada’s total GHG emissions by 20% by 2020.24

This program component is also consistent with a September 2007 speech by Canada’s Prime Minister at the UN Secretary General’s High Level Event on Climate Change,25 outlining the federal government’s priorities related to climate change. The principles identified in this speech provided the basis for Canada’s priorities and negotiating positions, which are to pursue a new global consensus on a legally binding international climate change agreement that is environmentally effective, comprehensive, and inclusive of all major emitters. These priorities and positions include:

Canada’s commitment to reduce the country’s total emissions by 17% from 2005 levels and by 60–70% from 2006 levels by 2050;

the need for a long-term approach that provides the time needed to develop and deploy transformative technologies to achieve these goals; and

a series of principles26 underlying Canada’s approach that could assist in forging consensus around a new international framework for long-term action. These principles emphasize the following:

o Balancing environmental protection with economic growth: taking sustained action to reduce emissions while achieving low-carbon growth and sustainable development and not unduly burdening the growth of any single country.

24 This commitment was subsequently revised to reducing Canada’s total GHG emissions by 17% from 2005 levels by 2020 (see http://www.climatechange.gc.ca/default.asp?lang=En&n=D43918F1-1) so as to align with the target of the United States.25 The Prime Minister was an invited speaker at this international meeting, where he presented Canada’s commitments on climate change and clean technology (http://pm.gc.ca/eng/media.asp?id=1827). 26 The principles can be found at www.climatechange.gc.ca/default.asp?lang=En&n=5497F282-1.

20

Page 38: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

o Having a long-term focus: putting international policies in place to achieve deep emissions reductions over time, aiming to cut global emissions in half by 2050.

o Supporting the development and deployment of clean technologies: stimulating innovation and rapid and widespread deployment of low-carbon technologies through measures such as strategic technology initiatives and investments.

o Engaging, and seeking commitments from, all major economies: working toward emission reduction obligations for all countries (particularly the largest-emitting economies) that suit their unique circumstances.

o Supporting constructive and ambitious global actions: working with North American partners and all UNFCCC parties through the negotiation process to reach a fair, effective and comprehensive global climate agreement.

These principles also reflect Canada’s revised approach to: address climate change within the country as Canada seeks to modify and evolve the position initially reflected in Turning the Corner; adapt to the economic downturn; and align its domestic targets and work with the current U.S. Administration on a new long-term agreement to address international climate change.

Rating: Achieved

2c International Financial and Other Obligations

This program component is consistent with direction provided in Speeches from the Throne and Turning the Corner.

Specifically, obligations to establish the National Registry for emissions and support the ITL contribute to the Government of Canada’s national commitment to address climate change by providing an official UN-sanctioned forum where companies can trade Kyoto-supported emissions.

The technology partnership activities pursued within this program component (M2M and REEEP) demonstrate alignment with Government of Canada priorities outlined in Canada’s Innovation Strategy, which coordinates science and technology trade activities across federal departments. As a result of the Strategy, the Government of Canada increased focus and placed more emphasis on innovation, science and technology, as key elements of increasing Canada’s prosperity. The subsequent strategy, Mobilizing Science and Technology to Canada’s Advantage, set out a comprehensive, multi-year science and technology agenda for the nation. In 2008, the Government of Canada released Seizing Global Advantage: A Global Commerce Strategy for Securing Canada’s Growth and Prosperity, which outlines the Government’s international priorities for strengthening Canadian engagement in the global economy. These broad Government of Canada priorities correspond with the program activities undertaken within this program component, including: fostering technology partnerships abroad through M2M using Canadian climate change mitigation technologies; and participating and contributing to international capacity development programs related to climate change mitigation science and technology through REEEP. Other technology partnerships, including the APP, the Global Carbon Capture and

21

Page 39: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Storage Institute, the Carbon Sequestration Leadership Forum (CSLF) and the IEA, are noted in other program components where relevant.

Rating: Achieved

2d Asia–Pacific Partnership

The APP is well-aligned with federal government priorities and the federal mission of balancing environmental protection with economic development. The APP supports Canada’s domestic and international air quality, climate change and clean technology objectives, and is consistent with direction provided in multiple federal Throne Speeches (2008–10) and the Turning the Corner initiative. Several Canadian projects funded through the APP also support the objectives of reducing GHG emissions and ensuring clean air, water, land and energy while ensuring a strong economy.

The following are the priorities of the federal government (as outlined in the 2008–10 Speeches from the Throne) that align with the objectives and goals of the APP:

addressing climate change and preserving Canada’s environment by reducing GHG emissions and ensuring that 90% of our electricity needs are met by non-emitting sources;

working on global cooperation to advance the climate change agenda, which includes an agreement that includes all the world’s major GHG emitters, and to provide funding to help developing economies reduce their emissions and adapt to climate change;

enhancing energy security, which includes pursuing new cleaner-energy supplies and enhancing energy efficiency;

expanding investment and trade and diversifying opportunities for Canadian business, through bilateral and multilateral trade agreements;

stimulating the economy and supporting Canadian industries; and investing in clean energy technologies.

The federal government recognizes that reducing global emissions over the long term will require the development and deployment of clean, low-carbon technology by the world’s major emitters. Canadian participation in the APP supports Canadian priorities in clean energy research and development, such as large-scale carbon capture storage (CCS), renewable energy and energy efficiency. Activities in the APP are also aligned with the broader government priority to strengthen bilateral relationships (e.g., with the United States and China).

Rating: Achieved

3. Are program components within the International Actions Theme aligned with the priorities of participating departments?

22

Page 40: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

3a Particulate Matter Annex

This program component is clearly aligned with the priorities of EC.

EC leads all aspects of work completed under this component, which is consistent with the Department’s mandate, strategic outcomes and program activities. A PM Annex with commitments from Canada and the United States to improve air quality by reducing SO2 and NOx emissions and PM can lead to a substantive decrease in transboundary flows of these pollutants and an increase in health and environmental benefits for Canadians.

As described in subsection 4(1) of the Department of the Environment Act,27 and as presented on EC’s website, the Department’s mandate is to “preserve and enhance the quality of the natural environment, including water, air and soil quality; conserve Canada’s renewable resources, including migratory birds and other non-domestic flora and fauna; conserve and protect Canada’s water resources; carry out meteorology; enforce the rules made by the Canada - United States International Joint Commission relating to boundary waters; and coordinate environmental policies and programs for the federal government.”28

Furthermore, work under this component is directly aligned with EC’s third Strategic Outcome29: “Threats to Canadians and their environment from pollution and waste.are minimized” This Strategic Outcome is supported by the “threats to Canadians, their health and their environment from air pollutants and greenhouse gas emissions are minimized” program activity. As such, work to complete a PM Annex that includes commitments to minimize air pollutants and improve air quality is aligned with this outcome as it aims to reduce the potential health risks to Canadians caused by poor air quality.

Rating: Achieved

3b International Partnerships and Negotiations

This program component is strongly aligned with the mandates and priorities of all participating departments. In particular, advancing Canadian positions within the climate change negotiations assists EC in delivering its environmental protection mandate, while also enabling DFAIT to fulfill its role in advancing Canadian interests internationally. Ongoing participation in international climate change fora also supports NRCan in its efforts to promote Canadian interests internationally and to position Canada as a world-leading steward of our natural resources.

In addition, this program component is consistent with the departmental strategic outcomes, key program activities and operational priorities of each participating department (see Table 4 below).30

Table 4: Departmental PrioritiesParticipating Department

Relevant Strategic Outcome

Relevant Program Activity

Operational Priorities

EC Canadians and their Clean air program Reducing GHG

27 Available at http://laws.justice.gc.ca/eng/E-10/FramesView.html. 28 Available at http://www.ec.gc.ca/introec/mandate.htm. 29 As reported in Part III of the 2008–2009 Departmental Performance Report, available at www.tbs-sct.gc.ca/dpr-rmr/2008-2009/inst/doe/doe-eng.pdf.30 Although IC is not a formal member of the Canadian delegation, that departmentey participates in interdepartmental working groups and provides information on an as- required basis.

23

Page 41: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

environment are protected from the effects of pollution and waste

emissions domestically and through international agreements

NRCan Canada is a world leader in the environmentally responsible development and use of natural resources

Clean energy Address climate change and air quality through clean energy

DFAIT Canada’s international agenda

International policy advice and integration

United States and the western hemisphere (air agreements, climate change)

Rating: Achieved

3c International Financial and Other Obligations

The activities under International Financial and Other Obligations are aligned with DFAIT and EC departmental priorities.31

Within this component, DFAIT serves as the focal point for overseeing financial contributions to the UNFCCC in order to meet the assessed funding requirements, providing administrative support and management coordination for the provision of assessed funding on behalf of Canada. EC serves as technical lead on developing the National Registry for emissions credits, managing the connection to the UN-based interactive ITL, participating on behalf of Canada in technology partnerships (e.g., REEEP), and leading Canada’s technology trade initiatives through M2M.

DFAIT is responsible for Canada’s Innovation Strategy noted above (see question 3). DFAIT’s priorities include strengthening Canada’s missions abroad, leading a government-wide approach to formulating and implementing policies on foreign affairs and international trade as well as related programs, and promoting international trade and commerce.

The technology partnerships programming (e.g., M2M), led by EC, correspond with the Department’s mandate of being a science-based department with a priority to address risks to health and environment from air pollution, and to reduce GHG emissions through international agreements, as stated in the 2009–2010 Environment Canada Report on Plans and Priorities. The M2M program also draws on technology research undertaken by EC and NRCan through the Program for Energy Research and Development, which also demonstrates coordination across participating departments. However, data from interviews suggest that the renewable energy / energy efficiency capacity-development-programming membership (e.g., REEEP) might more closely align with the priorities of NRCan, given the clear departmental mandate in its Energy Sector and, in particular, NRCan’s specific research mandate with respect to renewable energy technology deployment. There is evidence of NRCan’s participation in energy technology partnerships through other activities within other program components of this theme, such as the EGTT, APP, CSLF and IEA.

31 Neither NRCan nor IC participate in this program component.

24

Page 42: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Rating: Achieved

3d Asia–Pacific Partnership

The evaluation found that the APP and its activities are appropriately aligned with the mandates and priorities of the participating federal departments (EC, NRCan, IC and DFAIT) as articulated in their Reports on Plans and Priorities and as evidenced by their efforts and cooperation to promote Canada’s participation in the APP.

Below is a summary of the relevant priorities by department:32

EC: There is a direct link to EC’s priorities for GHG and air pollutant priorities and to the “climate change and clean air” program activity. APP project development corresponds with EC’s mandate to address risks to health and environment from air pollution, and the goal of reducing GHG emissions through international agreements.

DFAIT: There is a link to DFAIT’s priorities to enhance energy security, help Canadian business with opportunities in growing and emerging markets (such as China and India), and strengthen bilateral relations with the United States. DFAIT’s priorities include strengthening Canada’s international platform, bolstering Canada’s presence abroad, and ensuring that missions overseas are in places that matter most to Canadian interests.

NRCan: There is a direct link to NRCan’s priorities of addressing climate change and air quality through science, technology and adaptation. NRCan participation in the APP supports priorities in clean energy research and development, such as large-scale CCS, renewable energy and energy efficiency.

IC: There is a link to IC’s priorities of integrating the economic, environmental and social interests of Canadians, and helping Canadian industry become more competitive in the global economy.

Rating: Achieved

32 Priorities are referenced in Reports on Plans and Priorities (2008–2010).

25

Page 43: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.4 Evaluation Issue: Relevance – Alignment with Federal Roles and Responsibilities

Issue 3: Alignment with Federal Roles and Responsibilities

Indicators Overall Rating

4. Is there a federal role and responsibility for the Government of Canada to deliver program components within the International Actions Theme?

Demonstrated evidence of federal role and responsibility in the area of international actions for climate change and air quality

Absence/presence of similar programs within the federal and/or provincial governments

Extent to which similar programs complement/duplicate one another

Achieved

SUMMARY: There is a clear federal role and responsibility for the Government of Canada to deliver this International Actions Theme, as the federal government has the responsibility to address environmental issues across international boundaries. Therefore, the overall rating has been assessed as “achieved.”

4. Is there a federal role and responsibility for the Government of Canada to deliver program components within the International Actions Theme?

4a Particulate Matter Annex

There is a clear federal role and responsibility to address environmental issues across international boundaries, recognizing that provinces and territories play an active role in managing PM across Canada.

Key federal areas of responsibility related to PM and air pollution generally include: working with foreign governments, including the United States, to reduce transboundary and international sources of air pollution; regulating substances declared toxic under the Canadian Environmental Protection Act, 1999; regulating contaminants in fuels and enacting motor vehicle emission standards; and maintaining the National Pollutant Release Inventory.

In managing PM across Canada, provinces and territories typically serve as permitting authorities (e.g., by issuing and monitoring Certificates of Approval) for emissions from point sources such as smelters, thermal power plants, and oil refineries. They also regulate municipal and hazardous waste incinerators and control provincial transportation, including inspection and maintenance of on-road vehicles.

Given the shared responsibilities for air quality management in Canada,33 federal, provincial and territorial governments coordinate through the Canadian Council of Ministers of the Environment (CCME) and its Air Management Committee. Through the CCME, federal, provincial and territorial governments except Quebec developed the 2000 Canada-wide

33 The federal government has constitutional power to regulate interprovincial and international air pollution, and can also regulate air pollution under its exclusive jurisdiction over the criminal law. It also has the power to negotiate and enter into treaties on Canada’s behalf (e.g., the 1991 Canada–U.S. Air Quality Agreement). Provinces have a very broad power to regulate pollution that occurs within provincial boundaries. As such, they are the primary regulators of industrial air pollutants. They also have the power to delegate, to municipal and regional governments, the responsibility for dealing with local pollution problems. www.cleanair.ca/law_jurisdiction.html

26

Page 44: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Standards for PM and Ozone,34 which commit all governments to significantly reduce PM and ground-level ozone by 2010.

The Government of Canada also participated in a multi-stakeholder process that included representatives of provincial governments, industry and non-governmental organizations to develop a proposal for a comprehensive air management system to address air pollution in Canada.35

Rating: Achieved

4b International Partnerships and Negotiations

There is clear federal jurisdiction for this program component. In its 2006 overview report on climate change, the Commissioner of the Environment and Sustainable Development (CESD) noted that “The federal government has jurisdiction over environmental issues that cross international and provincial boundaries. It addresses national concerns about the environment and negotiates, signs and ratifies international treaties on behalf of Canada.”36

At the same time, provinces and territories have a critical role to play in implementing international commitments related to climate change and other environmental issues. For example, they have jurisdiction over natural resources within their boundaries, issue permits specifying allowable emission levels by a variety of emission sources, and have responsibility for provincial transportation and municipalities.

As such, significant cooperation is required between federal, provincial and territorial levels of governments to ensure coordinated and effective strategies to address climate change both domestically and internationally.

Rating: Achieved

4c International Financial and Other Obligations

There is a clear federal role and responsibility to address environmental issues across international boundaries, in particular with respect to air quality and climate change. Key federal areas of responsibility supporting climate change include working with and supporting the UN to develop international climate change protocols and agreements, and supporting UNFCCC activities since 1992, when Canada ratified the Convention.

Under the Convention, Canada is legally required to:

gather and share information on GHG emissions, national policies and best practices;

launch national strategies for addressing GHG emissions and adapt to expected impacts, including the provision of financial and technological support to developing countries;

cooperate in preparing for adaptation to the impacts of climate change; and

34 Document can be found at ? Document can be found at http://www.ccme.ca/assets/pdf/pmozone_standard_e.pdf. .35 The work of the group was completed in April 2010.36 Page 38, 2006 Report of the Commissioner of the Environment and Sustainable Development. to the House of Commons

27

Page 45: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

fund its annual share of UNFCCC activities (each country’s “share” is “assessed” by the Conference of the Parties [COP] of the UNFCCC).

The federal government has jurisdiction to implement program activities within this component. As a party to the UNFCCC, Canada is legally obligated to provide financial support through assessed funding. This support is coordinated through EC and DFAIT.

DFAIT has a mandate to represent Canada on all foreign policy missions by serving as the key point of contact for the UNFCCC. DFAIT carries out its primary role, i.e., as Canada’s point of contact for fulfilling international obligations, through processing all payments related to Canada’s assessed contributions to the UNFCCC. With respect to climate change, DFAIT is responsible for engaging in multilateral discussions on climate change in various international fora, including the UNFCCC and its subsidiary bodies. DFAIT also participates in ongoing dialogues on climate change policies with key partner countries. In addition, that department serves as the Government of Canada’s Designated National Authority for the Clean Development Mechanism and as Focal Point for Joint Implementation projects.

EC has long been a contributor to the Government of Canada’s international environmental agenda by advancing and sharing science expertise, as well as through negotiations and policy dialogue in international fora. In this role, EC leads some of Canada’s activities related to international clean technology partnerships intended to reduce GHG emissions and improve air quality globally. Provincial participation in the technology partnerships activities was not identified, but there is a potential role for provincial involvement in these programs as a participant, not necessarily as a funding contributor (e.g., M2M).

Rating: Achieved

4d Asia–Pacific Partnership

The federal government has the lead role in the APP, as the Partnership’s activities comprise international actions to address transboundary/global environmental issues (which is clear federal jurisdiction). There is cooperation among federal departments to coordinate technical and policy work related to Canada’s participation in the APP's Task Forces and to develop a process to evaluate and fund projects that promote the development, diffusion and deployment of clean technologies.

A number of similar programs (e.g., the CED, ecoEnergy Initiatives [NRCan], M2M, activities of the IEA, and activities supported by various industry associations) support the development of clean technologies. These programs are seen by stakeholders as complementary to APP activities, as opposed to being duplicative.

Rating: Achieved

28

Page 46: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.5 Evaluation Issue: Performance – Achievement of Expected Outcomes

Issue 4: Achievement of Expected Outcomes

Indicators Overall Rating

5. To what extent have intended outputs been achieved as a result of programs within the International Actions Theme?

Evidence of achievement of outputs as noted in the Logic Model and specific indicators noted in section 4.1

Progress Made; Attention Needed

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

Evidence of achievement of outcomes as noted in the Logic Model and specific indicators noted in section 4.1

7. Have there been any unintended (positive or negative) outcomes that can be attributed to the program areas within the International Actions Theme? Were any actions taken as a result?

Presence/absence of unintended outcomes

Where appropriate, documented management actions and/or lessons learned from unintended outcomes

8. Are there any external factors outside the International Actions Theme that influence the success of its programs?

Evidence of factors outside the International Actions Theme that have influenced the achievement of outputs and early outcomes

Where appropriate, documented management actions to address the influence of external factors

SUMMARY:While the majority of the intended outputs have been achieved across component areas, there have been some significant delays and the PM Annex work has not proceeded as planned. Although it is still early in the implementation phase, some progress has been made in achieving the immediate outcomes and no significant unintended outcomes have been identified. However, external factors have influenced the success of programs in this theme, particularly the increasing prominence of the climate change file, changes in policy direction and positions, and staff and Ministerial changes.

Due to the delays experienced and lack of progress in some component areas, the overall rating has been assessed as “Progress Made; Attention Needed.”

5. To what extent have intended outputs been achieved as a result of programs within the International Actions Theme?

5a Particulate Matter Annex

There are two expected outputs for the PM Annex: Partnerships and processes, and formal and informal consultations with domestic stakeholders. Overall, some progress was made toward achieving the expected outputs, with evidence indicating that activities completed to date have focused more on developing partnerships and processes with the United States to establish the PM Annex, and less on conducting formal or informal consultations with domestic stakeholders. Below is a summary of how this program component has achieved these expected outputs.

29

Page 47: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

A) Partnerships and processes outside the UNFCCCThe evaluation found evidence that a number of partnership and process activities (outside the UNFCCC) have been completed to design and develop the PM Annex, primarily in 2007 and 2008. During that time, Canada–U.S. negotiations toward the PM Annex were launched pursuant to a Statement of Intent announced in April 2007. The Statement marked the beginning of negotiations to develop the PM Annex, and EC officials emphasized the importance of this milestone as it underlines the need to address transboundary PM flowing from the United States into Canada. Three working groups were established to address mobile sources, monitoring and reporting, and the PM Annex Purpose Statement, and to complete work in preparation for the negotiation sessions.

Two negotiating sessions were held: one in November 2007 and one in May 2008. These provided an opportunity for both countries to collectively draft several elements of the Annex, including its:

Purpose Statement; pollutants of concern (i.e., the Annex is to address primary PM2.5 and two secondary

precursors, SO2 and NOx); definitions of the PEMAs for both countries (Canada indicated that its PEMA would

consist of Canada as a whole, while the U.S. PEMA would comprise 28 states across the northern part of the country); and

mobile-source commitments that could be included as part of the emission reduction commitments, given that transportation is a major contributor to PM.

The evaluation also found evidence highlighting that the work completed to date has been largely internal and has focused on preparing materials for the Canadian delegation in advance of the negotiating sessions. As such, the evaluation found evidence to confirm that work was completed to develop Canada’s negotiating mandate (e.g., discussion document identifying what elements should be included under the PM Annex, multiple conference calls between EC headquarters and regional offices / other government departments). Ongoing contact was also maintained between EC and the U.S. Environmental Protection Agency (EPA).

Currently, program activities are limited to sharing monitoring and reporting information about stationary sources and other condensable PM emissions, and ongoing collaboration on visibility issues.

B) Formal and informal consultations with domestic stakeholdersOutside of the activities to prepare Canada during the negotiation sessions, the evaluation found that the program components within the International Actions Theme have made contributions to further the understanding of air quality issues at the regional level. For example, from 2007 to 2009 the British Columbia Lung Association received an annual contribution to support activities to further understand visibility issues in British Columbia. These activities included workshops in 2008 and 2010 (with representatives from EC, the EPA, the B.C. Ministry of Environment, Metro Vancouver and the Fraser Valley Regional District) to discuss historical and current visibility work in Canada and the U.S. Similarly, program resources have been allocated to the EC regional offices to complete a series of

30

Page 48: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

pilot studies on visibility in western Canada and, more broadly, ongoing work to collect information about ambient air emissions and their associated health effects.37

The evaluation did not find evidence that the provinces, territories or other domestic stakeholders were consulted directly on the process to design the PM Annex. Program staff indicated that although specific consultations were not held, other processes, such as the CCME Canada-wide Standards for PM and Ozone, were used to keep Canadians apprised of the PM Annex negotiations.

Rating: Little Progress; Priority for Attention

5b International Partnerships and Negotiations

Intended outputs (discussed below) associated with the International Partnerships and Negotiations component have been largely achieved.

A) Contribution to international bodiesThere is strong demonstrated evidence of federal government participation in all major negotiating sessions conducted pursuant to the UNFCCC, including the three most recent meetings of the Conferences of the Parties to the UNFCCC (COP 13 in 2007 in Bali, COP 14 in 2008 in Poznan, and COP 15 in 2009 in Copenhagen). The Government of Canada also participated in the Ad Hoc Working Group on Further Commitments for Annex I Parties under the Kyoto Protocol (AWG-KP) and the Ad Hoc Working Group on Long-term Cooperative Action under the Convention (AWG-LCA), as well as meetings of other subsidiary bodies to the UNFCCC.38

There is also demonstrated evidence of ongoing participation by Canada in a number of expert groups established in support of the UNFCCC and its parties. For example:

Canada (represented by NRCan) is the elected representative for the developed countries and one of eight Annex I Party members of the 19-member globally represented EGTT, which supports and advances technology transfer activities. Canada is perceived to be an active and valuable contributor to these efforts.

Canada (represented by NRCan) most often leads the Umbrella Group’s Technology Negotiating Working Group39 during negotiations and between sessions.

Canada (represented by DFAIT) is one of three developed-country members of the Least Developed Countries Expert Group (LEG), which provides advice to least-developed countries on the preparation and implementation of national adaptation plans.

Canada (through DFAIT) has a seat within the UNFCCC’s Consultative Group of Experts on National Communications, which has a mandate to help improve the preparation of national communications required from developing countries under the Convention.

37 Documentation to confirm the completion of scientific activities funded through the International Theme was not submitted despite several requests from the evaluators. Evidence presented in this report represents the views of program staff and includes references found in summary documents and briefing materials. Scientific reports were not submitted. 38 Two permanent subsidiary bodies have been created by the Convention: the Subsidiary Body for Implementation and the Subsidiary Body for Scientific and Technological Advice. Additional subsidiary bodies may be established as needed.39 The Umbrella Group can be defined as: a loose coalition of non–European Union-EU developed countries, typically including Australia, Iceland, Japan, New Zealand, Norway, the Russian Federation, the Ukraine, and the United States.

31

Page 49: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

EC also participates in the OECD Climate Change Expert Group (formerly the OECD/IEA Annex I Expert Group), which is, globally, the key intergovernmental forum for technical discussion of long-term climate change issues. It is an ad hoc group of government officials from environment, energy, and foreign affairs ministries in countries that are listed in Annex I to the UNFCCC and countries that have acceded to Annex I commitments.40 Canada’s involvement in the Group includes participation in meetings and seminars, reviews of analytic papers relevant to the ongoing UNFCCC negotiations, input into the Group’s work, and presentations on relevant issues.

In addition, there is evidence of Canadian participation in a range of formal and informal groups within the negotiations, such as contact groups and expert meetings hosted by the UNFCCC. Canadian officials have also participated in a range of workshops and events relevant to the negotiations, such as:

a March 2007 UNFCCC workshop on reducing emissions from deforestation in developing countries;

a September 2007 UNFCCC workshop on adaptation planning and practices under the Nairobi work program on impacts, vulnerability and adaptation to climate change; and

a June 2008 workshop on methodological issues relating to reducing emissions from deforestation and forest degradation in developing countries.

NRCan has also built strong partnerships with organizations such as the IEA, the World Bank and the World Business Council for Sustainable Development, in an effort to advance thinking and discussions in areas such as technology transfer, innovative financing and intellectual property. As well, NRCan has provided expertise to the IPCC mitigation and adaptation working groups, contributing to the development of the IPCC’s 4th Assessment Report; and NRCan continues to work with the IPCC on the development of the 5th Assessment Report.

In some instances, Canada has either chaired or hosted discussions. For example, DFAIT convened a Chatham House Rule workshop on adaptation on behalf of the Government of Canada in 2009. DFAIT also participates in the Clean Development Mechanism Executive Board and Joint Implementation Steering Committee established pursuant to the Kyoto Protocol. Additionally, NRCan has advanced Canadian objectives and supported strategic direction by drafting text that resulted in all-party agreement on the Poznan Strategic Program on Technology Transfer, a key outcome of COP 14 and a key technology policy objective for Canada.

The evaluators also found evidence that Canada submitted required country-submissions to the UNFCCC, including national communications (submitted in March 2007 and February 2010) and national inventory reports (submitted in 2008 and 2009).

B) Policy and research papersThe development of briefing notes, backgrounders, and proposed position papers is a core activity undertaken under this program component. As such, there is extensive evidence of participating departments providing the Canadian delegation with input and advice on all key elements of the negotiations for relevant meetings. 40 Annex I countries include most OECD member states, and some countries from central and eastern Europe and the Commonwealth of Independent States that are undergoing the transition to a market economy. These are the primary countries participating in UNFCCC negotiations and national climate change policy development.

32

Page 50: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

DFAIT provided the delegation with input and advice on adaptation, developing country issues, non–Annex I national communications, and analytical work on the broader implications of climate change regarding human rights. DFAIT provided a table listing 40 briefing notes and backgrounders prepared on adaptation issues, and a listing of almost 200 briefing notes for senior officials. Similarly, NRCan provided over 100 documents as evidence of activity related to this program function, including: briefing notes and reports to inform Canada’s position on key topics such as technology and trade, CCS and clean technology; reports and analysis of key countries’ activities and negotiating positions; and summary reports to brief NRCan senior management and executives on Canada’s negotiating position and activities at UNFCCC meetings and other international fora. Additionally, EC provided over 80 documents indicating that instructions and scenario notes are prepared for the Canadian delegation in advance of UNFCCC and other international meetings, and that summary reports are completed to document and report to EC senior management on the delegation’s ability to achieve its objectives as well as highlight the key results and outcomes from the meetings.

DFAIT, NRCan and EC have also provided ongoing advice on bilateral engagement efforts, including how climate change affects bilateral foreign relations and foreign policy priorities, particularly with G8+5 countries. In addition, NRCan provided a submission to the UNFCCC review of technology commitments under the Convention (Articles 4.2 (c) and 4.5) and two submissions on the use of CCS as a Clean Development Mechanism project activity. These submissions explain Canada’s experience with developing, deploying and transferring climate-friendly technologies, providing recommendations and best practices that could be adapted to suit the national circumstances of other countries and position CCS as a measure to reduce global GHGs. NRCan also provides substantive advice on international climate change actions, particularly on issues relating to clean energy technology (including technology financing and intellectual property issues), adaptation practices, and forestry issues (i.e., deforestation in developing countries).

DFAIT, EC and NRCan have provided inputs and briefing notes for leader-level discussions on climate change, such as those conducted under the G8, as well as other multilateral fora such as the MEF. EC and DFAIT provide briefings materials for bilateral meetings conducted by the departments’ ministers and senior officials.

C) Negotiations under the UNFCCCThere is strong evidence of instruction documents, speaking points and associated briefing materials for major and inter-sessional international negotiations. In addition, the Government of Canada has established clear negotiating positions on all key topics under the UNFCCC (e.g., adaptation, deforestation, clean technology, etc.) as well as clear principles and positions related to the overall shape and content of a new long-term agreement on climate change.

Interdepartmental working groups have been established to share knowledge and analyze information to inform Canada’s negotiation position at the UNFCCC and in other multilateral fora. For example, as the lead for the interdepartmental Technology Policy and Analysis Working Group, NRCan provided leadership on energy- and technology-related issues and undertook policy analysis and research exercises, which directly supported the Chief

33

Page 51: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Negotiator and Ambassador for Climate Change, and the Government of Canada’s policy development and positioning on international climate change. Similarly, DFAIT coordinates the interdepartmental Working Group on Adaptation to facilitate analysis and develop positions on adaptation issues.

In addition, there is evidence that negotiators prepared summary reports for all major negotiations for use within the federal system, as well as analysis and associated work plans related to subsequent activities.

D) Partnerships and processes outside the UNFCCCThe evaluators found evidence of active Canadian participation in a number of multilateral meetings and partnerships outside of the UNFCCC. Canadian participation has focused on advancing the key principles and elements of its position related to a new long-term international agreement on climate change, and building support for these positions with potential allies.

Examples of this participation include evidence of:

participation in the Major Economies Process (2007, 2008) aimed at advancing emission reduction commitments by developing-country emitters;

sustained, high-level participation in the MEF (established in 2009), including ministerial and ambassadorial participation in key meetings and consistent, engaged participation by technical staff;

participation in G8 deliberations related to climate change (2007–2009), including sustained efforts to ensure relevant statements reflect Canadian interests and priorities; and

participation in a wide range of multilateral fora, including the Commonwealth Heads of Government, Asia–Pacific Economic Cooperation (APEC), Canada–European Union Summit, and Gleneagles Dialogue.

Additional evidence demonstrates that Canada has worked bilaterally with key emitting countries such as China and Mexico to formalize working relationships and identify opportunities for implementing new and clean technologies, in part through the Canada–China Working Group for Climate Change and the Canada–Mexico Partnership.

Canada and the United States established the CED in February 2009 to enhance joint collaboration on the development of clean energy science and technologies to reduce GHGs and combat climate change. Three joint working groups under the initiative are working on key issues, including: developing and deploying clean energy technologies, with a focus on CCS; expanding clean energy research and development; and building a more efficient electricity grid based on clean and renewable generation.41

As well, NRCan has played a leadership role in promoting the benefits of CCS. Canada leverages knowledge gained through research and development as well as domestic demonstration projects in order to internationally disseminate lessons learned, primarily via the IEA, CSLF and Global Institute on Carbon Capture and Storage. NRCan policy and technical experts engage in these multilateral fora to advance clean energy technology objectives, through such means as chairing the CSLF Strategic Planning Committee and representing Canada on the CSLF Capacity Building Governing Council. Canada’s

41 The CED is being evaluated in the summer of 2010, and is outside the scope of this International Actions Theme evaluation.

34

Page 52: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

proportionately large share of demonstration projects also contributes to the G8 goal of launching 20 CCS demonstration projects by 2010. Additionally, Canada (represented by NRCan) is a founding member of the Global Institute on Carbon Capture and Storage. NRCan has used its CCS technical and policy expertise to advance Canada’s objectives on the treatment of CCS under the Clean Development Mechanism in the UNFCCC negotiations. Moreover, participating in CCS-related fora has contributed to information sharing and learning, advancing Canada’s interests in the development and deployment of CCS.

E) Formal and informal consultations with domestic stakeholdersThe evaluators found evidence of regular and sustained engagement with provinces and territories, particularly in the year leading up to COP 15 in Copenhagen. For example, three meetings (chaired by Canada’s Chief Negotiator and Ambassador for Climate Change) with ADM- and DG-level officials from the provinces and territories occurred prior to COP15 as were 11 conference calls and more than 30 electronic exchanges of documents and relevant information.

In addition, one ministerial meeting was held prior to COP 15, with an additional session for participating ministers held in Copenhagen. These meetings were supplemented with bilateral ministerial meetings with Ministers of the Environment, and Premiers in some cases, from each of the provinces and territories.

Provinces and territories were also invited to participate in the Canadian delegation to the Copenhagen meeting, and were provided with instruction telexes and relevant briefing materials. As well, the Canadian delegation offered two daily meetings for provinces and territories in Copenhagen in addition to broader, daily stakeholder briefings.

Evidence was found of regular engagement with Aboriginal groups (e.g., representatives from the Assembly of First Nations, Métis National Council, Inuit Circumpolar Council), including three meetings, four conference calls and 10 electronic exchanges of documents and relevant information in the lead-up to Copenhagen.

Evidence of a two-tiered approach to civil society engagement was found. Evaluators were provided with a record of 15 engagements with environmental NGOs (e.g., the Pembina Institute), industry associations, the Federation of Canadian Municipalities, and NGOs (e.g., UNICEF) between March and December 2009. Multiple methods were used, including in-person interviews, teleconferences and video conferences.

In addition, the International Institute for Sustainable Development, with Government of Canada funding, facilitated three public policy dialogue sessions on issues relevant to the negotiations prior to COP 15. These sessions included information-sharing on key issues, as well as dialogue with the Chief Negotiator and Ambassador for Climate Change on issues of concern.

Rating: Achieved

35

Page 53: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

5c International Financial and Other Obligations

The intended outputs have been achieved as a result of programs within the International Actions Theme. However, there is some room for improvement with respect to the timeliness of achievement (see B below).

The following outputs were expected as a result of International Financial and Other Obligations:

A) Contributions to international bodies EC and DFAIT fulfilled their obligations to provide assessed funding to the UNFCCC in the four-year period of implementation of the International Actions Theme ($477,000 per year). EC’s non-assessed G&C funds ($323,000 per year) were allocated to REEEP and M2M. In addition, EC accessed other departmental funds to make payments to the UNFCCC ITL and OECD Climate Change Expert Group. This demonstrates a modest level of support, and moderate potential for non-assessed contributions to enhance Canada’s position abroad. Although the funding level is modest, M2M demonstrated potential to leverage funds from the United States and from private sectors in Canada and abroad. There was one project implemented in the M2M program during this time period (in China), and two others are in the planning stages (in Mexico).

There has not been an increase in the number of non-assessed EC G&C recipients. However, the design of the funding program for M2M and REEEP did not plan for increased recipients. The target recipients for EC G&C funds were strategically selected at the start of the four-year period, with annual contributions to support Canada’s strategic interests. EC’s contribution program was not designed to solicit new proposals from new recipients.

B) National Registry for carbon dioxide (CO2) emission unitsEC did fulfill international obligations to establish a National Registry for GHGs. The Registry was initialized with the ITL in June 2008. Canada was approximately one year late in fulfilling this obligation, due to the procurement contracting process with Public Works and Government Services Canada and the Treasury Board’s Common Look and Feel requirements for Government of Canada websites. The Registry was connected “live” to the ITL in February 2010. Canada’s obligations under the Kyoto Protocol do not include a requirement for public accounts in the Registry. As such, this was funded under a separate request.

Rating: Progress Made; Attention Needed

5d Asia–Pacific Partnership

The evaluation has found evidence that all the intended outputs for the APP as outlined in the CAA Logic Model have been achieved, although there were significant delays in achieving some outputs in the first two years (such as clean technology agreements under the APP not being completed).42

The following outputs were expected as a result of the APP:

A) Consultations with domestic stakeholders42 The reasons for the delay in achieving outputs are identified under Question 11.

36

Page 54: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Consultations with domestic stakeholders regarding Canada’s involvement in the APP were held prior to Canada joining the APP in October 2007. Consultations were held with industry groups such as the Industry Committee on Climate Change, Canadian Association of Petroleum Producers, Canadian Gas Association, Ford Canada, Imperial Oil, Quebec Business Council on the Environment, Cement Association of Canada, Aluminum Association of Canada, Canadian Steel Producers Association, and associations representing chemicals and electricity. Industry groups were largely in favour of Canada joining the APP but sought to better understand the links between APP and industry compliance considering obligations under CARA (i.e., whether the APP would be a type of credit mechanism that would help companies meet their domestic obligations).

B) Partnerships and processes outside the UNFCCCCreation of governance and organizational structures: Canada established APP operations, including creation of an APP Secretariat to coordinate Canada’s APP activities at the national and international levels, and a DG-level Interdepartmental Steering Committee (EC, NRCan, IC and DFAIT) to oversee Canada’s involvement in the APP. The Minister of the Environment was given overall responsibility for the APP, with the APP Secretariat housed within EC’s International Affairs Branch.

Creation of working groups: The APP Secretariat completed a consultation process with stakeholders, which led to the establishment of public/private domestic working groups for all eight APP Task Forces. Task Force Working Groups are chaired by departmental task force leads. Canada’s APP Secretariat engaged all Canadian industrial sectors covered by the APP Task Forces. This engagement covered business and professional associations, their members, and individual companies that expressed an interest in the APP, including, for example, the Cement Association of Canada, the Aluminum Association of Canada, the Canadian Steel Producers Association, the Net-Zero Energy Housing Coalition, and industry associations representing renewable energy sectors (wind, solar, etc.). Private sector participants in Canada’s APP activities have generally funded their own participation, including overseas travel.

Evidence of clean technology: Canada did not fund any APP projects during the first two years (2007–08 and 2008–09). However, Canada participated in and contributed to a number of existing projects, including the Management of Perfluorocarbon Emissions Project through the Aluminum Task Force, and the Indicators for Energy Saving project and State-of-the-Art Clean Technology Handbook project through the Steel Task Force. Canada announced the first round of approved projects in the summer of 2009, and the second round of approved projects in December 2009. Canada has now provided approximately $12 million for 28 “clean tech projects” that are currently being developed. Approximately $870,000 remains to be allocated to the last round of projects in 2010–11.

Rating: Achieved

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

6a Particulate Matter Annex

37

Page 55: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

The PM Annex is expected to achieve one immediate outcome: increased consideration of domestic stakeholder interests in the development of Canada’s positions. As part of their processes, the program components within the International Actions Theme consider domestic interests and concerns in order to inform Canada’s positions for negotiations with other countries. There is limited evidence to confirm that the PM Annex has achieved this expected immediate outcome.

A) Increased consideration of domestic stakeholder interests The evaluation was unable to confirm that domestic stakeholder interests have been reflected in Canada’s positions at the negotiation sessions completed in 2007 and 2008. As noted for the output on consultations with domestic stakeholders, the evaluation did not find any evidence to indicate that public engagements or consultations about the PM Annex were held with Canadians. Although program staff stated that CAA themes with a mandate to address domestic activities (e.g., clean air regulations and clean energy) and other national processes (e.g., CCME Canada-wide Standards for PM and Ozone) were used to consult with and inform Canadians about the PM Annex, no evidence was found concerning how these fora were used to collect information and develop Canada’s position.

Rating: Little Progress; Priority for Attention

6b International Partnerships and Negotiations

While some progress has been made in this area, the immediate outcomes associated with this program component, discussed below, have not been fully achieved to date.

A) Increased consideration of domestic stakeholder interestsInterviewed officials stated that participating departments were generally aware of stakeholder interests, with knowledge of their positions and main concerns regarding key issues in the negotiations.

Evaluators heard from interviewees that there was good technical engagement with provinces on issues related to Land Use, Land Use Change and Forestry (LULUCF), and that perspectives provided by provinces were incorporated into the ultimate position taken by Canada on this issue. In addition, there is evidence that, in Copenhagen, provinces and territories were granted greater access at more senior levels to negotiating instructions and the negotiations themselves than at previous Conferences of the Parties.43 For example, the federal Minister of the Environment invited all premiers to attend in Copenhagen, and provincial and territorial officials were invited to participate fully in the Canadian delegation. Ongoing consultations were also held with provincial and territorial representatives in the lead-up to Copenhagen.

While these efforts were clearly appreciated by those involved, some provincial partners and external observers felt that these activities represented opportunities for information sharing rather than providing direct input into the development of Canadian negotiating positions. As well, in interviews and focus group discussions, external observers suggested that consultations with NGOs regarding UNFCCC negotiations were inadequate in the lead-up to COP 15. These consultation mechanisms were perceived as focusing on informing the NGO community about Canada’s negotiation position as described by the Government of

43 Provinces were also included as members of the delegation in Poznan.

38

Page 56: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Canada, and did not provide a forum for exchanging information in support of developing a position that reflects the issues, concerns and opinions of the NGO community. This evidence suggests that this expected outcome was not fully achieved.

B) Canada’s positions are advancedWith respect to the advancement of Canadian interests within the climate change negotiations and other relevant international and multilateral fora, the evaluators found evidence of submissions made to the UNFCCC and key subsidiary bodies (including the AWG-KP and AWG-LCA) on issues of strategic concern to Canada. For example, submissions to the AWG-LCA included:

inputs to the negotiating text for consideration at the 6th Session of the AWG-LCA (submitted April 2009);

ideas and proposals on paragraph 1 of the Bali Action Plan (December 2008); views regarding the work program for the AWG-LCA (March 2008); and several joint submissions with Australia, Belarus, the European Community and its

member states, Iceland, Japan, New Zealand, Norway, the Russian Federation, Switzerland and Ukraine on information relating to possible quantified emissions limitation and reduction objectives.

Submissions to the AWG-KP included:

information and data on the mitigation potential of policies, measures and technologies (August 2007) and CCS; and

a series of 2008 and 2009 submissions on the treatment of LULUCF, including information on options for LULUCF accounting, and Reducing Emissions from Deforestation and Forest Degradation in Developing Countries.

In many cases, these submissions were provided directly by Canada. In other cases, they were submitted jointly with some of Canada’s allies, such as Australia and Japan. This provides evidence of Canadian coordination to advance common positions with various other parties to the negotiations, primarily through the Umbrella Group. Bilateral exchanges were also used for this purpose.

The evaluators also found evidence of Canada promoting its interests directly through concerted involvement in UNFCCC discussions related to the post-2012 international framework for action. This included direct interventions, as well as active participation in relevant contact groups and other informal discussions on key issues such as technology transfer, CCS, and more broadly on the scope and breadth of a new long-term agreement to address climate change. For example, Canada outlined its strategic interests through its National Statement at COP 15. Minister Prentice stated, in part:

We have come to Copenhagen to secure a fair, effective and comprehensive climate change agreement. We need an agreement that will put us on a path toward ambitious reductions in greenhouse gases and sustainable, low-carbon economic growth. An agreement that will ensure a growing supply of clean, affordable energy for all countries. An agreement that brings countries together to address our shared global economic and environmental challenges.…

39

Page 57: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Canada’s broad based actions to address climate change take into account our large diverse land mass, our growing population and the importance of our energy sector for meeting global demand. Our approach also reflects the strong economic ties between Canada and our neighbour, the United States, in the need to ensure our actions are aligned with our continental partners…

A new global agreement should consist of a single, comprehensive undertaking that includes measurable, reportable and verifiable commitments and actions covering the vast majority of greenhouse gas emissions in developed and developing countries. To be as fair and effective as possible, a new global agreement should support mutual confidence, and encourage countries to assume increased ambition over time. It needs to speed the development and deployment of clean, low-emitting technology. And it needs to support enhanced global action to help the poorest and most vulnerable countries.…44

According to interviewees, Canada has made a particular contribution (both in terms of financial support and with respect to issues of substance) to several of the UNFCCC expert groups, namely the EGTT and LEG. For example, NRCan has played a leadership role in facilitating developing country participation, advancing innovative financing objectives, and encouraging the role of the private sector to meet long-term financial needs to address global climate change. For instance, as a result of the partnerships that NRCan has built with key organizations such the World Bank and the World Business Council for Sustainable Development, representatives of these organizations are now meeting with the EGTT and negotiators, including Canada, to speak with private sector interests about advancing action on climate change, particularly with respect to the development and deployment of clean technologies.

The evaluators also found strong evidence of work to promote Canada’s interests at a range of international meetings addressing the post-2012 climate change framework. For example, federal officials participated in efforts to craft leaders’ statements for meetings of the G8, Commonwealth Heads of Government, and MEF. The statements emerging from these fora are consistent with Canadian interests as defined in federal negotiating instructions, and specific Canadian positions are reflected in a number of these statements.

For example, the July 2008 G8 Statement on Climate Change and the Environment confirms the significance of the IPCC’s 4th Assessment Report, and outlines a goal of achieving “at least a 50% reduction of global emissions by 2050, recognizing that this global challenge can only be met by a global response, in particular, by the contributions from all major economies, consistent with the principle of common but differentiated responsibilities and respective capabilities.” The statement also underscores that, “in order to ensure an effective and ambitious global post-2012 climate regime, all major economies will need to commit to meaningful mitigation actions to be bound in the international agreement to be negotiated by the end of 2009.”45

Canadian officials were active in advancing positions through strategic bilateral engagements at the leader, Ministerial, and officials levels. Canadian positions were advanced by the Prime Minister in bilateral meetings with priority countries such as China and Mexico, and were incorporated into or strengthened through key bilateral partnerships

44 From Canada’s National Statement at COP 15, December 18, 2009. 45 From G-8 Statement on Climate Change and Environment, paragraphs 23 and 24, July 2008.

40

Page 58: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

such as the Canada–China Climate Change Working Group and the Canada–Mexico Partnership (Environment and Forestry Working Group). Both of these bilateral partnerships have resulted in agreed-upon work plans, and projects initiated through operations and maintenance funding, APP partnerships, REEEP and/or M2M. Additionally, program staff highlighted that these initiatives increase awareness of trade opportunities in climate-friendly technologies between Canada and other participating countries.

The Government also signed a Memorandum of Understanding on climate change with China in December 2009 to strengthen Canada–China cooperation in areas such as energy conservation and efficiency, renewable energy, CCS, methane recovery and utilization, and sustainable land management.

The evaluators found evidence of bilateral discussions at different points in the negotiations with a range of developed and developing countries, such as Germany, the United Kingdom, Indonesia, South Africa, and Poland. Furthermore, through DFAIT, Canada’s network of missions and embassies was used to inform and support these meetings.

With respect to the outcomes of COP 15, the Copenhagen Accord does reflect some of the main themes of Canada’s position: it calls for a global solution toward significant long-term emission reductions, including commitments by both developed and developing countries; there was progress toward the concept of measurable, reportable and verifiable actions by developing countries; and it reflects a commitment to financing, particularly for least developed countries, in both the short- and long-term.

At the same time, not all of Canada’s stated interests are reflected within the Accord. For example, there are still two tracks of discussion under way (i.e., under the Kyoto Protocol and the UNFCCC), rather than the “single comprehensive undertaking” promoted by Canadian officials.

The Accord was not adopted by the COP as a whole. Rather, the COP “took note” of the Accord in a COP decision, which was developed by approximately 25 countries who served as “friends of the COP President.” Without adoption by the COP, the Accord has no clear formal or legally binding status, and the precise path forward toward refining and operationalizing it remains unclear. It also opens the door to multiple interpretations on key elements of the agreement by various countries and/or factions. Canada was not a member of this group, although several of its allies and partners through the Umbrella Group (including the United States) participated in its development.

It was difficult for the evaluators to ascribe the level of influence exerted by Canada in securing the objectives of either COP 15 or preceding COPs. There was no consensus among those interviewed on Canada’s role in securing the outcomes described, with program staff and senior management describing the delegation as having a much greater level of influence than that perceived by outside observers (including those interviewed and those participating in the focus group).

A number of external interviewees noted that Canada’s domestic performance in reducing emissions, and the fact that Canada has distanced itself from the Kyoto Protocol, have affected our country’s credibility to date, and will make it difficult for Canada to ultimately secure an agreement consistent with its interests. In particular, these observers believe that Canada may not be well placed to build sufficient bridges with key developing countries in

41

Page 59: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

order to secure their meaningful participation in and support for a post-2012 climate change agreement. Some interviewees did suggest that Canada’s ability to engage positively with other parties to the negotiations (particularly with developing countries) has improved during the period being evaluated. They noted that Canada was criticized in the lead-up to and during the Bali and Poznan COPs, due both to Canada’s domestic performance on climate change and its position related to the Kyoto Protocol. However, they indicated that with the creation of the Chief Negotiator and Ambassador position and the enhanced clarity that this brought to the structure of and messaging from the delegation, Canada’s ability to engage constructively on key issues was enhanced.

Rating: Progress Made; Attention Needed

6c International Financial and Other Obligations

The intended outcomes, outlined below, have been achieved as a result of International Financial and Other Obligations.

A) Canada has complied with financial and other obligationsThere is evidence in departmental reporting documents that indicate all assessed contributions have been made, and, due to the strict UNFCCC oversight processes, there is no evidence to indicate that these payments were not made in a timely fashion. Regarding non-assessed contributions for additional UNFCCC initiatives, EC made annual membership contributions to REEEP, and three project investments in M2M. In addition, EC made contributions to the OECD Climate Change Expert Group.

Regarding technology partnerships evaluated within this program component, there is evidence that Canada made all contributions in a timely manner to both technology programs, i.e., M2M and REEEP. The evaluators found evidence of Canada’s participation in the M2M steering committee, oil and gas subcommittee meetings, and partnership expo. These activities have contributed to enhancing Canada’s profile internationally with respect to clean technology. The M2M program has expanded Canada’s reach into new alliances with the U.S. EPA, China, and Mexico (government and private sectors). REEEP membership provides limited ability to influence how annual contributions are spent. However, since M2M is project-based, Canada can strongly influence how M2M contributions are spent, focusing spending on projects that have the greatest potential to decrease environmental risks by reducing methane emissions. Both REEEP and M2M are long-term programs, and their anticipated return on investment could not be determined at the time of the evaluation.

B) Canada’s positions are advanced Canada’s contribution to the UNFCCC assessed funding allows Canada to continue participation in the negotiations (see Partnerships and Negotiations for details on advancing Canada’s position in this respect).

Canada’s position of promoting clean technology partnerships to address air pollution issues was advanced through M2M. Clean technologies are being used in partnership with Mexican and Chinese industry sectors to address GHG emissions, and the M2M project investments have established new partnerships with the U.S. EPA, which offers Canada a broad platform to continue advancing its strategic technology interests.

42

Page 60: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Rating: Achieved

6d Asia–Pacific Partnership

The following outcomes were expected as a result of the APP:

A) Increased awareness of trade opportunities in climate-friendly technologiesEvidence indicates that Canadian APP Task Force–related industries have an overall increased awareness of trade opportunities relating to climate-friendly technologies. However, the initial portfolio of APP projects as described in the Executive Summary of Task Force Action Plans46 is weighted toward activities such as sectoral assessments, capacity building, identifying best practices, and technology research and demonstration, as opposed to actual technology deployment.

To promote the activities of the APP and increase stakeholder awareness of trade opportunities, APP representatives presented at a number of conferences/expos. Industry representatives in some task force groups (such as cement) communicate and work closely with individual industry stakeholders throughout Canada on APP activities.

Although there has been a small increase in awareness of trade-related opportunities with respect to climate-friendly technologies, it is still too early to assess project success let alone any climate change and air pollutant benefits because projects are in the process of being implemented.

Rating: Progress Made; Attention Needed

7. Have there been any unintended (positive or negative) outcomes that can be attributed to the program areas within the International Actions Theme? Were any actions taken as a result?

7a Particulate Matter Annex

The evaluation could not confirm any positive or negative unintended outcomes.

Rating: N/A

7b International Partnerships and Negotiations

There have been no significant unintended outcomes associated with this program component, but an opportunity exists to foster Canadian industry’s strengths in clean technology export.

Program documents demonstrate that there are positive opportunities arising from the international climate change negotiations held at COP 15. The approach anticipated under the Copenhagen Accord, wherein large developing countries will be articulating their climate change actions and technology requirements, will provide opportunities for Canadian

46 Asia–Pacific Partnership on Clean Development and Climate Executive Summary of Task Force Action Plans, October 31, 2006. Available at www.asiapacificpartnership.org/english/task_forces.aspx

43

Page 61: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

business to develop and promote clean technology internationally (e.g., opportunities beyond CCS, including technology deployment in areas such as enabling technologies for smart-grid development in countries looking to improve energy efficiency, sophisticated monitoring and control systems, etc.). Moreover, the negotiations are laying groundwork for private sector involvement and investment, particularly in developing countries, by improving the integrity of project development, addressing the needs for technologies for adaptation, and updating guidance on issues such as innovative financing, technology needs assessments and project development.

However, interviewees noted that declining resources can affect a department’s ability to maintain an appropriate level of human resources to develop the policy required to advance international commitments. There was a marked increase in international attention on climate change following COP 13 in Bali in 2007. Due to an associated increase in international meetings and events (within and outside the realm of the UNFCCC) to which delegation members had to travel, EC has had to supplement travel to these events through A-Base dollars, and other departments such as NRCan have had to cancel participation in events. The declining level of resources associated with this component (both for departments and for related G&C funds—see tables 1 and 2) is not proportional to the increased need for engagement in this area. For example, NRCan management noted the particular difficulty with the dichotomy between declining funding levels over the program period and the escalating demands on the international climate change file through UN processes and other fora such as the MEF, G8, IEA and APEC. Interviewees noted that NRCan funding expires in March 2011, yet the demands will continue over the long term given the necessity to conclude and subsequently implement a comprehensive international climate change agreement. Rating: N/A

7c International Financial and Other Obligations

There was one unintended negative outcome of this program area. The lateness of developing the national GHG registry and connecting it to the ITL had implications for at least one company that purchased credits but was not able to trade. In addition, this delay resulted in an investigation by the UNFCCC enforcement group into Canada’s compliance with UN obligations. EC program management responsible for the ITL connection considers this a very minor flaw in Canada’s otherwise solid reputation in fulfilling UNFCCC obligations over the past decade.

Rating: N/A

7d Asia–Pacific Partnership

The evaluation could not confirm any positive or negative unintended outcomes as a result of the APP.

Rating: N/A

44

Page 62: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

8. Are there any external factors outside the International Actions Theme that influence the success of its programs?

8a Particulate Matter Annex

The evaluation found evidence that external factors, both within and outside the program component, have affected the component’s ability to design, negotiate and complete the PM Annex.

Negotiations for the PM Annex were put on hold toward the end of 2008 as a result of shifting domestic policy contexts in both countries. In the United States, a July 2008 court decision to vacate (annul) the Clean Air Interstate Rule (CAIR), followed by a December 2008 Court of Appeals decision to remand the CAIR without vacatur,47 created significant uncertainty for the U.S. EPA regarding specific emission reduction commitments that could be included under the PM Annex. According to the evidence reviewed, the CAIR program (which was designed to reduce millions of tons of smog- and particulate-forming pollution from coal-fired power plants in 28 eastern states) was central to what the EPA would likely have proposed as targets under the Annex. Around this time, the change in U.S. administration48 led to a review of all existing environmental programs and their regulations. Since the United States does not enter into international agreements without first having specific regulations or legislation in place to ensure compliance, the United States made the decision to not proceed with the PM Annex negotiations until the status of CAIR (or any replacement to it) is resolved.

The evidence also suggests that at the same time, Canada began to consider alternatives to the Turning the Corner plan, which, once implemented, would have provided similar clarity regarding the types of emission reduction commitments and specific programs that could be referenced under the PM Annex. Ongoing coordination with the United States, however, is continuing under the broader bilateral Air Quality Agreement, and work to address transboundary PM is currently occurring under other CAA themes and international fora such as the Convention on Long-Range Transboundary Air Pollution and the Gothenburg Protocol to Abate Acidification, Eutrophication and Ground-level Ozone.

As a result of the changing policy context in Canada and the United States, and the significant turnover experienced in the Air Emissions Priorities Directorate, the evaluation found that the program was not able to meet its commitments fully, and under-spent in 2007 and 2008 for the PM Annex.

Rating: N/A

8b International Partnerships and Negotiations

External factors influence Canada’s ability to achieve its objectives related to the post-2012 international framework for action on climate change. These include decision-making processes under the UNFCCC and an associated lack of procedural rules. With respect to decision-making processes, substantive decisions under the Convention must be made by consensus that can enable a relatively small group of countries to block progress agreed to

47 Remand without vacatur is a mechanism by which a court remands back to an agency a decision in circumstances in which the court believes the agency rationale is flawed, yet the court declines to vacate the agency decision. 48 U.S. elections occurred on November 4, 2008, and Barack Obama assumed the presidency on January 20, 2009.

45

Page 63: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

by the majority of participants. This issue, and the aforementioned lack of procedural rules, will require attention in order for Canada to achieve the ultimate outcomes for this theme. Departments have indicated that they are working with international partners, through formal and informal processes, to move forward.

Perhaps of most significance, the new U.S. Administration and efforts to strengthen ties within North America have created an opportunity for Canada and the United States to explore regional initiatives to address climate change. The creation of the CED in 2009 is an example of these potential partnerships.

Rating: N/A

8c International Financial and Other Obligations

There was no evidence of external factors outside the Theme that influenced the success of this component.

Rating: N/A

8d Asia–Pacific Partnership

A number of external factors have influenced the success of the APP program in achieving its expected outcomes in a timely manner. The economic downturn starting in 2008 has led to tightening of budgets for many stakeholders, affecting the attendance of representatives at international task force meetings. Participation in project development for some industries has also been reduced due to budget constraints. The high overhead for participation in the APP, and the limited financial support from government, exacerbated the financial burden on industry stakeholders.

Another external factor that affected the success of the APP was the uncertainty around the domestic climate change agenda and associated GHG regulations. Industry indicated some hesitation to become involved in international APP projects, because they looked to focus resources on compliance with domestic GHG and air pollutant regulations first and foremost.

Staff turnover and ministerial changes (between 2008 and 2009) also led to delays and a lack of continuity in the APP, particularly with respect to final project approvals and the announcement of funding commitments. The first round of Canadian APP projects were not approved for several months (despite their successful reviews), delaying their progress.

Rating: N/A

46

Page 64: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.6 Evaluation Issue: Performance – Demonstrated Efficiency and Economy

Issue 5: Demonstrated Efficiency and Economy

Indicators Overall Rating

9. Are there more economic and efficient means of achieving objectives under the International Actions Theme?

Opinions of program managers, deliverers, partners and stakeholders on the ability of program elements to achieve intended results, compared to alternative design/delivery approaches

Opinions of program managers, deliverers and partners on whether investments under the International Actions Theme are a good use of public funds and whether the cost of producing outputs is as low as possible

Progress Made; Attention Needed

10. How could the efficiency of programs under the International Actions Theme be improved?

Opinions of program managers and deliverers on how the efficiency of programs under the International Actions Theme could be improved

11. To what extent have each of the International Actions Theme program areas been implemented, or are they on track to being implemented as planned and on time?

Evidence that the International Actions Theme is being implemented as designed

Evidence that program objectives have / have not been met within the anticipated time frame and/or budget

Barriers to program implementation are identified

Barriers to program implementation are resolved

Evidence that the contribution program implements funding agreements in a timely and well-managed manner

Evidence that applications to the contribution program are reviewed in a timely manner

12. Is the management and accountability structure for the International Actions Theme in place and functioning adequately to achieve expected results?

Opinions of program managers, deliverers and partners

Evidence of decisions being made Evidence that information, results and

decisions are being communicated to the appropriate parties

13. Are appropriate performance data being collected, captured and safeguarded? If so, is this information used to inform senior management / decision makers?

Presence of populated performance data systems, where appropriate

Evidence of collection of performance data, as appropriate

Evidence of decisions being made based on performance information

47

Page 65: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

SUMMARY:The evaluation found that the international climate change and clean air agenda is driven by the federal government’s positions and prescribed international requirements, which leave little flexibility in implementing alternative approaches to enhance economy and efficiency. However, options exist in some sub-areas (e.g., strategic allocation of G&C funds, location of the APP Secretariat). Options have been implemented to improve effectiveness, notably the establishment of the Chief Negotiator and Ambassador for Climate Change.

While there has been progress overall in implementing the Theme as planned, there have been significant delays in implementing the APP over the first two years, and the PM Annex has not proceeded as expected, due to delays in negotiations.

Integrated findings (applicable to all program components): An HMARF was developed for the entire Theme and is used for performance and financial

reporting at the corporate level. However, it has not achieved its other objectives of supporting decision making in the International Actions Theme or providing useful performance information to managers. In fact, the evaluation found that the International Actions Theme is not managed or implemented as a “theme” as originally articulated. Instead, individual program components are managed separately (with overall coordination of climate change areas). The inclusion of the PM Annex in this theme is considered a misfit with the other climate change program areas, and is more closely aligned with the domestic CARA than this theme. Further illustrations that the “theme” is not an integrated entity include: the intended DGMC is not operational (with program managers meeting only as required to manage individual programs); and there is departmental (not Theme) management of funds.

Performance data on the outputs generated from the International Actions Theme have been reported in Horizontal Performance Reports for the first two years. Compilation of this information is coordinated by the CAA–RMS with input from managers. Departments indicated that: instructions to fill out templates are often unclear and with short deadlines; there are challenges and risks associated with this type of performance reporting; and there also seems to be a lack of understanding around activities/outputs/outcomes for these policy-related programs. An assessment of this performance data49 resulted in a ranking for the International Actions Theme of “basic to fair,” with the quality weakening by the second year. The International Actions Theme performance data were assessed as failing to consistently report against expectations, excessively focused on activities as opposed to outputs and outcomes, and only recently reporting on progress or challenges.

Due to the delays in program implementation in some areas, and the fact that the HMARF and performance data are not used by or useful to managers, the overall rating has been assessed as “Progress Made; Attention Needed.”

9. Are there more economic and efficient means of achieving objectives under the International Actions Theme?

9a Particulate Matter Annex

As previously stated in section 4.4, the evaluation confirms that EC is the most appropriate federal department to complete activities to finalize the PM Annex. However, under the existing CAA structure, the majority of Canada–U.S. Air Quality Agreement work is funded by other sources (within CARA) to which the International Actions Theme makes a limited contribution.

49 An assessment of performance data for the International Actions Theme is available in the Review of CAA Reporting completed by Marbek in October 2009.

48

Page 66: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Rating: Progress Made; Attention Needed

9b International Partnerships and Negotiations

The international climate change agenda is driven by federal government positions and policy which, along with requirements in international frameworks and procedures, provide little flexibility in the approach and alternatives that Canada can take in implementing its related activities. As there are no clear alternatives for delivering the objectives of this program component, the evaluators consider the current approach to be the most economic and efficient means to achieve the program’s objectives. However, the evaluation noted that there is growing concern both internally and externally that funding is not adequate to resource international climate change activities. Interviewees from all participating departments note that dedicated funds are required to maintain the momentum of international negotiations, but, in some cases, negotiators were not able to attend international meetings due to declining available funds (e.g., EGTT meetings for NRCan).

Rating: Achieved

9c International Financial and Other Obligations

There are no alternatives identified for assessed contributions. There are no alternatives identified for other obligations such as the National Registry or ITL contributions. There were two alternatives identified for REEEP. There were no alternatives identified for M2M.

For non-assessed contributions, internal and external interviewees commented that M2M is seen as valuable and considered to be economic with efficient projects planned (i.e., projects are highly replicable with high GHG-reduction potential). REEEP, however, is viewed as being slightly less economic, though it is difficult to determine the program value for dollars spent in terms of improvements to air quality. The International Renewable Energy Agency (IRENA)50 and IEA51 were identified by program managers as potential alternatives to REEEP, but there is no evidence to suggest that either IRENA or IEA would be more cost-effective means of achieving the objectives than REEEP. No decision had been made at the time of this evaluation on contributing to these other initiatives instead.

Rating: Achieved

9d Asia–Pacific Partnership

The evaluation found that there could be more economic and efficient means of achieving objectives under the APP, but the APP is still a good use of public funds overall.

50 IRENA was established in January 2009, with 142 states and the European Union as members to date. IRENA promotes the adoption and sustainable use of all forms of renewable energy. Acting as the global voice for renewable energies, IRENA facilitates access to all relevant renewable energy information, including technical data, economic data and renewable resource potential data. IRENA aims to share experiences on best practices and lessons learned regarding policy frameworks, capacity-building projects, available finance mechanisms, and renewable energy–related energy efficiency measures.51 The IEA is an intergovernmental organization that acts as energy policy advisor to 28 member countries in their effort to ensure reliable, affordable and clean energy for their citizens. The mandate of the IEA incorporates the “three Es” of balanced energy policy–making: energy security, economic development and environmental protection. Current work focuses on climate change policies, market reform, energy technology collaboration, and outreach to the rest of the world, especially major consumers and producers of energy.

49

Page 67: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

The APP provides a number of benefits as expressed by participating stakeholders, such as allowing industry to collaborate with government on international climate change activities, providing opportunities for the development of projects with international partners, developing a common-level playing field between international competitors such as the Chinese in the aluminum and cement sectors, and exchanging valuable clean-technology expertise between member countries. These benefits suggest that the APP is a good use of public funds.

The evaluation found that the funding allocated to the APP has been largely used to foster bilateral and multilateral relationships, and is not necessarily driven by the most efficient means of achieving other objectives such as GHG reductions or trade promotion. In order to achieve any real reductions in GHGs or air pollutants (an objective of the federal government), stakeholders suggest that Canada should focus on fewer task forces (or fewer projects) and avoid spreading limited resources over too many smaller projects that are perceived to have fewer benefits for GHG and air pollutant reduction.

The evaluation found that the APP Secretariat located at EC may not be working well in all respects. Interviews with Task Force members support alternative approaches to managing the APP. For example, some interviewees believed that relocating the Secretariat to NRCan or developing a joint secretariat between NRCan and EC would be more efficient because the APP’s focus on the development and deployment of clean technologies is more aligned with the mandate and expertise of NRCan.

Rating: Progress Made; Attention Needed

10. How could the efficiency of programs under the International Actions Theme be improved?

10a Particulate Matter Annex

Program staff commented that efficiencies could be gained if domestic and international air quality activities were merged into a single funding source. It is believed that a single funding structure would provide opportunities for streamlining air quality activities, sharing information sources and developing partnerships, thus leading to a more coordinated and comprehensive approach to addressing PM. It is also believed that this approach would minimize planning and reporting requirements, and maximize the resources available for air quality activities.

Rating: Progress Made; Attention Needed

10b International Partnerships and Negotiations

Progress has already been made in enhancing the efficiency of programs associated with this theme component, and no specific additional measures have been identified.

The 2008 establishment of a Chief Negotiator and Ambassador for Climate Change, as well as a streamlined delegation housed together at EC, reflects a more efficient, coherent and coordinated approach to managing international partnerships and negotiations. This shift

50

Page 68: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

has also enabled the delegation to provide a clear Canada-wide voice on international climate change issues. NRCan’s International Environmental Policy Division increases efficiency by acting as a focal point for horizontal coordination on behalf of that department. This coordination relates to the overall policy framework on international climate change and its components (mitigation, technology, financing, adaptation and forestry), providing a strategic and holistic perspective on policy development and acting as a key contact with other departments. In addition to maximizing efficiency, this approach has ensured an integrated NRCan voice on international climate change issues. Rating: Achieved

10c International Financial and Other ObligationsThere were no specific programs identified as better suited to support Government of Canada objectives for this component, nor were any specific instruments to improve program efficiency identified. Canada’s participation in the international technology partnerships within this component appear to be operating efficiently, although the technology partnership programs funded within this component do not have specific program accountability or management structures available. Rather, they are contribution-based and beyond the control of EC. Other technology partnerships (e.g., on CCS) that also advance Canadian objectives on international climate change are not included within this program component.

Rating: Achieved

10d Asia–Pacific Partnership

The evaluation found that the efficiency of the APP program could be improved. Suggestions include the following:

Information regarding project funding should be clear and provided in a timely fashion so that industry can make well-informed decisions.

The project approval process could be improved—the call for project proposals could be done in a timely fashion, and adequate time could be given for project reviews. Also, more project-specific expertise and technical input should be provided in the review of projects.

Increased communication and clarity should be provided regarding the mandate of the APP and the direction of the APP post-2011 in order to provide reassurance to stakeholders and investors.

Due to the high costs of international travel for Canadian representatives, the number of international task force meetings should be reduced. Meetings should be better organized in order to focus on the exchange of valuable expertise and enhance collaboration between member countries. Project updates and other less-interactive aspects could be covered in conference calls or web-based meetings.

While progress has been made, there have been delays in implementing plans as expected—particularly over the first two years of the APP program, during which time there was significant under-spending: the full $4 million (G&C funds) allocated for 2007–08 was

51

Page 69: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

re-profiled ($2 million to 2008–09, $1 million to 2009–10 and $1 million to 2010–11), while in 2008–09 only $818,629 was spent with $2.4 million lapsed. In 2009, $12 million was committed to projects, with approximately $870,000 left for a third round of projects in2010–11.

Rating: Progress Made; Attention Needed

11. To what extent have each of the International Actions Theme program areas been implemented, or are they on track to being implemented as planned and on time?

11a Particulate Matter Annex

The evaluation found that the changing policy context in relation to addressing air pollution in both Canada and the United States to advance PM Annex–related work has delayed completion of the PM Annex and its required components.

As noted in the response to questions 5 and 6, some progress in achievement of outputs and early outcomes has been made in developing a PM Annex. However, the majority of these activities were completed between 2007 and 2008, with fewer activities in 2009 and 2010. Of the $550,000 annual allocation to develop the PM Annex, approximately 25% of the resources were spent in 2007–08, with a significant increase of 72% spent in 2008–09.

Table 5: PM Annex Planned Spending vs. Actual Spending, 2007–08 to 2008–092007–08 2008-09Planned Spending Actual Spending Planned Spending Actual Spending$550,000 $136,699 $550,000 $397,238

Although the evaluation could confirm that work plans for the period 2007–08 to 2009–10 exist, there was no systematic approach to reporting on which activities have or have not been completed. Similarly, the evaluation did not find an overarching timeline or critical path for completing the PM Annex.

Rating: Little Progress; Priority for Attention

11b International Partnerships and Negotiations

Negotiation and partnership activities have been consistently carried out pursuant to this component of the International Actions Theme. However, overall progress toward a post-2012 international climate change framework has been slow due to a variety of external factors largely outside departmental control (see above), and it is unlikely that such a framework will be in place within the time frame of this program component (i.e., 2011). Interviews with members of the delegation highlight that an increase in resources will be required to ensure that Canada can play an active role in finalizing a global long-term agreement on climate change. These interviewees underlined that the existing resource allocations are limited and do not meet the level of demand dictated by the international climate change agenda.

Rating: Progress Made; Attention Needed

52

Page 70: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

11c International Financial and Other Obligations

For assessed contributions, evidence indicated contributions were paid on time. For G&C funds, there is no evidence to indicate whether EC’s G&C program implements funding agreements in a timely and well-managed manner. Contributions are not application-based; there are no strict requirements to meet specific timelines. However, the M2M contribution program does appear to be moving slowly, with only one project implemented and two projects planned in a four-year time frame. This was observed in the absence of having accountability measures to evaluate given that the M2M is an international program operating outside of EC’s control. A strategic framework for selecting organizations to receive G&C contributions within the M2M partnership (e.g., project selection, timelines, accountability, etc.) was not found.

Other obligations were completed, such as fees to the ITL and development of the National Registry. Fee payments to the ITL were on time but development of the Registry was six months late and it took some time after that to connect to the ITL.

Rating: Progress Made; Attention Needed

11d Asia–Pacific Partnership

Evidence indicates that the APP has been implemented as initially designed. However, substantive evidence has been collected which indicates that program objectives have not been met within the anticipated time frame. Delays in project approvals and funding announcements appear to be the result of changes in senior levels of government, and senior levels of government focused on higher priority issues. The delays have led to project development being delayed, as well as:

a loss of interest by the private sector to engage in task force meetings and projects; a timeline crunch for projects (projects must spend funding within the year); and industry representatives placed in an awkward position with project partners.

No domestic barriers to program implementation were identified. Membership in the APP was approved by senior levels of government, and $20 million was allocated to the APP for implementation of the EC Secretariat ($4.6 million for operations and maintenance) and for the funding of projects ($15.4 million G&C funds).

Rating: Progress Made; Attention Needed

12. Is the management and accountability structure for the International Actions Theme in place and functioning adequately to achieve expected results?

12a Particulate Matter Annex

Program staff commented that the current reporting structure prescribed by the HMARF is useful for financial tracking. However, this structure is limited in terms of its ability to accurately reflect program performance and progress against planned activities and desired

53

Page 71: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

outcomes and is not appropriate for resolving barriers (e.g., policy shifts and approaches to address air emissions) or identifying alternative activities.

Furthermore, PM-related work in EC is tied to multiple funding requests, which program staff argue has led to multiple planning and reporting requirements that affect the Department’s ability to communicate a coherent and coordinated approach on how to plan, complete and report on the results of PM activities.

Rating: Progress Made; Attention Needed

12b International Partnerships and Negotiations

Program staff consistently highlighted that the HMARF is being used primarily for financial tracking rather than overall management of activities under the component. It was also noted that expectations, deliverables, roles and responsibilities are not always clear.

Program staff interviewees commented that communication between the CAA–RMS and other federal partners on roles, responsibilities and expectations could be improved to ensure that connections between issues and departmental mandates are made and that shared objectives are met.

Rating: Progress Made; Attention Needed

12c International Financial and Other Obligations

The management and accountability structure for assessed contributions is clear and functioning adequately to achieve expected results.

Information was not identified to verify whether there is a management and accountability structure for EC’s G&C funds for this program component (within this program component, the G&C funds relate primarily to REEEP and M2M). The management and accountability structure for other obligations, such as ITL fees, is clear and functioning adequately to achieve expected results.

Within this component, DFAIT serves as the focal point for overseeing financial contributions to the UNFCCC in order to meet the assessed funding requirements, and EC serves as technical lead on the National Registry for emissions credits, managing the connection to the ITL, participating on behalf of Canada in technology partnerships (e.g., REEEP), and leading Canada’s technology trade initiatives through M2M.

The management and accountability structure for assessed contributions is clear and primarily administrative: the UNFCCC Secretariat has oversight functions for this accountability check. For other UN-related obligations, such as the National Registry and associated ITL connection, the UN has a technical review committee that reviews technical aspects of the registry design and an enforcement branch to follow up on anything that is non-compliant with the provisions of the UNFCCC and Kyoto Protocol.

54

Page 72: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

With respect to supplementary funding provided through EC’s G&C funds (M2M, REEEP), a management and accountability structure was not identified. This may be because these programs are international-contribution-based programs, and largely operate outside the UN processes and beyond EC’s control. This is not seen as problematic in any significant way, since it is a contribution program that operates similarly to the UNFCCC contributions of assessed funding, which does not have a separate management and accountability structure either. Both the G&C funds and assessed funding amounts within this component are small, with project information publicly available on program websites.

Rating: Progress Made; Attention Needed

12d Asia–Pacific Partnership

Notwithstanding the suggestion of a potential relocation to NRCan (see Evaluation question 9 for the APP), the APP Secretariat at EC is well-organized and functioning effectively. Federal departments coordinate technical and policy work related to Canada’s participation in the APP’s task forces, and have developed a process to evaluate and fund projects that promote the development, diffusion and deployment of clean technologies.

However, expected outputs/outcomes from the APP have not been consistently understood by all stakeholders. More clarity on priorities among the desired outcomes and the resulting direction of the APP from EC’s APP Secretariat and senior levels of government would be advantageous.

As well, higher-level debriefs from Policy Implementation Committee meetings are not always provided, or are not provided in a timely manner to all Canadian stakeholders. This information would be very useful for their successful participation in the APP.

Rating: Progress Made; Attention Needed

13. Is appropriate performance data being collected, captured and safeguarded? If so, is this information used to inform senior management / decision makers?

13a Particulate Matter Annex

Performance data associated with this program component is reported in CAA Horizontal Performance Reports and departmental Reports on Plans and Priorities, as well as in reporting through the Canada–U.S. Air Quality Agreement. However, the source of this performance data is not clear. The evaluation found neither evidence of specific performance-measurement metrics for this component nor documentation (e.g., internal quarterly reporting, annual reports, etc.) to verify that program staff regularly report on the achievement of program activities in accordance with their work plans.

Rating: Little Progress; Priority for Attention

55

Page 73: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

13b International Partnerships and Negotiations

Performance data (outputs and spending levels) associated with this program component are reported in CAA Horizontal Performance Reports and departmental Reports on Plans and Priorities.

However, performance measures are aggregated for reporting purposes, and are not used, or seen as useful, by program managers. Managers commented that the performance framework and Logic Model were too rigid, too time-consuming to complete, and not able to adapt to the evolving policy/directional changes that have affected the program.

In addition, the existing HMARF structure is not reflective of all international climate change work under way throughout the federal government. A full understanding of progress associated with this component is impeded by the exclusion of NRCan’s Canadian Forestry Service, which is engaged in several key areas under the UNFCCC, and the Canadian International Development Agency, which can play an important role in bilateral partnerships and understanding the needs of least-developing countries.

Rating: Progress Made; Attention Needed

13c International Financial and Other Obligations

Performance data for assessed contributions is collected by the UNFCCC and publicly available. Senior managers are aware of its existence.

Performance data for supplementary funding through EC’s G&C funds to the M2M and REEEP programs was not available, and it is uncertain whether it is being collected.

The performance data collected for assessed contributions is clear and primarily administrative, overseen by the UNFCCC Secretariat. For other UN-related obligations (e.g., the National Registry and associated ITL connection), the UN has a technical review committee to review registry design, and an enforcement branch to address non-compliance. All of these cross-checks, review reports and investigative documents are publicly available on the UNFCCC website. This evaluation did not identify any evidence to show that EC senior managers access and use this publicly available information to support decision making within their obligations and technology programs in this theme.

With respect to supplementary funding provided through EC’s G&C funds (M2M, REEEP), a management and accountability structure was not identified, and no program performance data were made available upon request.

Rating: Progress Made; Attention Needed

13d Asia–Pacific Partnership

There is no evidence to suggest that performance data is being collected, captured and safeguarded at this time.

56

Page 74: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

The need for performance data to assess the success of the APP has been discussed at APP meetings, but such data have not yet been developed. Performance indicators such as the amount of GHG reductions per project have been suggested by APP stakeholders. The absence of any performance indicators symptomizes the lack of an overall results framework (noted in questions 2 and 9).

In 2006, an initial set of action plans were developed for each of the Task Forces, containing initial priority activities for implementation by each of the Task Forces. Objectives, immediate and medium-term specific actions, milestones, and potential projects were also included in the action plans. However, no evidence was found to suggest that progress of the Task Forces was compared against the goals and milestones outlined in the action plans.

Rating: Little Progress; Priority for Attention

4.7 Findings for DFAIT’s Class of Contributions in Support of International Environmental Programs

As previously described in section 3, the evaluation team was tasked to examine DFAIT’s Class of Contributions in Support of International Environmental Programs, and more specifically how that department complements the Government of Canada’s approach to achieving the outcomes intended for the International Partnerships and Negotiations component. As this was a distinct request of the evaluation, the findings unique to this contributions program are presented separately in this section.

The information presented below has been aligned with a select number of evaluation indicators that were included in the evaluation framework (see Annex 1 for details).

4.7.1 Overview

DFAIT is responsible for the disbursement of additional G&C funds under the International Actions Theme. Under this theme, DFAIT received $840,000, of which $280,000 was allocated in 2007–08, $230,000 in 2008–09, $180,000 in 2009–10, and an expected $150,000 in 2010–11.

Developed in April and September of 2007, the Terms and Conditions for a Class of Contributions in Support of International Environmental Programs and the Accountability, Risk and Audit Framework (ARAF) for a Class of Contributions in Support of International Environmental Programs provide the architecture and broad guidance for identifying and selecting recipients in accordance with Canada’s international environmental agenda.

The ARAF states that the key objective for payments under the Class of Contributions is: “To promote and enhance Canada’s role in international environmental efforts by supporting international environmental programs, activities and projects in support of Canadian foreign policy objectives.”

Achievement of the key objective is anticipated to lead to the following intermediate to long-term outcomes:

position Canada as an international leader in environmental protection;

57

Page 75: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

advance Canada’s foreign policy objectives in relation to sustainable development and environmental protection; and

facilitate the advancement of governance on global environmental issues through Canada’s support for multilateral environmental agreements and organizations as well as institutions and NGOs.    

58

Page 76: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

4.7.2 Findings

Evaluation Issues (and related questions):

Indicators Overall Rating

Issue 1: Continued Need for the Program

1. Are program components within the International Actions Theme connected with key international climate change and air quality needs?

Evidence that the contribution program supports international objectives

Achieved

Issue 4: Achievement of Expected Outcomes

5. To what extent have intended outputs been achieved as a result of programs within the International Actions Theme?

Evidence of the level of support and impact of non-assessed contributions on enhancing Canada’s position abroad on its objectives

Identification of opportunities for collaboration (enhanced networking at international meetings/conferences) through the approval of non-assessed contributions

Increase in the number of non-assessed contribution recipients

Achieved

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

Evidence that the impact of non-assessed contributions is targeted strategically and is timely to identify potential stakeholders and beneficiaries

Evidence that the reach has expanded, and new networks have been created and strategic alliances formed, through the non-assessed contributions

Evidence that non-assessed contributions are for activities to reduce harmful environmental risks

Evidence that the contribution program shows a return on investment to support international clean air objectives

Progress Made; Attention Needed

Issue 5: Demonstrated Efficiency and Economy

11. To what extent have each of the International Actions Theme program areas been implemented, or are they on track to being implemented as planned and on time?

Evidence that the contribution program implements funding agreements in a timely and well-managed manner

Evidence that applications to the contribution program are reviewed in a timely manner

Achieved

SUMMARY:

59

Page 77: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Evaluation Issues (and related questions):

Indicators Overall Rating

The evaluation confirms that DFAIT’s G&C program is: relevant and in high demand by members of the international climate change community; executed efficiently; used to better understand international climate change issues and communicate Canada’s position to organizations within and outside of the UNFCCC; and appropriate for achieving desired outcomes.

The section below indicates the more detailed findings by the evaluation questions and indicators selected as applicable in the evaluation framework (see Annex 1).

1. Are program components within the International Actions Theme connected with key international climate change and air quality needs?

Indicator: Evidence that the contribution program supports international objectives

The evaluation confirmed that DFAIT has provided $690,000 in funding to ten projects from five organizations between 2007–08 and 2009–10. These contributions were found to provide funding to initiatives that align with the class contribution’s Terms and Conditions and the key objective and anticipated (short to intermediate to long-term) outcomes referenced in the ARAF. Program staff commented that these contributions have been particularly valuable in advancing Canada’s understanding of adaptation and mitigation issues in Least Developed Countries (LDCs) and Small Island Developing States (SIDS).

Table 6: DFAIT Grants and Contributions, 2007–08 to 2009–10Voluntary Contributions: 2007–08

Global Climate Observing System

Support to the Central America and Caribbean component of the Technical Support Project for the Americas

$50,000

Center for Clean Air Policy Contribution to support the activities of the Center for Clean Air Policy’s Future Actions Dialogue $50,000

UNFCCC–Consultative Group of Experts Meeting

To support a workshop in which developing countries that are part of the UNFCCC are assisted in completing their reporting obligations

$50,000

UNFCCC/Kyoto Protocol: Supplementary Fund

Contribution to support additional UNFCCC negotiating sessions in 2008 $130,000

Voluntary Contributions: 2008–09

Global Climate Observing System

Support for the Central American and Caribbean component of the Technical Support Project for the Americas

$35,000

Center for Clean Air Policy Contribution to support the activities of the Center for Clean Air Policy’s Future Actions Dialogue. $40,000

UNFCCC/ Kyoto Protocol Supplementary Fund Support for ongoing activities and to the LEG $125,000

Organization for American States–Caribbean Support

Assessing and Mitigating the Impacts of Climate Change on Coastal Infrastructure in the Caribbean $30,000

 Voluntary Contributions: 2009–10 IPCC Secretariat Support to the IPCC to assess the scientific, technical

and socio-economic information relevant for the understanding of the risk of climate change and options for addressing it

$50,000

UNFCCC LEG To provide support to the program of work for the LEG, progress adaptation, and the development of GHG inventories in developing countries

$130,000

60

Page 78: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Total from 2007–10   $690,000

Rating: Achieved

5. To what extent have intended outputs been achieved as a result of programs within the International Actions Theme?

Indicator: Evidence of the level of support for and impact of non-assessed contributions on enhancing Canada’s position abroad on its objectives

Program staff highlighted that Canada has contributed to international organizations (e.g., IPCC, Global Climate Observing System), with a mandate to conduct science in order to help clarify and develop a shared understanding of the issues and challenges associated with climate change that need to be discussed at the international negotiations. Thus, contributions to these international organizations are seen to be valuable because they help provide a sound base of evidence for developing Canada’s international positions.

Indicator: Identification of opportunities for collaboration (enhanced networking at international meetings/conferences) through the approval of non-assessed contributions

DFAIT’s voluntary contributions consist of those that are supplementary to both assessed and non-assessed contributions. Contributions flowing to the UNFCCC have been particularly useful in contributing to the work completed by the LEG Expert Working Group, of which Canada is one of three participating developed countries. As a result of providing supplementary funding to this group, Canada is well-placed to maintain an ongoing working relationship with developing countries and demonstrate leadership in the areas of adaptation and mitigation. Moreover, DFAIT’s non-assessed contributions have allowed it to establish ongoing partnerships and dialogue with organizations outside of the UNFCCC (e.g., the Organization of American States, the Global Climate Observing System, etc). Program staff indicated that this funding is particularly helpful for communicating Canada’s positions to a broader audience as well as maintaining a presence in the international community in order to address international climate change.

Indicator: Increase in the number of non-assessed contribution recipients

DFAIT has provided 10 contributions to different recipients: four in 2007–08, four in2008–09, and two as of December 31, 2009 (fiscal year 2009–10). The evaluation was unable to confirm that department’s planned recipients for fiscal year 2010–11.

Rating: Achieved

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

Indicator: Evidence that the impact of non-assessed contributions is targeted strategically and is timely, in order to identify potential stakeholders and beneficiaries

The Terms and Conditions provide criteria for selecting eligible recipients, and indicate that a selection committee within DFAIT’s Environment, Energy and Sustainable Development Bureau will select eligible recipients that support the achievement of the class of contributions’ anticipated results. The evaluation confirmed that although a formal selection

61

Page 79: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

committee has yet to be established, the Bureau’s Climate Change and Energy Division ensures that funds are awarded to recipients in accordance with recipient criteria outlined in the Terms and Conditions. The evaluation could not identify a formal plan that demonstrates alignment between Canada’s objectives or negotiating position and the projects funded under the Class of Contributions in Support of International Environmental Programs. However, the evaluation confirmed that the funding concentrates on enhancing Canada’s understanding of global climate change issues such as mitigation and adaptation, and that this enhanced understanding is used to advance Canada’s leadership in the LEG and international negotiations.

Indicator: Evidence that the reach has expanded, and new networks have been created and strategic alliances formed, through the non-assessed contributions

The evaluation notes that program staff believe the G&C program is valuable for communicating Canada’s position to developing countries and international organizations, and addressing specific topics (e.g., effects of climate change in SIDS).

Interviews with recipients revealed that the overall level of awareness of DFAIT’s program is low in the international community, with several interviewees indicating they were surprised that DFAIT was providing funding to advance global thinking on international climate change. In some cases, interviewees commented that they would have assumed that departments with a natural resource or environment mandate (e.g., EC, NRCan) would be offering this type of funding as opposed to DFAIT.

Indicator: Evidence that non-assessed contributions are for activities to reduce harmful environmental risks

Although the evaluation could not confirm that the results of a specific grant or contribution have directly reduced harmful environmental risks, there is strong evidence to indicate that the non-assessed contributions assist with developing a more fulsome understanding of the adverse effects caused by climate change in key topic areas such as adaptation and GHG emission reduction.

Indicator: Evidence that the contribution program shows a return on investment to support international clean air objectives

The evaluation was unable to determine a link between the contribution program and DFAIT’s international climate change objectives. Performance reporting that demonstrates how projects funded by the contribution program are contributing to objectives was not found.

However, interviews with program staff and recipients underlined that the program is well-run and supports a global effort to further the level of understanding and thinking on international climate change. Additionally, interviews with recipients confirm that program funds are valuable for pooling resources to fund large-scale or ongoing programs dedicated to addressing international climate change.

Rating: Progress Made; Attention Needed

62

Page 80: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

11. To what extent have each of the International Actions Theme program areas been implemented, or are they on track to being implemented as planned and on time?

Indicator: Evidence that the contribution program implements funding agreements in a timely and well-managed manner

Contribution recipients indicated that funding is distributed to organizations quickly. Furthermore, recipients stated that DFAIT’s application and review process is neither rigid nor time-consuming and that the rules, restrictions and oversight mechanisms are minimal compared to processes and procedures required by other donors.

Indicator: Evidence that applications to the contribution program are reviewed in a timely manner

Interviews with contribution recipients confirmed that the application and selection process is efficient, and that when accepted, payments flow to the organizations quickly.

Rating: Achieved

63

Page 81: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

5 Conclusions

The conclusions for the CAA International Actions Theme by the five key evaluation issues are presented below.

5.1 Relevance

Issue 1: Continued Need for the Program

1. The program components and activities under the CAA International Actions Theme are relevant and continue to be needed to achieve expected outcomes.

Findings from each program area indicate that the implemented activities reflect the need for action on climate change and air quality based on scientific evidence. Program components are designed to address climate change and air quality needs and continue to be relevant. Canada’s long-term goals are in line with commitments by G8 leaders to reduce global emissions by at least 50% by 2050.

Findings from each program component also indicate that the activities have adapted appropriately to changes in federal government policy and priorities. However, federal priorities and work in this area do not necessarily reflect provincial/territorial views on the actions needed to address climate change and provincial/territorial perspectives are critical in order for federal positions on climate change to be truly national in scope. Thus, further work in engaging the provinces and territories is necessary to achieve increased consideration of domestic stakeholders’ interests, particularly for the PM Annex.

In addition, the evaluation has shown that the International Actions Theme’s contribution programs have been implemented in support of the intended international objectives, and continue to be needed to work toward the expected outcomes.

Issue 2: Alignment with Government Priorities

2. Activities are in direct alignment with federal government priorities.

Each program component has demonstrated that the implemented activities have been as a direct result of priorities articulated in federal Throne Speeches from 2006 to 2009, as well as responding to the 1997 Canada–U.S. Joint Plan of Action on air quality, the 2007 Turning the Corner Plan, and Canada’s commitment to develop and deploy clean technologies and provide economic benefits to Canadian industry. Actions are also aligned with the broader Government priority to strengthen bilateral relationships (especially with major developing-country emitters such as China) and promote closer cooperation in North America (e.g., joint action on transboundary PM and alignment of Canada–U.S. positions on climate change). Thus, activities are in direct alignment with federal government priorities.

In addition, there remain opportunities to more closely align Canada’s industrial strategies and activities with the approach emerging from the Copenhagen Accord (which anticipates that large developing countries will articulate their climate change

64

Page 82: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

actions and technology requirements), and thus take advantage of opportunities for Canadian business. This would better align program activities with Canada’s federal priority to develop and deploy clean technologies and provide economic benefits to Canadian industry.

3. Activities are aligned with the priorities of participating departments.

The evaluation found that the program components and activities are appropriately aligned with the priorities of the participating federal departments, as articulated in their Program Activity Architectures and Reports on Plans and Priorities. These include:

• EC: direct linkage to the “climate change and clean air” program activity and “GHG and air pollution reduction” priorities;

• DFAIT: direct linkage to the “international policy and advice and integration” program activity and the “U.S. and hemisphere” priority;

• NRCan: direct linkage to “clean energy” program activity and priority;• IC: direct linkage to “economic development” program activity and priority to “foster

internationally competitive businesses and industries” priority.

Issue 3: Alignment with Federal Roles and Responsibilities

4. Activities are aligned with federal roles and responsibilities.

The federal government has the appropriate lead role in this theme, as activities relate to international actions to address transboundary/global environmental issues, which is an area of federal jurisdiction. The evaluation also found that the federal roles are clear and well-coordinated among the federal partners, and no duplication was found with other similar program areas.

However, given that provinces and territories have a crucial role in implementing the targets and timetables that Canada agrees to internationally, there is also a role for them. While it was found that there had been more structured and integrated efforts to involve provinces/territories after the Chief Negotiator position was established (i.e., in 2008 compared to 2006), particularly in climate change negotiations on the Copenhagen Accord, they were not engaged in other components such as negotiating the PM Annex.

5.2 Performance

Issue 4: Achievement of Expected Outcomes

5. With some delays, progress has been made in the achievement of outputs in component areas.

The evaluation found the achievement of the following expected outputs:

• assessed contributions have been delivered to international bodies;• the National Registry is in place;• negotiations on a new Accord have been carried out under the UNFCCC;

65

Page 83: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

• policies, negotiating positions and advice have been produced for climate change, although only initial research and early negotiating positions have been completed for air quality components;

• there has been participation in a variety of international bodies and initiatives outside of the UNFCCC, and support has been provided to foster partnerships (e.g., a range of initiatives, such as the EGTT, LEG and contribution programs, that have built capacity, increased understanding in areas such as adaptation, advanced discussions on key issues in negotiations, and advanced clean technologies); and

• formal and informal information sessions have taken place with other levels of government, Aboriginal groups and domestic stakeholders, to support climate change activities.

However, the evaluators found significant delays in achieving certain activities and outputs:

• the National Registry was approximately six months late;• many activities under the APP were not initiated in the first two years; and • the PM Annex work has not proceeded as planned.

6. Some progress has been made in the achievement of immediate outcomes.

The evaluation found the achievement of the following expected immediate outcomes:

• Canada has complied with its financial and other obligations under international climate change treaties and agreements.

• Canada’s positions with respect to the post-2012 international climate change framework have been clearly and consistently communicated within the context of the UNFCCC, in a range of other multilateral and bilateral fora, and with strategic allies such as members of the Umbrella Group. In addition, a number of these Canadian positions are now reflected in the Copenhagen Accord, although key gaps remain. However, it is difficult to attribute the advancement of these interests directly to Canadian international activities under this theme, particularly given the multiple factors involved in international negotiations.

• There has been improved information sharing and more formal engagement mechanisms with provinces and territories, but enhanced consultation activities have not necessarily resulted in increased consideration of stakeholder interests in Canada’s positions. For the PM Annex, this evaluation found no evidence that stakeholders were consulted directly, and it was not clear how Canada’s position was developed or how the views of domestic partners and stakeholders have had an influence or have been meaningfully reflected.

• There has been a small increase in the awareness of trade-related opportunities related to climate-friendly technologies.

• While bilateral and multilateral relationships have been advanced through voluntary (i.e., non-assessed) contributions, and contributions have contributed to a shared understanding of issues and advancing Canada’s position abroad, there is no evidence yet of a return on investment in terms of GHG reductions.

It is too early to ascertain the degree to which the intermediate or final outcomes can be achieved.

66

Page 84: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

7. No significant unintended outcomes were identified, but opportunities are emerging from the UNFCCC process to enhance Canada’s export opportunities and partnerships for clean technology.

No significant unintended outcomes (positive or negative) were found due to program activities. However, significant opportunities are emerging from the UNFCCC process that Canada can build upon to enhance its export opportunities and partnerships for clean technology, although these linkages seem not yet to be explicitly made with Canada’s industrial strategies.

8. External factors have influenced success and delayed the achievement of outcomes.

A number of significant external factors have influenced (and continue to influence) the success of the program in achieving its outcomes fully and in a timely manner, including the following:

• The increasing prominence of climate change in the broad international arena, and the associated proliferation of meetings within and outside the realm of the UNFCCC (beyond expectations), is placing larger-than-planned demands on program resources, particularly around the need for more and more frequent international meetings and related program support.

• Staff turnover and ministerial changes have led to delays and a lack of continuity on some program files, such as the PM Annex and the APP.

• Changes in Canadian and U.S. policy direction and positions on key issues have affected the ability of Canada to exercise influence and achieve expected goals (e.g., the PM Annex not yet being achieved). The emergence of a new U.S. position and engagement on climate change, and evolving positions of large developing country emitters, has largely determined the outcome of international climate change negotiations.

• Domestic performance on climate change (and Canada’s distancing from the Kyoto Protocol) has affected Canada’s credibility in international negotiations, particularly in 2006–2008.

• At Copenhagen, the UNFCCC decision-making process (e.g., consensus required) and associated lack of procedural rules created challenges and barriers to achieving international climate change outcomes in line with federal interests.

Issue 5: Demonstrated Efficiency and Economy

9. More economic and efficient means to achieve objectives are limited and have not been the focus of program implementation.

The international climate change and clean air agenda is driven by federal government positions and policy, which, along with prescribed requirements of international frameworks and procedures, provide little flexibility in the approach and alternatives that federal government departments can take in implementing its program component activities.

That said, choices exist for Canada to use its voluntary contributions to international bodies in the most economic and efficient manner. The evaluation found that these

67

Page 85: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

contributions have been largely used to foster bilateral and multilateral relationships, and that the choices made are not necessarily driven by the most efficient means of achieving other objectives such as GHG reductions or trade promotion. There is no overarching strategic framework in place to guide the economic allocation of contributions, and the multiple goals of programs are not prioritized.

In addition, the efficiency of the APP program could be improved in terms of communication with stakeholders and investors, and the project approval process. The evaluation also found that the APP Secretariat may not be best-placed to take efficient advantage of the federal expertise in clean energy (i.e., its location in EC as opposed to leveraging NRCan’s expertise).

As well, the PM Annex is better-placed under CARA, because the majority ofCanada–U.S. Air Quality Agreement work is funded under this initiative and a single funding source will streamline activities and provide a more streamlined and coordinated approach to addressing PM.

10. Some options to improve efficiency have been implemented.

The evaluation found two key initiatives that have been implemented to improve efficiencies. First, the establishment of a Chief Negotiator and Ambassador for Climate Change in the federal government was widely seen as highly beneficial and a key factor in improving coherence and coordination on the climate change file. Second, the establishment of the HMARF, which aims to enhance efficiencies in management across the entire CAA, is useful for corporate and financial reporting on the Theme but has not been applied in a meaningful way at the program level by managers.

11. There has been progress in program implementation as planned, but there have been significant delays in some areas.

While progress has been made, as noted in conclusions 5 and 6, there have been delays in implementing plans as expected—particularly during the first two years, during which time there was significant under-spending on the International Actions Theme. Much of the under-spending was due to delays in allocating APP funds (some funds were re-profiled), with some under-spending on the PM Annex as well, due to delays in negotiations.

Conversely, there has been slight over-spending on the International Partnerships and Negotiations component, with a higher than expected workload demand (including increased meetings and travel) due to heightened global action on climate change and the need to secure a post-2012 agreement. If international actions continue on the same track, supplemental resources will be required in future years for this component area.

The evaluation found that the DFAIT contribution program was implemented in a timely and strategic manner, with full allocation of funds each year to date.

12. The HMARF is in place and facilitating Government-wide reporting of financial performance, but is not yet supporting decision making at the program level.

68

Page 86: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

The HMARF, and the supporting infrastructure of committees and the CAA–RMS, were established to provide clarity on management, reporting and accountability for the CAA. The International Actions Theme has a distinct Logic Model and performance measurement strategy under this HMARF. It has provided performance reports for the first two years (2007–08 and 2008–09) that are included in Canada’s Performance Report.

The HMARF has achieved its objective of facilitating Government-wide reporting of financial performance for the International Actions Theme. However, in the first few years, it has not yet achieved its other objectives of supporting decision making in the International Actions Theme or providing useful performance information (only the financial component is used by managers).

In addition, the evaluation found that the International Actions Theme is not managed or implemented as a “theme” as originally articulated. Instead, individual program components are managed separately (with overall coordination of climate change areas), with an operational DGMC that oversees activities beyond the management of individual programs. The inclusion of the PM Annex in this theme is considered a misfit with the other climate change program areas, and is more closely aligned with the domestic CARA than this theme.

13. Performance data are collected and reported, but not fully across all program components, and are not used by managers.

While the evaluation found that performance data were used for reporting to senior management, Treasury Board and Parliament, there was no evidence of decisions made in the program areas based on this performance data. Managers emphasized that the performance framework and Logic Model for the International Actions Theme were too rigid, too time-consuming to complete, and not able to adapt to the evolving policy/directional changes that have affected the programs. In addition, the performance framework does not include all international climate change and air quality activities under way in the Government of Canada (e.g., work by the Canadian Forestry Service and Canadian International Development Agency, Agriculture and Agri-Food Canada), providing challenges in fully assessing Canada’s performance.

6 Recommendations

The following recommendations were developed based on evaluation findings and conclusions. These recommendations are directed to the responsible ADMs of the International Affairs Branch at EC, the Global Issues Branch at DFAIT, the Energy Sector at NRCan, the Industry Sector at IC and the Associate ADM of the Environmental Stewardship Branch at Environment Canada. They were developed to continue improving and advancing the CAA International Actions Theme.

Recommendation 1: EC, in consultation with other participating departments, should review and implement options for enhancing coordination and management of program components in order to improve the strategic direction and effectiveness for the Theme overall.

69

Page 87: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Although there is some overall coordination across departments, each program component within the International Actions Theme is essentially implemented and managed separately by the department(s) responsible. This theme is not managed or implemented as a “theme” as originally articulated. Therefore, areas for improving possible synergies and connections between the goals of program components and departmental mandates should be explored, and formal mechanisms should be put in place (e.g., regular senior-level meetings across participating departments) to ensure that shared objectives for this theme and the CAA overall are met.

Recommendation 2: EC should consider the costs and benefits of integrating PM Annex program activities with CARA activities.

Streamlining all air quality activities together within the Department can lead to a more coordinated and coherent approach to planning, completing and reporting on the results for initiatives related to PM. Under the existing CAA structure, the majority of air quality activities are funded by sources within CARA. Since PM Annex activities are more closely aligned with the domestic objectives for CARA than to international climate change activities, EC should explore the appropriateness of integrating PM Annex resources and activities with those of CARA. This integration should facilitate the coordination of all work related to PM at EC.

Recommendation 3: EC should improve the effectiveness of its engagement with provinces, territories and other domestic stakeholders for the PM Annex, by considering the most appropriate formal mechanisms for consultation that will provide strategic direction for the future. There was little evidence that the provinces, territories and other domestic stakeholders were engaged as part of the PM Annex’s negotiations process. Given the role that these groups play in implementing clean air and clean energy programs and technologies, regional partners and other levels of governments should be engaged to enhance opportunities and articulate region-specific issues and context to inform priorities and negotiating positions for the Annex. Because the Annex experienced some delays in implementation, it is imperative to develop effective processes that will enable the Annex to move forward efficiently toward achieving its expected outcomes in the future. EC should therefore review existing processes for consultations and consider formal mechanisms for implementation of consultations that will provide strategic direction for the Annex’s negotiating positions.

Recommendation 4: EC, DFAIT and NRCan should develop an overarching strategic framework to guide the selection of international climate change voluntary contributions to international bodies and programs.

Within the International Actions Theme, there are multiple and sometimes conflicting goals assigned to Canada’s voluntary (non-assessed) contributions to international bodies and programs, which makes it difficult to determine the cohesiveness of these contributions in meeting broader thematic objectives. While some program components allocate funds using specific selection criteria based on their specific objectives, some components allocate funding based on opportunity rather than proactively. Therefore, an overarching framework should be developed to guide the selection of the International Actions Theme’s voluntary contribution projects. This framework should articulate the overarching strategic priorities,

70

Page 88: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

criteria for selection, appropriate terms and conditions, and broad processes to review, select, approve and monitor/oversee contribution projects.

This recommendation’s aim is to ensure that all funds are allocated to those initiatives that can best achieve the Theme’s broad expected outcomes. This overarching strategic framework should apply to all voluntary contributions in the Theme, to enable a more integrated process for soliciting, reviewing and approving proposals based on common expected outcomes and priorities (e.g., GHG reduction potential, trade-development potential, or value in developing partnerships, or some combination of these or other objectives). The intention is to have a Theme-level framework and a set of priorities that govern all voluntary grants and contributions programs under the Theme. However, this framework is intended to guide, not replace, the component-specific processes that are already in place for program components.

To support international partnerships and negotiations, this contributions strategy should also consider the goal of developing strategic alliances with developing countries, particularly with LDCs and SIDS, to better understand the potential adverse effects associated with climate change (e.g., adaptation in SIDS and LDCs), thereby building on commitments to support the poorest and those most vulnerable to climate change. Engagement with other federal departments not included in the International Actions Theme (e.g., the Canadian International Development Agency) may be beneficial.

Recommendation 5: EC, NRCan, DFAIT and IC should consider how APP program activities and outputs can be used to provide trade opportunities for the development, diffusion and deployment of clean technologies. The initial portfolio of APP projects focused less on technology deployment and more on capacity building and technology research. However, opportunities for Canadian businesses to develop and promote clean technologies internationally are emerging as large developing countries articulate their climate change actions and technology requirements to reduce GHG emissions. In order to maximize these opportunities (e.g., enabling technologies for smart-grid development in countries looking to improve energy efficiency), federal departments should explore how the International Actions Theme’s voluntary contributions can be used to increase support for these emerging trade opportunities for Canadian businesses.

Recommendation 6: EC, DFAIT, NRCan and IC should develop and implement mechanisms that will ensure the ongoing collection of meaningful and accurate performance measurement data.

Given that this evaluation occurred early in program implementation, the achievement of immediate outcomes was assessed at this time. Ongoing performance measurement data are required to assess progress toward achievement of intermediate and final outcomes in subsequent evaluations. However, some program components under this theme do not have a clear strategy for collecting ongoing performance information. As well, the performance measurement data collected as part of the HMARF are not used by, or seen as useful for, program managers. Therefore, EC, in consultation with participating departments and working with CAA–RMS, should enhance the performance measurement strategy for this theme, including the development of specific, measureable performance indicators. A clear plan should then be developed for the ongoing collection of performance measurement data, and mechanisms should be established in all participating departments to maintain the

71

Page 89: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

accuracy and usefulness of these data for each of the program components under this theme.

For example, performance indicators should be developed and a mechanism should be implemented to collect and maintain these performance measurement data for APP-funded projects. While it is too early to assess project success, there should be a system in place that will capture outcomes for funded projects so that these project outcomes can be assessed against established APP objectives in the future. Because of the dynamic nature of the policy directions and context of some activities under the International Actions Theme, it is challenging to develop accurate and useful ongoing performance indicators. Flexible strategies for program-component-level performance measurement and reporting may be considered to account for changing policy directions, but a clear plan should be developed that describes how these data will be recorded and managed.

7 Management Response

This section outlines the management response to the evaluation recommendations. The Assistant Deputy Ministers (ADMs) of the International Affairs Branch at Environment Canada, the Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), the Energy Sector at Natural Resources Canada (NRCan), the Industry Sector at Industry Canada and the Associate Assistant Deputy Minister (AADM) of the Environmental Stewardship Branch at Environment Canada, accept the evaluation and all of its recommendations, and have provided a plan to implement the following management actions in response to the evaluation recommendations.

Recommendation 1: Environment Canada, in consultation with other participating departments, should review and implement options for enhancing coordination and management of program components to improve the strategic direction and effectiveness for the theme overall.

The ADM of International Affairs Branch at Environment Canada (EC) agrees with this recommendation.Environment Canada will work with the Department of Foreign Affairs and International Trade (DFAIT), Natural Resources Canada (NRCan) and Industry Canada to review and implement options for enhancing coordination and management of program components to improve the strategic direction and effectiveness for the theme overall. Meetings of all International Actions Theme Program Director Generals will be held every four months to facilitate communication across departments at the management level, provide strategic direction, discuss interconnections between programs, provide a forum for thematic issues and to review level of achievement towards the International Actions Theme goals

Timeline Deliverable(s) Responsible PartyMarch 31, 2011 Quarterly meetings of all

International Actions Theme Program Directors General (first meeting to be held in summer of 2010).

Director General, Climate Change International Directorate, EC

72

Page 90: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Recommendation 2: Environment Canada should consider the costs and benefits of integrating PM Annex program activities with Clean Air Regulatory Agenda (CARA) activities.

The ADM of the International Affairs Branch and the AADM of the Environmental Stewardship Branch agree with this recommendationParticulate Matter (PM) Annex activities are more closely aligned to the domestic objectives of the Clean Air Regulations Agenda (CARA) than to international climate change activities. Both the CARA and PM Annex programs’ existing funding will expire March 31, 2011. Senior management has considered the efficiencies gained from single-point reporting instead of reporting to two funding programs as well as the strong alignment with CARA-related domestic goals and concluded that work related to the PM Annex would be more effective and efficient if included in the CARA program. Initial consideration for future funding of CARA is underway and includes the PM Annex. The PM Annex will begin to officially integrate into the CARA program this fiscal year, while continuing to report through the International Actions Program.

Timeline Deliverable(s) Responsible PartyMarch 31, 2011 Integrate PM Annex

activities into CARA activities this year.

Director General, Strategic Priorities Directorate, Environmental Stewardship Branch, EC

Recommendation 3: Environment Canada should improve the effectiveness of their engagement with provinces, territories and other domestic stakeholders for the PM Annex by considering the most appropriate formal mechanisms for consultation that will provide strategic direction for the future.

The AADM of Environmental Stewardship Branch agrees with this recommendationThe content of the PM Annex will be based on the respective domestic policies to address air pollution in Canada and the U.S.

Provinces and territories were consulted formally on Canada’s proposed domestic approach in 2007 through the Environmental Planning and Protection Committee (EPPC) of the Canadian Council of Ministers of the Environment.

These consultations led to the formation of a multi-stakeholder process that included members of provincial and territorial governments, industry and environmental groups. A Canadian domestic approach to address air pollutants has been the subject of the work of this group over the last two years.

We will be working to further elaborate our approach (as will the U.S) and when both countries are in a position to engage in negotiations, an appropriate process for engagement, using existing modalities where possible, will be set up.

There is also provincial representation on the subcommittees set up under the Canada-US Air Quality Agreement that allows for direct participation in the process.

Timeline Deliverable(s) Responsible PartyTBD Domestic consultations Director General, Strategic

73

Page 91: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

take place prior to further negotiations on the PM Annex under an appropriate process.

Priorities Directorate, Environmental Stewardship Branch, EC

Recommendation 4: Environment Canada, Foreign Affairs and International Trade Canada, and Natural Resources Canada should develop an overarching strategic framework to guide the selection of voluntary contributions to international bodies and programs.

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT) and Energy Sector of Natural Resources Canada (NRCan) agree with this recommendationEC will, in close consultation with DFAIT and NRCan, work to create an overarching strategic framework to guide the selection of voluntary (non-assessed) contributions to international bodies and programs.

A strategic framework was developed to ensure that projects funded under the Asia Pacific Partnership program met defined objectives. The experience of developing an APP strategic framework will serve as a useful guide to development of the overarching framework.

The implementation of the framework will guide the allocation of contribution funds to recipients that will best contribute to achieving the program objectives.

The framework will be developed in this final funded year to ensure the balance of funds is distributed in the most effective manner.

Should a new program of international actions be funded beyond the current financial year that includes similar voluntary contributions, this framework could be used as the basis for managing the new funds.

Timeline Deliverable(s) Responsible PartySeptember 30, 2010. Develop a strategic framework Lead:

Director General, Climate Change International Directorate, EC

Support:Director General, Energy and Sustainable Development Directorate, DFAIT

Director General, Energy Policy Branch, NRCan

March 31, 2011 Test the framework using contribution funds for fiscal year 2010-11.

Lead:Director General, Climate Change International Directorate, EC

Support:

74

Page 92: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Director General, Energy and Sustainable Development Directorate, DFAIT

Director General, Energy Policy Branch, NRCan

Recommendation 5: Environment Canada, NRCan, DFAIT and Industry Canada should consider how program activities and outputs can be used to provide trade opportunities for the development, diffusion and deployment of clean technologies

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry Canada (IC) agree with this recommendationThe clean energy technology program has worked with Canadian private industry, many of which are Small and Medium Sized Enterprises (SMEs), on baseline-setting and research projects with some deployment of clean technology projects. SMEs account for 45% of gross domestic product (GDP), much of the economy’s growth, 60% of all jobs in the economy, and 75% of net employment growth52 As we become aware, through the climate change negotiation sessions and other relevant international processes covered by the program, of emerging trade opportunities related to climate change, we agree that we need to increase SMEs awareness of these opportunities in order to increase deployment of clean technology.

The Departments will create a plan for better communication of clean technology trade opportunities to SMEs and will implement some activities prior to funding termination on March 31, 2011. The plan can be used as designed if the program is continued or as a foundation if the program is continued with changes.

In addition to ensuring information sharing with the Canadian SMEs, we can better inform the international community of Canada’s developing clean technology products to increase trade opportunities. For example, participation in the APP has resulted in sharing of technology information and products among the APP countries thereby increasing world awareness of Canada’s clean technology.

Established in 2009, and not included in the scope of this Evaluation, EC leads the Canada-U.S. Clean Energy Dialogue (CED), with support from NRCan and DFAIT. The CED aims to enhance bilateral collaboration in advancing clean energy technology solutions to reduce greenhouse gas emissions and address climate change. The focus areas of the CED include: expanding clean energy research and development; advancing the development and deployment of clean energy technologies, with a focus on carbon capture and storage; and building a more efficient electricity grid based on clean and renewable generation.  EC, NRCan and DFAIT will continue to work together under the CED, and where appropriate, engage with other relevant departments, such as IC, to advance the development and deployment of clean energy technologies. A result of this program could be an increase in

52 Public Works and Government Services Canada, “Importance of SMEs.” Accessed online on 21 May 2009 at http://www.tpsgc-pwgsc.gc.ca/app-acq/pme-sme/importance-eng.html.

75

Page 93: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Canadian business trade opportunities in clean technologies, although this is not a specific goal of the program.

Beyond the International Actions Theme, NRCan leads a number of key partnerships with organizations such as the International Energy Agency, the Climate Technology Initiative and the Global Carbon Capture and Storage Institute (GCCSI), which contribute significantly to advancing private sector engagement and should be further leveraged in order to maximize support for Canadian business opportunities on clean technologies.

Also beyond the Theme, Industry Canada has been participating actively in the International Partnership for Hydrogen & Fuel Cell in the Economy (IPHE) and working with Canadian companies to build hydrogen and fuel cell industry worldwide. IC is also engaged in the Steel Committee of the Organization for Economic Cooperation and Development (OECD) and the North American Steel Trade Committee (NASTC). While neither the Steel Committee of the OECD nor the NASTC is a forum that is meant to deal with climate change issues, they are often raised by industry representatives and IC officers as being important. These issues will no doubt affect the sector's competitiveness going forward because of different environmental obligations for developed and developing countries

Timeline Deliverable(s) Responsible PartyOctober 31, 2010 NRCan, IC, DFAIT and

EC representatives will develop an action plan to increase the Canadian SMEs’ awareness of increased opportunities in clean technology

Lead:Director General, Climate Change International Directorate, EC

Support:Director, Resource Manufacturing and Value-Added, Industry Sector, IC Director, Partnerships, Climate Change International Directorate, EC

Director, International Affairs, NRCan

Director, Climate Change and Energy Division, DFAIT

March 31, 2011 Implement activities identified in the action plan.

Lead:Director General, Climate Change International Directorate, EC

Support:Director, Resource Manufacturing and Value-Added, Industry Sector, IC

76

Page 94: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Director, Partnerships, Climate Change International Directorate, EC

Director, International Affairs, NRCan

Director, Climate Change and Energy Division, DFAIT

Recommendation 6: Environment Canada, DFAIT, NRCan and Industry Canada should develop and implement mechanisms that will ensure the ongoing collection of meaningful and accurate performance measurement data.

The ADMs of International Affairs Branch at Environment Canada (EC), Global Issues Branch at the Department of Foreign Affairs and International Trade (DFAIT), Energy Sector of Natural Resources Canada (NRCan) and the Industry Sector at Industry Canada (IC) agree with this recommendationAs mentioned in the Evaluation Report, gathering meaningful performance measurement data for policy-related outcomes is extremely difficult and must be qualitative in nature rather then quantitative. We recognize that we need better measurement of outcomes for the policy-related components of the program and EC will, in close cooperation with DFAIT and NRCan, enhance the performance measurement framework to be more flexible regarding the policy-related components of the International Actions Theme and establish a method of tracking performance for use in decision making.

The APP and Methane to Markets components of the program provide an opportunity to track performance data against program goals (reductions in greenhouse gases, economic benefits for Canadians and accelerating development of clean technology). On project completion, the companies will be asked to report on the achievements of their projects against the three priorities.

Timeline Deliverable(s) Responsible PartySeptember, 2010 Establish a system to

track performance of policy-related activities under the program against qualitative performance measurement indicators.

Lead:Director General, Climate Change International Directorate, EC

Support:Director General, Environment, Energy and Sustainable Development Bureau, DFAIT;

Director General, Energy Policy Branch, NRCan

February, 2011 Establish a system to report level of Director General, Climate

77

Page 95: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

achievement of meeting the three priorities of the APP program.

Change International Directorate, EC

78

Page 96: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

ANNEX 1 – Evaluation Framework

Question What should be observed Indicators Sources/MethodsRELEVANCE

Issue 1: Continued Need for Program 1. Are program components

within the International Actions Theme connected with key international climate change and air quality needs?

Activities under the International Actions Theme are connected with key international climate change and air quality needs

Evidence that International Actions Theme programs adapt with the emergence of relevant scientific evidence on air quality and climate change

Evidence that International Actions Theme programs adapt to changes in government policy and priorities

Evidence that the contribution program supports international objectives*

Document or File Review

Key Informant Interviews

Issue 2: Alignment with Government Priorities 2. Are program components

within the International Actions Theme aligned with federal government priorities?

Alignment with federal government priorities

Extent to which strategic directions within International Actions Theme match federal government priorities

Document or File Review

Key Informant Interviews

3. Are program components within the International Actions Theme aligned with the priorities of participating departments?

Alignment with the departmental priorities of EC, DFAIT, NRCan, and IC

Extent to which strategic directions within International Actions Theme match priorities of each of the participating departments

Document or File Review

Key Informant Interviews

Issue 3: Alignment with Federal Roles and Responsibilities 4. Is there a federal role and

responsibility for the government of Canada to deliver program components within the International Actions Theme?

Activities under the International Actions Theme are connected with demonstrated federal role and responsibilities

Demonstrated evidence of federal role and responsibility in the area of international actions for climate change and air quality

Absence/presence of similar programs within the federal and/or provincial governments

Extent to which similar programs complement/duplicate one another

Document or File Review

Key Informant Interviews

79

Page 97: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/MethodsPERFORMANCEIssue 4: Achievement of Expected Outcomes 5. To what extent have intended

outputs been achieved as a result of programs within the International Actions Theme?

Demonstrated achievement of the following outputs:

Document or File Review

Key Informant Interviews a. Contributions to international

bodies Evidence of payment of Canada’s

assessed contributions to UNFCCC Evidence of participation in

UNFCCC expert groups, workshops and other events

Evidence of the level of support and impact of non-assessed contributions on enhancing Canada’s position abroad on its objectives*

b. National Registry for CO2 emission units

Extent to which National Registry is established and maintained

80

Page 98: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/Methodsc. Formal and informal consultations

with domestic stakeholders Number of consultations with

Environment NGOs prior to UNFCCC sessional meetings

Number of consultations with industry prior to UNFCCC sessional meetings

Number of consultations with federal, provincial and Aboriginal organizations prior to UNFCCC sessional meetings

Evidence of domestic consultations held prior to entry into APP

Identification of opportunities for collaboration (enhanced networking at international meetings/conferences) through the approval of non-assessed contributions.*

Increase in the number of potential non-assessed contribution recipients.*

d. Policy and research papers, negotiating positions and advice

Research reports, advice and policy support provided to key negotiators

Evidence of facilitating research, studies and communication of national and international clean issues

e. Negotiations under UNFCCC Evidence of decisions, conclusions, summaries, statements and workplans from UNFCCC meetings

81

Page 99: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/Methodsf. Partnerships and Processes

outside UNFCCC Evidence of creation of appropriate

governance and organizational structures to oversee APP activities and objectives

Evidence of creation of APP taskforce working groups

Evidence of clean technology through related agreements under APP

Evidence of decisions, conclusions, workplans, summaries, and statements from multilateral meetings and partnerships which support Canadian positions

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

Evidence of progress towards the achievement of the following intended immediate outcomes

Document or File Review

Key Informant Interviews

a. Canada has complied with its financial and other obligations under international climate treaties and agreements

Demonstrated evidence that Canada’s total annual assessed contribution to UNFCCC has been paid in timely fashion

Evidence of payment of supplementary contributions for additional UNFCCC initiatives

Submission of Initial Report of National Registry for CO2 emissions under the Kyoto Protocol

Evidence that Canada paid annual membership fees to REEEP

Evidence of clean technology transfer related agreement under M2M

82

Page 100: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/Methodsb. Increased consideration of

domestic stakeholders’ interests in the development of Canada’s negotiating positions

Increased awareness of domestic stakeholder interests when developing Canada’s negotiating positions on transboundary air pollution and international climate change

Evidence that the impact of non-assessed contributions is targeted strategically and is timely to identify potential stakeholders and beneficiaries*

c. Increased awareness of trade opportunities in climate-friendly technologies to reduce transboundary flows of air pollutants or to address international climate change

Evidence of increased awareness among Canadian APP Task Force-related industries

Evidence that reach has expanded and new networks created and strategic alliances formed through the non-assessed contributions*

d. Canada’s positions for a future international climate change agreement are advanced

Submissions are made to UNFCCC detailing Canada’s views on the post-2012 Climate Change Agreement.

Active participation at international workshops and meetings detailing Canada’s views on the post-2012 Climate Change Agreement.

Canada’s views are reflected in decisions, conclusions, workplans, summaries, and statements

Evidence that non-assessed contributions are for activities to reduce harmful environmental risks*

Evidence that the contribution program shows a return on investment to support international clean air objectives*

83

Page 101: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/Methods7. Have there been any

unintended (positive or negative) outcomes that can be attributed to the program areas within the International Actions Theme? Were any actions taken as a result of these?

Observed unintended outcomes can be attributed to activities within International Actions Theme

Where appropriate, actions were taken to address unintended impacts

Presence/absence of unintended outcomes

Where appropriate, documented management actions and/or lessons learned from unintended outcomes

Document or File Review

Key Informant Interviews

8. Are there any external factors outside the International Actions Theme that influence the success of its programs?

Factors external to the International Actions Theme that have influenced the achievement of results

Evidence of factors outside the International Actions Theme which have influenced the achievement of outputs and early outcomes

Where appropriate, documented management actions to address the influence of external factors

Document or File Review

Key Informant Interviews

Issue 5: Demonstrated Efficiency and Economy9. Are there more economic and

efficient means of achieving objectives under the International Actions Theme?

Alternative design/delivery methods that are less costly/effort-intensive, enable quicker achievement of results, and /or enable achievement of better results

Opinions of program managers, deliverers, partners and stakeholders on the ability of program elements to achieve intended results, compared to alternative design/delivery approaches

Opinions of program managers, deliverers and partners on whether investments under International Actions Theme are a good use of public funds and whether the cost of producing outputs is as low as possible

Document or File Review

Key Informant Interviews

10. How could the efficiency of programs under the International Actions Theme be improved?

International Actions Theme activities show opportunities for increases in efficiency

Opinions of program managers and deliverers on how the efficiency of programs under the International Action Theme could be improved

Document or File Review

Key Informant Interviews

11. To what extent have each of the International Actions Theme program areas been

Program implementation is consistent with defined approach

Evidence that the International Actions Theme is being implemented as designed

Document or File Review

Key Informant

84

Page 102: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

Question What should be observed Indicators Sources/Methodsimplemented, or are they on track to being implemented, as planned and on time?

Evidence that program objectives have/have not been met within anticipated timeframe and/or budget

Barriers to program implementation are identified

Barriers to program implementation are resolved

Evidence that the contribution program implements funding agreements in a timely and well-managed manner*

Evidence that applications to the contribution program are reviewed in a timely manner*

Interviews

12. Is the management and accountability structure for the International Actions Theme in place and functioning adequately to achieve expected results?

Management structure for activities is defined, implemented and functioning effectively, including accountabilities, responsibilities and roles of all participating departments

Expected outputs and outcomes are clear and understood internally and externally

Overlap/duplication of effort is minimized and no gaps in programming exist

Opinions of program managers, deliverers and partners

Evidence of decisions being made Evidence that information, results

and decisions are being communicated to the appropriate parties

Document or File Review

Key Informant Interviews

13. Is appropriate performance data being collected, captured and safeguarded? If so, is this information used to inform senior management/decision makers?

Required performance data collection systems are in place, where appropriate

Performance data are collected against outputs and is used to inform decision-making

Presence of populated performance data systems, where appropriate

Evidence of collection of performance data, as appropriate

Evidence of decisions being made based on performance information

Document or File Review

Key Informant Interviews

85

Page 103: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

ANNEX 2 – Summary of Findings

EvaluationIssue Achieved

Progress Made;AttentionNeeded

Little Progress;Priority for Attention

Not Applicable

Relevance:Issue 1: Continued need for the program

1. Are program components within the International Actions Theme connected with key international climate change and air quality needs?

Ö

Issue 2: Aligned to federal government priorities

2. Are program components within the International Actions Theme aligned with federal government priorities?

Ö

3. Are program components within the International Actions Theme aligned with the priorities of participating departments?

Ö

Issue 3: Program consistent with federal roles and responsibilities

4. Is there a federal role and responsibility for the government of Canada to deliver program components within the International Actions Theme?

Ö

Performance:Issue 4: Achievement of intended outcomes

5. To what extent have intended outputs been achieved as a result of programs within the International Actions Theme?

Ö

6. To what extent have immediate outcomes been achieved as a result of programs within the International Actions Theme?

Ö

7. Have there been any unintended (positive or negative) outcomes that can be attributed to the program areas within the International Actions Theme? Were any actions taken as a result of these?

Ö

8. Are there any external factors outside the International Actions Theme that influence the success of its programs?

Ö

86

Page 104: EVALUATION OF THE CLEAN AIR AGENDA ...€¦ · Web view1 Introduction 1 2 Background 2 2.1 Profile – International Actions Theme Program Components 2 2.2 Governance Structure for

Audit and Evaluation Branch Evaluation of the CAA International Actions Theme

EvaluationIssue Achieved

Progress Made;AttentionNeeded

Little Progress;Priority for Attention

Not Applicable

Issue 5: Demonstrated Efficiency and Economy

9. Are there more economic and efficient means of achieving objectives under the International Actions Theme?

Ö

10. How could the efficiency of programs under the International Actions Theme be improved?

Ö

11. To what extent have each of the International Actions Theme program areas been implemented, or are they on track to being implemented, as planned and on time?

Ö

12. Is the management and accountability structure for the International Actions Theme in place and functioning adequately to achieve expected results?

Ö

13. Is appropriate performance data being collected, captured and safeguarded? If so, is this information used to inform senior management/decision makers?

Ö

87