Evaluation in the Government of Canada
Robert Lahey
Senior Director, Centre of Excellence for Evaluation
Treasury Board Secretariat of Canada
16 October 2001
Centre of Excellence for Evaluation 2
1. Canada at a Glance2. Key Players in Evaluation3. Perspectives on the Past4. TBS Study 5. New Agenda: Results for Canadians 6. New Evaluation Policy 7. TBS Centre of Excellence for Evaluation8. Community Renewal: Competency Profile9. Resources
Outline of Presentation
Centre of Excellence for Evaluation 3
Sovereign parliamentary democracy
Population 31 million
Federation of 10 provinces and 3 territories
Areas of federal responsibility include: defence, criminal law, postal service, census, copyrights, trade regulation, external relations, money and banking, transportation, citizenship, and Indian affairs.
Canada at a Glance
Centre of Excellence for Evaluation 4
Government Accountability
Parliament of CanadaHouse of Commons
Auditor General(appointed)
-Independent audits of govt operations
Parliamentary Committees
PM and Cabinet
TB Minister
Treasury Board Secretariat(secretary appointed)-government policy- oversees spending
Public servants
All other ministers
Federal depts/agencies(DMs appointed)
-government operations- approved budgets
Public servants
Centre of Excellence for Evaluation 5
Treasury Board Secretariat (TBS) Government’s management board (financial,
management and human resources) Comptrollership function Centre of Excellence for Evaluation Policies and standards Capacity building Links evaluation and performance
measurement
Key Players in Evaluation
Centre of Excellence for Evaluation 6
Auditor General (AG)
Conducts independent audits of government operations
Produces periodic oversight reports on the conduct of evaluation
Promotes accountability and best practices
Reports directly to Parliament
Key Players in Evaluation
Centre of Excellence for Evaluation 7
Departments/Agencies Deputy Ministers (DMs) accountable for the
application of Evaluation Policy within their departments
Heads of Evaluation implement policy as per TBS standards and guidelines
Internal accountability and reporting to DMs External accountability and reporting to TBS and
Parliament
Key Players in Evaluation
Centre of Excellence for Evaluation 8
Graphical Interpretation of the Extent of Formal Evaluative Activity in the Federal Government
1960s 1970s 1980s 1990s 2000s
Perspectives on the Past: Activity Graph–
Inte
nsi
ty o
f ac
tivi
ty
+
Centre of Excellence for Evaluation 9
Growing demand for information to support increasingly complex and costly programs
New management systems created for financial administration and planning, programming and budgeting
Evaluation as a practice not yet formalized
Lessons Learned Need for formal evaluation increases as resources
become scarcer and the identification of priorities becomes more important
1960s
Centre of Excellence for Evaluation 10
Building of key infrastructure elements in departments
Treasury Board Evaluation Policy (1977) was the first formalized evaluation policy in Canada
Evaluation Policy centre created within the new Office of the Comptroller General (1978)
Lessons Learned Necessary, but not sufficient, conditions for an
effective evaluation system are: government investment and support; formalized policies and standards; and, leadership for capacity building
1970s
Centre of Excellence for Evaluation 11
High expectations for evaluations not always met Government-wide review led by Parliament
Lessons Learned Evaluation quality depends on an approach that
balances: timeliness, usefulness, methodological purity, client requirements and cost
1980s
Centre of Excellence for Evaluation 12
Evaluation capacity affected by general government downsizing
AG report (1993) indicated renewal of evaluation capacity needed
New Review Policy (1994) linked evaluation closer to internal audit
Lessons Learned Critical mass in capacity is required to ensure
evaluation remains credible, relevant and strategic
1990s
Centre of Excellence for Evaluation 13
TBS study pointed to uneven delivery of evaluation across departments
Strong support to reinvest in evaluation Commitment to linking evaluation to broader
accountability and reporting requirements Objectivity, not independence, needed
TBS Study (2000)
Centre of Excellence for Evaluation 14
Agenda to:
Improve government services and respond to demands for better value and transparency
Move from reporting on results to managing for results
Applied through a wide-reaching series of initiatives, including the new Policy on Evaluation (2001)
Perspective on modern management shared by the political level and the Public Service
New Agenda: Results for Canadians (2000)
Centre of Excellence for Evaluation 15
Re-emphasis on the need for evaluation to be strategic, comprehensive and systematic
Scope broadened to include programs, policies and broad initiatives
Provides guidance on standards
Evaluators are encouraged to work directly with managers to build evaluation into the “life cycle” of programs
Highlights links between evaluation and results-based performance measurement
New Evaluation Policy: Highlights
Centre of Excellence for Evaluation 16
Managers are responsible for the active monitoring of their programs
Government is committed to the public reporting of evaluations
Centre of Excellence for Evaluation created in TBS
New funding available for departments to build and meet future evaluation capacity
Highlights (continued)
Centre of Excellence for Evaluation 17
Six standards for evaluation: Evaluation planning and issues Competency Objectivity and Integrity Consultation and Advice Measurement and Analysis Reporting
Guidance based on ‘good practices’
New Evaluation Policy: Standards
Centre of Excellence for Evaluation 18
TBS Centre of Excellence for Evaluation (CEE)
Pro-active leadership role in the renewal of evaluation in Canada through:
Building evaluation capacity Communicating and networking Policy development, implementation and
evaluation Repositions and renews the evaluation function
across government Emphasizes partnerships and knowledge
management
Centre of Excellence for Evaluation 19
Draft report (2001) identifies two key attributes of successful evaluators:
Cognitive capacity Ability to communicate well
The right people: Respect diversity Work collaboratively and openly Recognize and diffuse conflict Demonstrate sensitivity, tact and empathy Care about professional practices and standards
Community Renewal: Competency Profile
Centre of Excellence for Evaluation 20
Resources: Evaluation Infrastructure Checklist
Cultural institutions are prepared to
divulge information managers trust that
assessments will be objective agencies are willing to be
reviewed managers have the courage
to make changes and implement recommendations
evaluation function is prepared to evaluate itself
relevant accountabilities have been clarified
Operational technical, professional and
financial resources are available
time is sufficient evaluation policies and
standards are in place business/strategic plans are
developed need for objectivity can be
met authority exists to oversee
evaluations and act on findings
Concise Version
Centre of Excellence for Evaluation 21
Treasury Board Secretariat of Canada (TBS) www.tbs-sct.gc.ca
TBS Centre of Excellence for Evaluation www.tbs-sct.gc.ca/eval
TBS, Guide for the Development of Results-based Management and Accountability Frameworks
www.tbs-sct.gc.ca/eval/pubs/rmaf-cgrr/rmaf-cgrr-01-e.asp
TBS, Evaluation Policy www.tbs-sct.gc.ca/pubs_pol/dcgpubs/TBM_161/ep-pe_e.html
Resources: Selected GoC Web Sites
Centre of Excellence for Evaluation 22
Robert LaheySenior DirectorCentre of Excellence for EvaluationTreasury Board of Canada Secretariat300 Laurier StreetOttawa, Ontario K1A 0R5CANADA
Telephone: (country code 1) – 613 - 957-2602
Fax: (country code 1) – 613 - 952-1782
Email: [email protected]
Contact Information