As presented to the Global Colloquium on Engineering Education

Deborah Wolfe, P.Eng.October 2008

The Canadian Process for Incorporating Outcomes

Assessment in the Accreditation Criteria

Today’s Presentation

BackgroundAccreditation Criteria, Policies and ProceduresOutcomes AssessmentImplementation of outcomes assessment

Canadian Engineering Accreditation Board

Created by Engineers Canada in 1965

Membership:» 15 volunteers, all ing./P.Eng.» Range of disciplines and backgrounds from across Canada» Rely on specialist volunteers during program evaluation visits

Main Responsibilities» Identify programs whose graduates are prepared to enter the

profession of engineering » Develop accreditation criteria, processes, procedures» Quality assurance » Continuous improvement

Background

Accreditation: Results

Over 250 accredited programs» 40 post-secondary institutions

» Over 70 fields of study

» 55,000 students

» 10,500 graduates per year

Background

Accreditation

Identify programs whose graduates are prepared to enter the profession of engineering

Undertaken only upon invitation by the university

Undergraduate engineering programs only

Programs are accredited, not departments or faculties

Accreditation, Criteria, Policies and Procedures

Two Components of Evaluation

A. Qualitative evaluation of the program environment

B. Quantitative and qualitative evaluation of the curriculum

Accreditation, Criteria, Policies and Procedures

Licensure

Dean, Department Chair, Program Coordinators, and faculty teaching courses which are primarily engineering science and engineering design are expected to be licensed professional engineers in Canada.

Curriculum development and control should be in the hands of licensed professional engineers

All faculty teaching engineering design should be licensed professional engineers in Canada and faculty teaching engineering science should be in the process of attaining licensure

Accreditation, Criteria, Policies and Procedures

Outcomes Assessment

Many accreditation systems started to move to outcomes assessment in the 1990’sABET led the movement with Engineering Criteria 2000 (EC2000)Washington Accord developed model graduate attributes in 2004-2005

Outcomes Assessment

What is ‘Outcomes Assessment’?

Outcomes: Statements that describe what students are expected to know and be able to do by the time of graduationAssessment: Processes that identify, collect, analyze, and report data that can be used to evaluate achievementProgram Objective: (ABET and others) Broad statements which describe what is expected of graduates a couple of years after graduation

Outcomes Assessment

Philosophy Create a set of desired graduate attributesInstitution sets objectives in consultation with constituenciesEnsure competent faculty, facilities, etcMeasure graduate attributes Require feedback to objectives and educational processEnsure feedback loop is operational

Outcomes Assessment

Observations on Outcomes Assessment

Outcomes assessment has theoretical appeal - especially if quality assurance is the objectiveIf each institution sets its own objectives, can outcomes assessment provide a “standard”? Most of the evidence used to demonstrate that outcomes have been achieved is really input rather than output-relatedA lot of institutional work is required to develop potential of outcome assessmentAssessment methods can and probably will improve with use There is a lot of subjective (if expert) judgment requiredOutcomes assessment is flexible

Outcomes Assessment

Criteria RevisionsPrevious criteria:– Prescriptive?– Restricts innovation, continuous improvement?– Not competency-based?

Revisions and reorganization requiredIntroduction of graduate attributes and outcome based assessment

Outcomes Assessment

Criteria Revision Process

The Accreditation Board worked with stakeholders (deans, regulatory bodies) for several years to revise the criteriaRecognized need to improve the synchronization of criteria, questionnaire and decision lettersThe goal is to be ready for Washington Accord monitoring in 2008-2009 ~ our international review

Outcomes Assessment

Criteria

New criteria start with attributes that align with Washington Accord modelReformatted to promote greater precision in decision lettersStrengthened licensing criteria at the request of the Constituent Members

Outcomes Assessment

Graduate Attributes

1. Knowledge Base for Engineering

2. Problem Analysis3. Investigation4. Design5. Use of Resources6. Use of Engineering Tools7. Individual and Teamwork8. Communication

9. Professionalism10. Impact on Society and the

Environment11. Ethics and Equity12. Engineering Economics

and Project Management13. Life Long Learning

Outcomes Assessment

Other Changes

Requirement for an articulated mission statement and strategic plan‘Basic Science’ changed to ‘Natural Science’Removed reference to a Bachelors degree (for international compatibility)Added explicit references for the requirement for documentation on authority for program and admissionsNarrowed and tightened licensure requirements

Outcomes Assessment

Next TasksDevelopment of outcome measurement tools, used to provide evidence that required attributes are being developed in the graduate in collaboration with Canada’s National Council of Deans of Engineering and Applied ScienceReview and, if possible, simplification of all documents, policies and procedures to focus on new criteriaDevelopment of training for accreditation visitors and institutions

Implementation of Outcomes Assessment

Criteria Consultation ProcessMajor workshops in September 2006 and 2007Criteria proposed and consultation undertaken by the Accreditation Board (2007)Criteria revised and approved by the Accreditation Board (February 2008)Approved by Engineers Canada (May 2008)Implementation (Fall 2009 visits)Transition: deficiencies first applied on graduate attributes, feedback loop and new quantitative requirements (2014)

Implementation of Outcomes Assessment

www.engineerscanada.caceab@engineerscanada.ca