Accreditation of Canadian Medical Schools: History and the Current Practices

May 2010

Dr. Linda N. Peterson, Assistant Secretary

Committee on the Accreditation of Canadian Medical Schools

Canadian Medical Schools

17 Medical Schools in Canada

1-6 per province

Oldest 1854 Queen’s University School of Medicine

Newest 2005 Northern Ontario School of Medicine

Undergraduate Medical Education Programs and Distributed Campuses

Canadian Medical Schools are Accredited by Two BodiesAccreditation of undergraduate medical education programs

leading to the MD degree

Committee on Accreditation of Canadian Medical Schools (CACMS)

Liaison Committee on Medical Education (LCME)

Overview of the Accreditation of Canadian Medical Schools

Abraham Flexner 1910

Overview of the Accreditation of Canadian Medical Schools

• Flexner in 1910 (8/155 were Canadian)– Only 5 schools were identified as excellent,

how many were Canadian?

• LCME was created in 1942 and accreditation became regular for Canadian schools after 1952

• CACMS created in 1979 to reflect the difference in health care delivery

• Dual accreditation using a joint process

Accreditation

• A standards-based, peer-review process of continuous quality assurance / improvement of the function, structure, and performance of a medical education program

CMA AFMC AMA AAMC

Accreditation of Programs leading to the MD degree

CACMS and LCME

• Both Committees meet 3 times a year within a week or two of each other.

• LCME meets in Washington or Chicago, alternating between cities every year.

• The two Committees have a joint meeting once every three years.

CACMS and the LCME

• CACMS voting members include 4 AFMC, 4 CMA, 1 LCME (Co-chair LCME), 2 CFMS and 2 FMEQ, and 2 public members

• LCME voting members include 6 AAMC, 6 AMA, 1 CACMS (CACMS Chair), 2 student members (AAMC and the AMA), and 2 public members

CACMS representation on the LCME and its Subcommittees

• The CACMS Chair is an LCME member

• Subcommittee on Standards– Chair of CACMS and Assistant CACMS Secretary

attend Standards Subcommittee meetings

• Subcommittee on Policy– Secretary of CACMS attends Policy Subcommittee

meetings

CACMS-LCME Joint Process• Survey team has one LCME member

• Survey report documents the findings (one single report goes to both Committees)

• Committees deliberate separately and come to independent accreditation decisions

• Secretariats reconcile Committees’ decisions

• A single consolidated letter of accreditation

• Follow-up actions

• More on this later in the session

Secretariat Staff

Each Accreditation Committee has a Secretariat Staff– Former senior members of a Faculty of Medicine

with extensive experience in medical education, administration and accreditation

Secretariat Staff Roles 1 Secretariat Staff provide support to their respective committees- • Survey team training- surveyors, secretaries and chairs• Develop educational materials for schools, students and

surveyors• Organize and implement and participate in survey visits• Conduct fact-finding visits, mandated and requested

consultation visits

Secretariat Staff Roles 2

Secretariat Staff provide support to their respective committees-

• Document and communicate the decisions of the committees

• Organize systematic reviews of the standards and policies and procedures of the committees

• Provide consultative advice to schools on all matters related to accreditation

Secretariat StaffCACMS

Nick Busing, MD, Secretary, CACMS

Linda Peterson, PhD, Assistant Secretary

LCME

Dan Hunt, MD, MBA, Co-Secretary (AAMC) 2010-2011

Robert Sabalis, PhD, Assistant Secretary (AAMC)

Barbara Barzansky, PhD, MHPE, Co-Secretary (AMA) 2009-2010

Dan Winship, MD, Assistant Secretary (AMA)

CACMS and the LCME Secretariat can help schools

• CACMS and LCME members can NOT provide consultation

• The CACMS and LCME Secretariat do NOT participate in the accreditation meetings/decisions/or provide information on schools

• The CACMS and LCME Secretariat CAN provide consultation support

CACMS/LCME Standards

Functions and Structure of a Medical School129 standards (128 apply to Canadian

schools) in 5 sections:• Institutional Setting (IS) – 16 standards• Education Program Leading to the MD Degree (ED) - 49

standards• Medical Students (MS) – 39 standards• Faculty (FA) – 13 standards• Educational Resources (ER) – 12 standards

The CACMS-LCME Full Survey

• Canadian medical schools accreditation cycle is 8 years• Preparation for a full survey begins ~18 months before

the on-site visit • Three documents produced during a Self-study

preceding the onsite visit– Medical Education Database– Institutional Self-study– Student conducted Independent Student Analysis

• Survey Team reviews the program- Standards based• Survey Report• Review and Decision by the CACMS and the LCME

Accreditation standards are NOT negotiable

• CACMS/LCME accreditation surveys are SUMMATIVE

• CACMS/LCME accreditation should be used as leverage for change

• “Two year rule” CACMS and the LCME expect that any areas of noncompliance should be brought into compliance within two years

Three components of preparing for the Full Survey

• Completing the Medical Education Database

• Conducting the Institutional Self-Study (ISS)

• Independent Student Analysis

Medical Education Database 1• The database is the mirror image of the standards• Items contained in the database are linked to specific

standards and are used to judge compliance• Let’s look at some examples:

– ED-2– ED-31

The questions represent the elements of the standard and the data mentioned represent the evidence necessary to evaluate compliance with the standard.

A SINGLE BASE YEAR, i.e., the most recently completed academic year

Medical Education Database 2

• Each section should be completed by the person most knowledgeable about the corresponding topic

• Care should be take to ensure accuracy and consistency of data across sections of the database– ED-31 on mid-clerkship feedback, the clerkship forms

• Faculty member who is the Self-study coordinator is responsible for ensuring that the database undergoes a comprehensive review to identify missing items or inconsistencies in the reported information.

• The Self-Study Taskforce should review the database to identify areas of noncompliance

Institutional Self-studyMain objectives:• To certify that a medical education program meets

prescribed standards• To promote institutional self-evaluation and improvementBrings together representatives of the administration, faculty

(>100), student body(5) and other constituencies to:– Collect and review data about the medical school and its

educational program– Identify institutional strengths and issues requiring action– Define strategies to ensure that the strengths are maintained and

any problems are addressed.

Organizing and Conducting an Institutional Self-study

• Select the right people to be the Self-study (Faculty member) and logistics (administrative) coordinators– Organized, knowledgeable, respected, committed

• ISS Taskforce- select “true leaders” who can make things happen and have institutional memory (include alumni, university officials, partners, etc.)

Organizing and Conducting an Institutional Self-study

• Ideally, the database materials and the Self-study document will be written coherently and in the same “voice” throughout

• Complete an honest and complete self-appraisal• A “good survey outcome”: one in which the

school’s self-study findings and the survey team’s findings are similar

Independent Student Analysis

STUDENT-MANAGEDQUESTIONNAIRE TO ALL STUDENTS

STUDENT COMMITTEEANALYSIS OF DATA

INDEPENDENT STUDENT ANALYSIS: STRENGTHS AND CONCERNS + SUMMARY

Independent Student Analysis

• The student survey covers not only the teaching program but access and responsiveness of the faculty, quality of the facilities including technology, lockers, lounge and study space

• The survey team uses the Student Analysis and the AAMC Canadian Graduation Questionnaire (CGQ) which includes multiple factors, student support services, financial debt and data on mistreatment

• Poor student participation in the CGQ (incentives are OK) and/or the student survey may adversely affect the review of the program

Timeline for the Independent Student Analysis

• Process begins almost at the same time as the Self-study

• Co-chair model of one fourth year and one second year student is helpful for continuity and building corporate knowledge

• Survey development, implementation, collection and analysis takes time

• Executive summary and the full report should be ready ~5 months before the full survey

Before the Survey Visit • Ideally, the Self-study Taskforce has access to

the student report several months before the materials have to be submitted to CACMS/LCME survey team

• Schools can use the Survey Report Writing Guide- these are questions the survey team must answer and the evidence they have to use

• Faculty corrects as many areas of noncompliance as possible and creates effective plans to address those that remain

Self-study materials sent to the Survey Team

• The Medical Education Database along with the Institutional Self-Study (ISS) and the Independent Student Analysis should be sent to the full survey team about 12 weeks before the survey visit

Organizing the Survey Visit

• Holds are placed on the calendars of all key participants:• University chief executive• Dean, UGME, Student Affairs, and Admissions

Deans and senior leadership• Department chairs• Heads of affiliated hospitals• Course/clerkship directors• Career, academic, financial aid and personal

counsellors, student health and mental health staff members as appropriate

Organizing the Survey Visit 2

• In programs with more than one geographically separated campuses, it may be necessary to add members to the survey team and/or another day into the schedule

Organizing the Survey Visit 3• Sunday evening to Wednesday morning- full survey• Sunday evening to Tuesday morning- limited• “Snapshot” in time• Administrative (logistics) Self-study Coordinator in

Canadian schools works with the survey team secretary to develop the visit schedule

• Work on the schedule soon after the database is submitted

• No gifts of any type for the team

CACMS-LCME Survey Team• 1 team representing CACMS and LCME

– For Canadian Schools, team includes:• Chair appointed by CACMS• Secretary appointed by CACMS• Team member appointed by CACMS• Team member appointed by the LCME• Student (member of CACMS) – often but not always• Faculty fellow appointed by CACMS – often but not

always

Purpose of the Survey Visit• Survey team answers questions regarding compliance

with standards raised by the database, the Self-study, and the Independent Student Analysis

• Fill in gaps in information- follow the Survey Report Writing Guide

• Verify information and impressions from the three documents

• Have questions answered• Collect updated information

Survey Visit & Exit Conference• The survey team reviews compliance with all 128 standards• Meets with university, hospital and faculty leadership, course

and clerkship directors, students, department chairs, junior faculty, library and educational resource administrators

• Exit conferences with the dean and university leadership to review a list of findings (organized by the sections of standards)

• This list will be provided to the dean• The Exit Conference IS NOT THE FINAL WORD

After the Survey Visit: The Survey Report

• Team writes the one single survey report which documents the findings - linked to individual standards

• The report will begin with the summary of the team’s findings linked to specific standards

• This initial report is reviewed by the CACMS and the LCME Secretariats

• After incorporation of Secretariat comments, the report is sent to the dean for identification of any factual errors

After the Survey Visit Review the Survey Report• The dean reviews the survey report carefully

– The final report will become the formal record of the visit and will be used by the CACMS and the LCME as the basis for the accreditation decisions

• The dean provides timely feedback on any errors of fact in the report to the CACMS and LCME Secretariats

• If there is a disagreement with the “tone” of the report– Concerns are forwarded via the CACMS and LCME Secretariats– If these are not resolved, the concerns are documented in WRITING-

this letter will accompany the report on the CACMS and LCME meeting agendas

CACMS and the LCME Separately Decide Accreditation Status• CACMS and the LCME deliberate separately

and come to independent accreditation decisions– The final survey report is reviewed by 2 reviewers

from both Committees. The two reviewers come to consensus on a recommended decision; followed by Committee discussion and a Committee vote

• Secretariats reconcile Committees’ decisions

Possible accreditation decisions

• Full continuing accreditation for an eight-year term (with or without follow-up from the school)

• Full continuing accreditation with a reduced term.• Continuing accreditation with no fixed term pending

outcome of• A status report• A limited (focused) survey• A Secretariat fact-finding survey• A Secretariat consultation

• Place program on Warning of Probation• Place program on Probation (appealable)• Withdraw accreditation (appealable)• Deny accreditation (appealable)

CACMS/LCME Consolidated Letter of Accreditation• A process for reconciling accreditation decisions has been

developed by the two Committees and now covers the full range of actions

• Secretariats reconcile Committees’ decisions• All areas of noncompliance identified by each committee are

included• The decision about accreditation status is the most

conservative and the follow-up actions are the most comprehensive

• A single consolidated letter of accreditation

Citations – 2004-2008Canadian schools

Institutional Setting

4%

Educational Program

59%

Faculty6%

Medical Students

23%

Educational Resources

8%

Canadian Top 10 Areas of Noncompliance2004-2008

1. ED-2 Required clinical experiences and monitoring

2. ED-8 Comparability of educational experiences across instructional sites

3. ED-24 Residents’ preparation for their roles in teaching and evaluation

4. ED-30 Formative and Summative evaluation

5. ED-31 Mid-course and Mid-Clerkship Feedback

Canadian Top 10 Areas of noncompliance2004-2008

6. ED-33 Curriculum Management

7. ED-37 Monitoring curriculum content

8. ED-35 Systematic review of the curriculum

9. MS-32Student mistreatment

10.ED-1 Educational program objectives

New AFMC Interim Accreditation Review Process• AFMC Deans- Working Group on

Accreditation

• To assist the schools in achieving and maintaining compliance with accreditation standards

• Checklists and an Interim Survey

• FORMATIVE

AFMC Future of Medical Education in Canada• 10 recommendations for the direction of

medical education

• Accreditation is seen as a leverage for change

• AFMC Deans strongly support measuring school’s achievement of these recommendations – informal moving to a more formal process in the future

Resources• http://afmc.ca/education-cacms-e.php• http://www.lcme.org/pubs.htm

– Functions and Structure of a Medical School (Standards for Accreditation of Medical Education Programs leading to the MD degree)

– LCME Rules of Procedure– Guide to the Institutional Self-Study– LCME Medical Education Database– Guidelines for the Conduct of Accreditation Survey Visits– The Role of Students in the Accreditation of Medical Education Programs– Survey Report Writing Guide– Accreditation Issues related to Distance Learning– Guidelines for new and developing Medical Schools– Guidelines for Increasing Class Size in Medical Schools Accredited by the LCME or CACMS– Template for reporting a Proposed Class Size Increase– Template for reporting a Proposed new or Expanded Branch Campus to the LCME and

CACMS– Defining the scope of LCME Accreditation: Programs, Campus and Tracks– List of LCME-accredited medical schools in the US and Canada– AFMC Future of Medical Education in Canada