atec 2015 - commission presentation-3 · •61 program directors • site visitor workshops •...

4/7/15

1

Assuring and Enhancing Quality Education in Athletic Training

ATEC 2015Dallas, TX

INTRODUCTIONS & OVERVIEW

Mark Laursen, MS, ATC CAATE President Boston University

ExecuAve CommiCee

Mark Laursen, MS, ATC President Boston University

Mark Merrick, PhD, ATC, FNATA President Elect The Ohio State University

Jack Weakley Secretary/Treasurer Public Member

Ray Castle, PhD ,ATC Louisiana State University

Chad Starkey, PhD, ATC Ohio University

AthleAc Training Members

Paula Maxwell, PhD, ATC James Madison University

LesLee Taylor, PhD, ATC Texas Tech University Health Sciences

Eric Sauers, PhD, ATC, FNATA A.T. Still University

Valerie Herzog, PhD, ATC Weber State University Commissioner-Elect

AdministraAve Member Physician Members

LaMont Cavanaugh, MD American Academy of Family Physicians Representative University of Oklahoma School of Medicine

Gregory Frazer, PhD Administrative Member Dean and Professor Rangos School of Health Sciences Duquesne University

Joseph Guettler, MD AOSSM Representative Oakland University William Beaumont School of Medicine

Bernard Griesmer, MD American Academy of Pediatrics Representative HealthTracks Center, Springfield, MO

T R A N S F O R M I N G T H E P R O F E S S I O N T H R O U G H Q U A L I T Y E D U C A T I O N

CAATE Office

Ashley Ahearn, MS, ATC AccreditaAon Assistant

Micki Cuppe?, EdD, ATC ExecuAve Director

Julie Cavallario, MS Ed, ATC AccreditaAon Assistant

4/7/15

2


Overview

•  Analytics •  Public Accountability & Transparency •  Quality Improvement Current•  Quality Improvement Future•  Questions•  Examining the Professional Degree•  Public Comments ANALYTICS

Mark Merrick, PhD, ATC, FNATA CAATE President-‐Elect The Ohio State University

Current Number

of Programs

Average Total Annual

Graduates

Percentage of

Graduates who took BOC Exam

Average Graduates per program

(Exam Candidates)

Program Average 1st A?empt BOC Exam Pass Rate

Program Average Any-‐

A?empt BOC Exam Pass Rate

Programs Non-‐

Compliant with

Standard 11

Percentage of all

Programs Non-‐

Compliant with

Standard 11

365 3629.7 94% 10.1 (9.5) 79% 90% 96 26%

Current Three Year GraduaAon and BOC Exam Totals (2012-‐14)

* Data exclude new programs with < 3 yrs data (N=4)

0

20

40

60

80

100

120

140

0-‐9%

10-‐19%

20-‐29%

30-‐39%

40-‐49%

50-‐59%

60-‐69%

70-‐79%

80-‐89%

90-‐100%

1 1 8 5 9 27

45 58

82

125

Num

ber o

f program

s per ra

nge

1st A?empt Pass Rate Range (3yr avg)

1st A?empt BOC Exam Pass Rate ALL PROGRAMS:

2012-‐2014 exams (current)

Majority of students in these programs fail 1st aCempt at board exam, N=24

30 of 45 are at 65% pass rate or beCer

0

50

100

150

0-‐9% 10-‐19% 20-‐29% 30-‐39% 40-‐49% 50-‐59% 60-‐69% 70-‐79% 80-‐89% 90-‐100%

1 1 8 5 9 27 45 58

82

125

Num

ber o

f program

s per ra

nge




0

50

100

150

0-‐9% 10-‐19% 20-‐29% 30-‐39% 40-‐49% 50-‐59% 60-‐69% 70-‐79% 80-‐89% 90-‐100%

3 2 5 9 10 26

42 55

90 123

Num

ber o

f program

s per ra

nge



2011-‐2013 exams (1 year ago)

New Programs 4

Non-‐compliant 96

Compliant 265

Standard 11 Compliance 2012-‐14: All Programs (count)

4/7/15

3

New Programs

3

Standard 11 Compliance 2012-‐14:

Bachelor's Programs (count)

Compliant 237

Non-‐compliant 96

New Programs

1

Non-‐compliant

1

Standard 11 Compliance 2012-‐14:

Master's Programs (count)

Compliant 28

For 2012-‐14, Bachelors programs are 13.2 Ames more likely to be non-‐compliant on Standard 11 than Masters programs

0

50

100

150

0-‐9% 10-‐19% 20-‐29% 30-‐39% 40-‐49% 50-‐59% 60-‐69% 70-‐79% 80-‐89% 90-‐100%

1 1 8 5 9 27

44 56 81

100

Num

ber o

f program

s per

range


1st A?empt BOC Exam Pass Rate BACHELOR PROGRAMS:


0

50

100

150

0-‐9% 10-‐19% 20-‐29% 30-‐39% 40-‐49% 50-‐59% 60-‐69% 70-‐79% 80-‐89% 90-‐100%

0 0 0 0 0 0 1 2 1 25

Num

ber o

f program

s per

range


1st A?empt BOC Exam Pass Rate MASTER'S PROGRAMS:


*30% of bachelor’s and 86% of master’s programs had 1st aCempt pass rates >90%

Number of Programs

Average Total Annual

Graduates

Average Percentage

of Graduates who took BOC Exam

Average Exam

Candidates per

Program

1st A?empt BOC Exam Pass Rate

Any A?empt BOC Exam Pass Rate

Number of Non-‐

Compliant with

Standard 11

Percentage of Non-‐

Compliant with

Standard 11

Bachelors 335 3336.0 94% 9.4 78% 89% 95 28%

Masters 30 293.7 98% 9.9 93% 98% 1 3%

Total 365 3629.7 94% 9.5 79% 90% 96 26%

BOC Exam Performance By Program Type

*Professional programs average 10.1 graduates per year. 17% of programs graduate 5 or fewer per year and 57% of programs graduate 10 or fewer per year.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0

1st A

?em

pt Pass R

ate (3yr avg)

Avg. Annual Program Graduates

2012-‐14 1st ACempt BOC Exam Pass Rate v. Annual Graduates

PUBLIC ACCOUNTABILITY & TRANSPARENCY

Eric Sauers, PhD, ATC, FNATA CAATE Commissioner A.T. SAll University


CHEA Recognition

•  CHEA Board action September 30, 2014

•  Have interim report due July 1, 2017

4/7/15

4


Key Issues in Higher Education Accreditation

•  Quality Assurance AND Quality Improvement

•  Reducing costs and regulatory burden of accreditation

•  Consumer protection via increased transparency and accountability


Important Policies Impacting Higher Education Accreditation

•  Reauthorization of the Higher Education Act

•  National Advisory Committee on Institutional Quality and Integrity

•  College Rating System

•  Gainful Employment Rule

•  The Supporting Academic Freedom through Regulatory Relief Act


Implication for the CAATE and AT Education

•  Focus on outcomes to reduce the regulatory burden

•  Greater emphasis on quality improvement•  Increased consumer protection through greater

transparency and accountability•  Consider key metrics of rating: access, cost, and

outcomes•  Mindful of gainful employment mindset

QUALITY IMPROVEMENT -‐ CURRENT

Micki CuppeC, EdD, ATC ExecuAve Director


Quality Improvement eAccreditation

•  All Substantive Changes now done electronically–  Request for degree change–  Program director change–  Change in college/school

•  Benchmarking now available•  CAATE Analytics Report

4/7/15

5

Benefits of A CAS

Programs •  Free parAcipaAon •  Exposure to wider

applicant pool •  Benchmarking vs.

naAonal or idenAfied cohorts

•  Maintain individual workflow and admissions standards

Applicants •  Single online

applicaAon •  Easy, intuiAve interface •  24x7 mobile access •  Real-‐Ame updates •  One point of contact

for support

AssociaAon •  Real-‐Ame data &

common standards •  Accurate staAsAcs on

professions •  Assists with workforce

projecAons, future trends, and recruitment strategies


Workshops•  2014 Conference a success

– Over capacity– 125 participants

•  6 Deans •  15 Department

Chairs•  61 Program Directors

•  Site Visitor Workshops•  District workshops


Accreditation Conference 2015

•  October 16-18, 2015•  Hilton Tampa Downtown•  Larger space with room for more attendees•  2 ½ days of programming geared toward program personnel and

administration

QUALITY ASSURANCE IN THE SITE VISIT PROCESS

Leslee Taylor, PhD, LAT, ATC CAATE Commissioner Texas Tech University Health Sciences Center


Site Visitor Training

Focus of the site visit teams, CAATE Committees, & Commission is to look for compliance NOT noncompliance

Training•  Site Visitor Training & Administrator’s Session

–  ATEC – ~35 site visitors & 26 Administrators–  St. Louis– June 22-23, 2015

•  Site Visitor Chair Training–  CAATE Accreditation Conference October 2015


Site Visit Purpose

•  Validate the self-study report and compliance with the appropriate CAATE Standards

•  To aid CAATE in reaching an accreditation decision that is defined, consistent, free of personal biases, free of conflicts of interest, free of non-sanctioned interpretations, and upholds respect for institutional autonomy.

4/7/15

6


Site Visit ProcessOnce Program Submits Documents via eAccreditation

Site Visitor Selection & Notification•  Site Visit Committee makes SV team assignments

–  Size of Institution/Carnegie Classification, –  Conflict of Interest (Site Visitor & Institution), etc.

•  Three person team: –  SV Chair–  SV Team Member–  SV Reader

•  Within one week of receiving the SV assignment, the SV chair must make contact with the program and team members

–  Please let the CAATE office know if you do not hear from your SV chair.


Site Visit ProcessReview of Self Study•  Pre-Site Visit Module in eAccreditation

–  Each site visitor independently indicates a preliminary degree of compliance for each standard

•  If needed, the SV chair will contact the program to request any additional materials or to clarify areas of question

–  Via eAccreditation by reopening specific standards

Site Visit Agenda •  All site visits will use the 3 day agenda

–  Agenda lengthened to allow adequate time meet with everyone and visit clinical sites unrushed. Change made based on feedback from programs & site visitors

–  On-going evaluation based on feedback


Site Visit Process

On-Site Visit•  Preliminary Conference

•  Standards are the minimum requirements that a program must meet

•  Meet with the program director daily to keep him/her

apprised of concerns, areas of non-compliance, etc.

•  Exit Conference


Site Visit ProcessSite Visit Report•  Introduction – Brief History of the Program•  Strengths – Positive Aspects of the Program•  Standards – Report areas of non-compliance with a rationale

–  If non-compliance cited, indication that the SV team cannot find compliance; the rejoinder allows the institution to demonstrate compliance

•  Recommendations – Suggestions that might strengthen the program

•  Individuals interviewed – List name, credentials, and role in program


Quality AssuranceThe CAATE is striving to improve the SV process and increase the quality of its site visits. An integral part of our quality assurance process is obtaining valid feedback from all individuals (peer, program, and review team) involved in the site visit process.

• Cumulative data will be collected and sent to all site visitors. – Allows the site visitor to see his/her status compared to others – Aids Site Visit Committee in recommending retention and training

•  Evaluation includes professionalism, knowledge of Standards, and preparedness for visit (i.e. fair and objective, maintained confidentiality, unbiased, and appropriately dressed).

•  Only as good at the feedback we receive about the process and site visitors.


Site Visit Numbers*

* All numbers beginning with 2015-‐2016 subject to change un;l documents submi=ed.

2014-‐2015

2015-‐2016

2016-‐2017

2017-‐2018

2018-‐2019

ConAnuing AccreditaAon

Professional (Bac & Post Bac) 22 29 27 52 83

Post-‐Professional Degree 0 5 5 4 -‐-‐

Residency 0 0 0 2 -‐-‐

SubstanAve Change – Level of Degree

Professional 6 8 5 3 -‐-‐

Post-‐Professional Degree 0 1 1 -‐-‐ -‐-‐

IniAal AccreditaAon

Professional (Bac & Post Bac) 4 12 -‐-‐ -‐-‐ -‐-‐

Post-‐Professional Degree 0 -‐-‐ -‐-‐ -‐-‐ -‐-‐

Residency 3 -‐-‐ -‐-‐ -‐-‐ -‐-‐

Total Number of Visits 35 55 38 61 83

4/7/15

7


Substantive Change – Level of Degree•  Purpose

–  The Substantive Change Process is designed for Programs that are mid‐accreditation cycle and wish to voluntarily change the level of the without completing an entire self-study. (Professional & Post-Professional Degree)

•  Eligibility–  This process is available annually to programs that are mid-

accreditation cycle and are in good standing (i.e., not on probation).

•  Timeline–  MAY 1st – Notification of Intent & review fee are due–  AUGUST 1st - Substantive Change Application & Documents are

due

QUALITY ASSURANCE IN THE REVIEW PROCESS

Paula Maxwell, PhD, ATC CAATE Commissioner James Madison University

Review CommiCee •  Made up of non-‐CAATE Commissioners

–  Review CommiCee Chair –  Review Team Coordinators –  Review Team Members (3-‐4 per team) –  Three teams for Professional Programs; Two teams for Post Professional

Programs •  Liaison from the Commission to the Review CommiCee

Review Commi?ee Chair

Review Team #1

Review Team #2

Review Team #3

Review Team #4

Review Team #5

Commissioner Liaison to the RC

CAATE Commission

Roles of Review Commi?ee Members

•  Commissioner Liaison: –  Resource person

•  Review Commi?ee Chair: – Organizes and tracks reports – Assigns reports to teams according to Conflicts of Interest

–  Resource person for the Review Teams –  Reviews all reports for consistency and accuracy once RT finish

–  Communicates with the CAATE and the Standards CommiCee

–  Completes annual assessment of NCs to suggest areas for change or clarity


Roles of Review Committee Members•  Review Team Coordinator:

–  Organizes team and reports within the individual team–  Tracks reports and maintains timelines–  Communicates with the Review Committee Chair and

other team Coordinators–  Helps review reports with team members

•  Review Team Members:–  Review site visit reports, rejoinders and progress

reports–  Communicate with the site visitors

Review Process Site visit report/rejoinder/progress report submiCed via

eAccreditaAon

Report checked for typos, grammaAcal errors, proper format, completeness

Review CommiCee Chair and Review Team(s) noAfied that report is ready

Review Team Reviews the report Call SV Team if SV Report

Review CommiCee Team #2 reviews report if complicated, has potenAal negaAve outcomes or if team members are not in agreement; Release back to RC Chair

for final review

Review CommiCee Chair reviews for accuracy, consistency, and appropriateness of NCs – Returns report

to CAATE Office via eAccred

Report forwarded to Commission for acAon

4/7/15

8

•  The Review Team analyzes each Standard to determine its level of compliance. If a Standard is deemed “non-compliant,” the Review Team will ensure there is adequate evidence and sufficient description detailing why the Standard is non-compliant and what must be submitted to demonstrate compliance.

•  Review process takes 6-8 weeks to complete•  Multi-step process with several evaluators to gain accuracy and

consistency•  No official NCs until the review process is complete

–  Unless health and safety, discrimination or fair practices are involved, the Commission does not see or act on the Site Visit report until after the program has an opportunity to address the NCs (i.e. submit the Rejoinder).

Review Process


How to Get Involved

•  Call for new members this spring

•  Training in October –  In conjunction with Accreditation Conference

•  You are encouraged to apply

QUALITY IMPROVEMENT -‐ FUTURE



CAATE Vision

Improving Health by assuring and recognizing excellence in AT education.


eAccreditation Enhancements in the Future

•  Focus Group from all Users–  Program Administrators, Site Visitors, Review Team,

Annual Report

•  Examine the platform to make the process more robust and user friendly


Encouraging and Recognizing Excellence

•  Creating standards and a process that encourages creativity and excellence.

•  Create an environment that encourages innovative and new curricula that prepare athletic trainers for our future place in health care.

•  Provide a purposeful continuum of education.

4/7/15

9


Creating an educational format which produces graduates that……

•  Have a strong scientific foundational knowledge.•  Are clinically competent.•  Want athletic training as a career.•  Have a place in healthcare as a Level II

Provider.•  Possess The Institute of Medicine Core

Competencies for Health Professionals


Currently, the Commission believes:

•  Standards should be continually examined

•  Required professional knowledge needs to change as medical practice and needs change.

•  There should be periods of full-time clinical engagement

•  Programs should have faculty with specific areas of expertise

•  AT programs should be housed within schools of health professions

Questions EXAMINING THE PROFESSIONAL DEGREE



Examining the Professional Degree

•  Currently, the Commission believes:–  Compelling evidence indicates that better outcomes

are being achieved at the masters level –  The elements of quality improvement we have

identified will be easier to implement at the masters level

–  It is important to facilitate programs who want to transition their program to the post-baccalaureate level


The Commission also believes…..

•  To accomplish our mission, changes need to

continue to be made in professional education

regardless of the level of the degree.

4/7/15

10



•  Commitment to an objective evaluation and transparent process are warranted prior to making any decision. •  Online solicitation of stakeholder feedback

–  Current open comment •  Evaluation of information as it becomes available•  Collaboration with our strategic partners



•  Electronic open comment –  Began February –  Ends March 20–  # Comments to date =

•  Discussing at meeting next week•  Continue conversations with strategic partners

CommentsATEC 2015Dallas, TX


Accreditation Conference 2015

•  October 16-18, 2015•  Hilton Tampa Downtown•  Larger space with room for more attendees•  2 ½ days of programming geared toward program personnel and

administration