updates from the residency review committees for internal medicine & pediatrics
DESCRIPTION
Updates from the Residency Review Committees for Internal Medicine & Pediatrics. MPPDA Meeting April 10, 2014 Caroline Fischer, MBA, Executive Director, RC- Peds Jerry Vasilias, PhD, Executive Director, RC-IM. What’s new…. NAS is here… Brief summary of NAS Highlights from last RC meeting - PowerPoint PPT PresentationTRANSCRIPT
Updates from the Residency Review Committees for Internal Medicine & Pediatrics
MPPDA MeetingApril 10, 2014
Caroline Fischer, MBA, Executive Director, RC-PedsJerry Vasilias, PhD, Executive Director, RC-IM
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
What’s new…
• NAS is here… • Brief summary of NAS• Highlights from last RC meeting• Workflow Changes• Highlights from Policy and Procedures Manual• Citations and non-citations• Milestones• Recent news
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
One step back…
© 2013 Accreditation Council for Graduate Medical Education Information Current as of December 2, 2013
Why NAS?
• To achieve promise of outcomes-based accreditation • Annual review of programs to identify “problem
programs” to help them improve• Reduce the burden of accreditation
Some key elements of NAS:• Most data in NAS already in place• Annual ADS data entry replaces PIFs• Self-studies every 10 years• Site visited only when “issues” arise• “Internal Reviews” no longer required• Programs in good standing can innovate with “detail” PRs
• Significant % of common and specialty PRs are “detail”
• Citations, Areas for Improvement
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
The Data Elements of NAS…
The following are the primary annual data elements:
1) Program Attrition2) Program Changes3) Scholarly Activity4) Board Pass Rate 5) Clinical Experience Data6) Resident Survey7) Faculty Survey8) Milestones
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Spotlight: Clinical Experience Data
The following are the primary annual data elements:
1) Program Attrition2) Program Changes3) Scholarly Activity4) Board Pass Rate5) Clinical Experience Data6) Resident Survey7) Faculty Survey8) Milestones
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Annual Data Element #5:Med-Peds: Clinical Experience Data
• Composite variable on perceptions of clinical preparedness.
• How measured: 4th year residents’ responses to RS • IM:
• Adequacy of clinical and didactic experience in IM, subs, EM, & Neuro
• Variety of clinical problems/stages of disease?• Experience w patients of both genders and a broad age range?• Continuity experience sufficient to allow development of a
continuous therapeutic relationship with panel of patients• Ability to manage patients in the prevention, counseling,
detection, diagnosis and treatment of diseases appropriate of a general internist?
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Performance Indicator # 5:Med-Peds: Clinical Experience Data• Peds:
• How well prepared are you to perform procedures without supervision? (List from PRs)
• How well prepared are you to perform patient care activities without supervision?
• How satisfied are you with the patient volume, range of patient ages, variety of medical conditions, and extent of progressive responsibility in the care of patients?
• How satisfied are you with the educational experiences to help you achieve competency in patient care skills?
• How satisfied are you with aspects of your longitudinal outpatient experience?
• Are you well prepared to competently practice general pediatrics?
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Where did most of the NAS data elements come from? • In 2009, data modeling project began to identify factors
that predicted high and low program performance• Model was replicated• Data elements were assessed to determine “relative risk”
to predict low performance• Selection of Elements needed to be
• Obtainable• Meaningful• Correlates w/ prior decisions• Passed statistical “muster”• Used in combination
• Understand that this is a work-in-progress• New data elements likely in future
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Role of Review Committees in NAS
• Utilize data and judgment to: • concentrate efforts on problem programs• determine whether accreditation standards are violated and
provide useful feedback for programmatic improvement• determine whether these violations (citations) rise to a level
requiring alteration in accreditation status• motivate programs to rapidly improve, rather than playing the
“accelerating accreditation action game”• over time, understand and refine the nuances of the process
• How? • Annually reviewing programs using set of data elements• Using SV as needed, and conducting complete review of the
program q10 years• Using a “PIF-less”, team based, department wide evaluation of
programs
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Two steps forward….
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
NAS Year 1: Ground Rules
• Basic operational principle of NAS: • RCs will take an accreditation action on every program
annually.• All med-peds programs will receive notice regarding
accreditation status btw January and July.
• At January and March 2014 meeting, RCs reviewed NAS data submitted in AY 2012-2013• ADS annual update information submitted in fall of 2012• Faculty and Resident survey data from early spring of
2013• Responses to “previous citations” was current
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
NAS Year 1: Ground Rules
• All programs on warning or probation seen by reviewers
• All programs identified by NAS data as “troubled” underwent review by RC staff and then members• What data elements were triggered?
• Not all data elements have same importance/weight • Board scores and resident survey have more weight
– Cognizant of changes to PR for Board score pass rates for Peds
• Are programs still getting used to data elements (e.g., scholarly activity table)?
• Are there patterns/trends in data?
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
What did we expect would happen in NAS?
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
NOT this…
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
In OAS…82% of med-peds programs had a review cycle between 3-5 years *
* ACGME Data Resource Book 2012-2013, based on 78 med-peds programs. Book available on www.acgme.org.
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
NAS Conceptual Model Expected Outcomes
STANDARDS
StructureResources
Core ProcessDetailed Process
Outcomes
InitialAccreditation
New Programs
StructureResourcesCore ProcessDetailed ProcessOutcomes
Withhold AccreditationWithdrawal of Accreditation
2-4% 15% 75%
6-8%
Accreditation with Warning
New Programs,Accredited Programs with Major Concerns
Probationary Accreditation
StructureResourcesCore ProcessDetailed ProcessOutcomes
ContinuedAccreditation
Accredited Programs without Major Concerns
Continued Accreditation with
Commendation
StructureCore ProcessResourcesDetailed ProcessOutcomes
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Withdrawal of Accreditation
Good Standing 75%
Withheld/Withdrawn 6-8%
Warning/Probation
15%
New (Initial) 2-4%
NAS Conceptual Model Expected Outcomes
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
NAS Year 1: Expected vs Actual Outcomes
NAS Projections
75%
15%
2-4%6-8%
Continued Accreditation(Good Standing)
92%)
Warning/Probation 1%
New Programs (Initial) 1%
* Site visit scheduled 6%
80 Med-Peds Residency Programs
How will NAS change the flow of information and the work of
the RC and the program?
Reporting & RC Review TimelinesAY 2012-2013
Reporting & RC Review TimelinesAY 2013-2014
What new Policies and Procedures do I need to know about ?
• Review cycles are gone! Why? Annual Accreditation.• Except for new programs, they get 2
yr cycle + a site visit • No longer “propose” adverse actions
• Probation only after site visit• Core and Subs have same status
options• Probation now an option for subs
• Citations as well as non-citations• Core + Subs re-coupled…again
• Self-study = RC will see entire dept• Probation for core = probation for
subs
Highlights from New P&P Manual
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Accredited ProgramContinued Accreditation (CA)
CA w/Warning
Other (e.g. egregious)
Site Visit
Probationary Accreditation*
Withdrawal of Accreditation**
CA w/Warning
CA
* Probation cannot exceed 2 years** Does not require Probation first
Accreditation Status OptionsContinued Accreditation
Significant changes: “Continued Accreditation w Warning” appears as such on website; an adverse action no longer “proposed,” can be granted only after a site visit; subs now can also be put on “probationary accreditation.”
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Citations in NAS
Citations are not new• Identify areas of non-compliance• Linked to specific requirements• Responses required in ADS • Citations are given and removed by RC (not by staff)Phase I specialties:
• Citations received in NAS (after July 1, 2013): will require an RC member to review annually.
• Citations received in OAS (given prior to July 1, 2013): will go away after two cycles of continued accreditation in NAS with no new citations.
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Areas for Improvement (AFI)
AFIs are new in NAS• “General concerns”• May be given by staff (RC rules) or by RC
members • May not be specifically linked to a requirement• Do not require written response in ADS• Expectation that AFIs will be monitored locally
• PD and GMEC will work to resolve• AFIs will be tracked by RC
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Citations vs AFIs
• In OAS, the main mechanism to provide feedback was through citations
• In NAS, we have 2 methods: citations and AFIs• Citations require annual review by a member of the RC
In all likelihood…citations will be used more sparingly, in hopes that AFI’s trigger appropriate
local program improvement.
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Milestones
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Reporting Milestones: Med-Peds• ANNUAL reporting of reporting milestones• Report BOTH IM (22) and Peds (21) milestones• Semiannual evaluation process as usual/required• Plan will be revisited
AY 2013-14• Reporting period for both IM & Peds milestones =
May 1 – June 15, 2014
AY 2014-15• Reporting period for both IM & Peds milestones =
May 1 – June 15, 2015
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Reporting Milestones: Use by RCs
• De-identified, aggregate (program) data will be used as ONE data element RC can look at
• Cannot be fully used for several years• Initially, ascertain that programs are reporting• Next, check for completeness of data, etc• Review of patterns or trends will be important
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Clinical Competency Committee• The PD must appoint a CCC. (Core)
• Must be composed of at least 3 faculty (Core)
• Others eligible for appointment to the committee include faculty from other programs and non- physician members of the health care team. (Detail)
• Written descriptions of responsibilities (Core)
• Review all resident evaluations semi-annually (Core)
• Prepare/assure reporting of milestones evaluations of each resident to ACGME semi-annually (Core)
• Make recommendations to the PD for resident progress, including, promotion, remediation and dismissal (Detail)
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Clinical Competency Committee cont.
• PRs do not specify composition; each program may decide best structure
• PRs do not limit PD’s role• PRs do not define specialty, degree, role for
members of CCC• “Best practices” may be defined by community• Review FAQ
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Recent News: Single GME Accreditation System
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Other Recent News
• Faculty Development slide decks available on ACGME’s site
• Resident survey results will not be available until mid-June• ACGME leadership
aware this will be problematic for many and agreed to reassess for next yr.
Program ResourcesACGME Contacts
• Questions related to ADS: • Samantha Alvarado ([email protected]) 312.755.7118;
• Questions related to site visit: • Ingrid Philibert ([email protected]) 312.755.5003• Jane Shapiro ([email protected]) 312.755.5015• Penny Lawrence ([email protected]) 312.755.5014
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Questions? • Questions related to requirements or notification letter:
• IM Staff:• Jerry Vasilias ([email protected]) 312.755.7477• Karen Lambert ([email protected]) 312.755.5785• Billy Hart ([email protected]) 312.755.5002• Jessalynn Watanabe ([email protected]) 312.755.5784
• Peds Staff:• Caroline Fischer ([email protected]) 312.755.5046• Denise Braun-Hart ([email protected]) 312.755.7478• Kim Rucker ([email protected]) 312.755.7054• Luz Barrera ([email protected]) 312.755.5077
© 2014 Accreditation Council for Graduate Medical Education (ACGME)
Thank you.
“You can’t teach an old dogma new tricks.”
Dorothy Parker