if you need technical assistance please email us at acme...

42
The webinar will begin shortly. If you need technical assistance please email us at [email protected] You may also send us a message via the questions/chat box in the lower left-hand corner of your screen.

Upload: others

Post on 09-Jul-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

The webinar will begin shortly.If you need technical assistance please email us at [email protected]

You may also send us a message via the questions/chat box in the lower left-hand corner of

your screen.

Page 2: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Educational Outcomes Measurement:What We’ve Learned, What’s Ahead

Derek T. Dietze, MADerek T. Dietze, MADerek T. Dietze, MADerek T. Dietze, MAPresident

Presentation given at the 2006 Alliance for CME

Annual Meeting as:

Executive Director of CME

Veritas Institute for Medical Education, Inc.

Hasbrouck Heights, New Jersey

Page 3: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Session Objectives

1.1.1.1. Evaluate your organizationEvaluate your organizationEvaluate your organizationEvaluate your organization’’’’s educational s educational s educational s educational outcomes measurement (EOM) in the context outcomes measurement (EOM) in the context outcomes measurement (EOM) in the context outcomes measurement (EOM) in the context of othersof othersof othersof others’’’’ effortseffortseffortsefforts

2.2.2.2. Identify opportunities to advance EOMIdentify opportunities to advance EOMIdentify opportunities to advance EOMIdentify opportunities to advance EOM

3.3.3.3. Describe the potential impact of EOM on the Describe the potential impact of EOM on the Describe the potential impact of EOM on the Describe the potential impact of EOM on the future of CMEfuture of CMEfuture of CMEfuture of CME

You will be better able to:You will be better able to:You will be better able to:You will be better able to:

Page 4: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Who Is in the Audience Today?

1.1.1.1. Medical education/communication company Medical education/communication company Medical education/communication company Medical education/communication company

(MECC)(MECC)(MECC)(MECC)

2.2.2.2. HospitalHospitalHospitalHospital

3.3.3.3. Medical schoolMedical schoolMedical schoolMedical school

4.4.4.4. Medical specialty societyMedical specialty societyMedical specialty societyMedical specialty society

5.5.5.5. Health systemHealth systemHealth systemHealth system

6.6.6.6. Commercial interestCommercial interestCommercial interestCommercial interest

7.7.7.7. OtherOtherOtherOther

1. What is your affiliation?

Page 5: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Who Is in the Audience Today?

2. I have been involved in CME for _____2. I have been involved in CME for _____2. I have been involved in CME for _____2. I have been involved in CME for _____

1 = less than 1 year1 = less than 1 year1 = less than 1 year1 = less than 1 year

2 = 1 to 3 years2 = 1 to 3 years2 = 1 to 3 years2 = 1 to 3 years

3 = 3 to 5 years3 = 3 to 5 years3 = 3 to 5 years3 = 3 to 5 years

4 = 5 to 10 years4 = 5 to 10 years4 = 5 to 10 years4 = 5 to 10 years

5 = more than 10 years5 = more than 10 years5 = more than 10 years5 = more than 10 years

Page 6: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

3. To what extent do you agree with the following statement? Compared with other organizations, my

organization is doing a great job of

educational outcomes measurement (EOM).

Page 7: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

4. 4. 4. 4. To what extent do you agree with the To what extent do you agree with the To what extent do you agree with the To what extent do you agree with the

following statement?following statement?following statement?following statement?

I know what to do to improve my I know what to do to improve my I know what to do to improve my I know what to do to improve my

organizationorganizationorganizationorganization’’’’s EOM.s EOM.s EOM.s EOM.

1 = Strongly Disagree1 = Strongly Disagree1 = Strongly Disagree1 = Strongly Disagree

2222

3333

4444

5 5 5 5

6 = Strongly Agree6 = Strongly Agree6 = Strongly Agree6 = Strongly Agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

Page 8: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

5. To what extent do you agree with the following statement? Implementing EOM is essential to the

long-term success of my organization.

Page 9: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

6. To what extent do you agree with the following statement? I need to learn a lot more about EOM.

Page 10: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

CME should be a

strategic asset

for improving

performance

and enhancing

patient care.

CME in the

public interest.

Page 11: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

From the CEO of the ACCME (Dec. 2005)

““““On the research front, primary studies and On the research front, primary studies and On the research front, primary studies and On the research front, primary studies and

syntheses no longer need to ask if CE, in general, syntheses no longer need to ask if CE, in general, syntheses no longer need to ask if CE, in general, syntheses no longer need to ask if CE, in general,

improves practice or other outcomes because there improves practice or other outcomes because there improves practice or other outcomes because there improves practice or other outcomes because there

is so much evidence that many kinds and is so much evidence that many kinds and is so much evidence that many kinds and is so much evidence that many kinds and

combinations can do so.combinations can do so.combinations can do so.combinations can do so.””””

From Robertson, Umble and Cervero, Journal for Continuing

Education in the Health Professions 23, 146, 2003

Page 12: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

EOM: The Past

• EarlyEarlyEarlyEarly----mid 1990s: discussion focused on mid 1990s: discussion focused on mid 1990s: discussion focused on mid 1990s: discussion focused on

the nature and definitions of the nature and definitions of the nature and definitions of the nature and definitions of ““““educational educational educational educational

outcomesoutcomesoutcomesoutcomes””””

• Alliance for CME conference on Alliance for CME conference on Alliance for CME conference on Alliance for CME conference on

educational outcomeseducational outcomeseducational outcomeseducational outcomes

• EarlyEarlyEarlyEarly----adopters and innovatorsadopters and innovatorsadopters and innovatorsadopters and innovators

• Since 2000: Since 2000: Since 2000: Since 2000: ““““we need to do EOM!we need to do EOM!we need to do EOM!we need to do EOM!””””

Page 13: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

The Value of CME

““““A CME evaluation should focus on A CME evaluation should focus on A CME evaluation should focus on A CME evaluation should focus on identifying,

measuring, and describing the and describing the and describing the and describing the value provided provided provided provided

by an intervention that leads to enhancedby an intervention that leads to enhancedby an intervention that leads to enhancedby an intervention that leads to enhanced

performance, improved improved improved improved healthcare quality, and and and and

reduced costs.reduced costs.reduced costs.reduced costs.””””

Moore DE. A framework for outcomes evaluation in the continuing professional development of physicians. In: Davis D, Barnes BE, Fox R, eds. The Continuing Professional Development of Physicians: From Research to Practice. Chicago, Ill: American Medical Association; 2003.

Page 14: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Levels of Educational Outcomes

Changes in Changes in Changes in Changes in selfselfselfself----reportedreportedreportedreported knowledge of the participants; knowledge of the participants; knowledge of the participants; knowledge of the participants;

development development development development of competenceof competenceof competenceof competenceLearningLearningLearningLearning

Changes in Changes in Changes in Changes in observedobservedobservedobserved practice performance; the practice performance; the practice performance; the practice performance; the

application application application application of learning; the application of competenceof learning; the application of competenceof learning; the application of competenceof learning; the application of competencePerformancePerformancePerformancePerformance

Changes in the health status of patients due to changes Changes in the health status of patients due to changes Changes in the health status of patients due to changes Changes in the health status of patients due to changes

in in in in participantparticipantparticipantparticipant practice behaviorpractice behaviorpractice behaviorpractice behaviorPatient HealthPatient HealthPatient HealthPatient Health

Changes in the health status of a population of patients Changes in the health status of a population of patients Changes in the health status of a population of patients Changes in the health status of a population of patients

due to changes in due to changes in due to changes in due to changes in widespreadwidespreadwidespreadwidespread practice behaviorpractice behaviorpractice behaviorpractice behaviorPopulation HealthPopulation HealthPopulation HealthPopulation Health

The number of people who registered and/or participatedThe number of people who registered and/or participatedThe number of people who registered and/or participatedThe number of people who registered and/or participatedParticipationParticipationParticipationParticipation

The degree to which participantsThe degree to which participantsThe degree to which participantsThe degree to which participants’’’’ expectations about the expectations about the expectations about the expectations about the

setting and delivery of the CME activity were metsetting and delivery of the CME activity were metsetting and delivery of the CME activity were metsetting and delivery of the CME activity were metSatisfactionSatisfactionSatisfactionSatisfaction

Adapted from: Moore DE. A framework for outcomes evaluation in the continuing professional

development of physicians. In: Davis D, Barnes BE, Fox R, eds. The Continuing Professional Development

of Physicians: From Research to Practice. Chicago, Ill: The American Medical Association; 2003.

Page 15: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Measurement: Alliance for CME Competencies Survey*

• Competency 2.12, Competency 2.12, Competency 2.12, Competency 2.12, “…“…“…“…offer educational offer educational offer educational offer educational

consultation that emphasizes measurement in consultation that emphasizes measurement in consultation that emphasizes measurement in consultation that emphasizes measurement in

physician organizations to identify goals for physician organizations to identify goals for physician organizations to identify goals for physician organizations to identify goals for

education that are specific to the practice and education that are specific to the practice and education that are specific to the practice and education that are specific to the practice and

measurablemeasurablemeasurablemeasurable…………,,,,””””

• Rated the lowest in terms of Rated the lowest in terms of Rated the lowest in terms of Rated the lowest in terms of ““““responsibilityresponsibilityresponsibilityresponsibility”””” and and and and

““““attainmentattainmentattainmentattainment”””” by CME leaders, managers, and by CME leaders, managers, and by CME leaders, managers, and by CME leaders, managers, and

coordinators (2.6/2.6).coordinators (2.6/2.6).coordinators (2.6/2.6).coordinators (2.6/2.6).

*King R, North S. Alliance for CME Competency Areas Research Project Report.Alliance for CME; 2004.

Page 16: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Forces Driving EOM

• Accountability for effectiveness, impact, and use Accountability for effectiveness, impact, and use Accountability for effectiveness, impact, and use Accountability for effectiveness, impact, and use

of resourcesof resourcesof resourcesof resources

• Desire for Desire for Desire for Desire for ““““exemplaryexemplaryexemplaryexemplary”””” statusstatusstatusstatus

• Performance improvement movementPerformance improvement movementPerformance improvement movementPerformance improvement movement

• MOC and MOL requirements/initiativesMOC and MOL requirements/initiativesMOC and MOL requirements/initiativesMOC and MOL requirements/initiatives

• Pay for performance movementPay for performance movementPay for performance movementPay for performance movement

• Increasingly a requirement of commercial supportIncreasingly a requirement of commercial supportIncreasingly a requirement of commercial supportIncreasingly a requirement of commercial support

• ACCME is ACCME is ACCME is ACCME is ““““raising the barraising the barraising the barraising the bar”””” for providersfor providersfor providersfor providers

• CME communityCME communityCME communityCME community’’’’s desire for continuous s desire for continuous s desire for continuous s desire for continuous

improvementimprovementimprovementimprovement

Page 17: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Performance Improvement Movement

October 6, 2004Performance Improvement Activities approved for creditStage A: Learning from current practice performance assessmentStage B: Learning from the application of PI to patient careStage C: Learning from the evaluation of the PI effort

May 23, 2005Internet Point of Care now eligible for credit> Review original clinical question(s).> Identify the relevant sources from among those consulted.> Describe the application of their findings to practice.

American Medical Association

Page 18: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Performance Improvement Movement (cont.)

• American Academy of Family Physicians: American Academy of Family Physicians: American Academy of Family Physicians: American Academy of Family Physicians:

Performance Improvement in PracticePerformance Improvement in PracticePerformance Improvement in PracticePerformance Improvement in Practice

Page 19: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

MOC and MOL Initiatives

• July 2003: ABFM introduced new MOC requirements July 2003: ABFM introduced new MOC requirements July 2003: ABFM introduced new MOC requirements July 2003: ABFM introduced new MOC requirements

for family physicians.for family physicians.for family physicians.for family physicians.

• Part IV requirement replaced the traditional clinical Part IV requirement replaced the traditional clinical Part IV requirement replaced the traditional clinical Part IV requirement replaced the traditional clinical

chart review. Focus on improving quality of care, rather chart review. Focus on improving quality of care, rather chart review. Focus on improving quality of care, rather chart review. Focus on improving quality of care, rather

than on recordthan on recordthan on recordthan on record----keeping.keeping.keeping.keeping.

• Each FP must complete 1 QI project during his or her Each FP must complete 1 QI project during his or her Each FP must complete 1 QI project during his or her Each FP must complete 1 QI project during his or her

recertification cycle to comply with Part IV.recertification cycle to comply with Part IV.recertification cycle to comply with Part IV.recertification cycle to comply with Part IV.

American Board of Medical Specialties: Part IV Maintenance of Certification

“Evidence of evaluation of performance in practice”

Page 20: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Board of Directors Action Plan Fiscal 2004

“Assure ACCME accreditation remains relevant by addressing the needs of medical boards to quantify learning and practice outcomes of individual physician learners.”

Page 21: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Commercial Interests’ Expectations

Increasingly, commercial interests are requiring Increasingly, commercial interests are requiring Increasingly, commercial interests are requiring Increasingly, commercial interests are requiring

CME providers to:CME providers to:CME providers to:CME providers to:

• Include a sound outcomes measurement plan Include a sound outcomes measurement plan Include a sound outcomes measurement plan Include a sound outcomes measurement plan

in grant requests in grant requests in grant requests in grant requests

• Become more familiar with EOM methodologies Become more familiar with EOM methodologies Become more familiar with EOM methodologies Become more familiar with EOM methodologies

and tools and tools and tools and tools

• Report results for evaluation and future Report results for evaluation and future Report results for evaluation and future Report results for evaluation and future

planningplanningplanningplanning

Page 22: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Current Status of EOM

• Increasing understanding of EOM among some Increasing understanding of EOM among some Increasing understanding of EOM among some Increasing understanding of EOM among some CME providers and MECCsCME providers and MECCsCME providers and MECCsCME providers and MECCs

• Some measurement methodologies gaining Some measurement methodologies gaining Some measurement methodologies gaining Some measurement methodologies gaining acceptance and popularityacceptance and popularityacceptance and popularityacceptance and popularity

• Few standards or best practicesFew standards or best practicesFew standards or best practicesFew standards or best practices

• Some measurement beyond Some measurement beyond Some measurement beyond Some measurement beyond ““““satisfactionsatisfactionsatisfactionsatisfaction”””” into into into into ““““learninglearninglearninglearning”””” and and and and ““““performanceperformanceperformanceperformance””””

• Lack of scientific rigorLack of scientific rigorLack of scientific rigorLack of scientific rigor

• Much talkMuch talkMuch talkMuch talk

• Limited experimentationLimited experimentationLimited experimentationLimited experimentation

• FearFearFearFear

Page 23: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

EOM Challenges

• EOM competence, skills, and resourcesEOM competence, skills, and resourcesEOM competence, skills, and resourcesEOM competence, skills, and resources

• TimeTimeTimeTime

• High level support & resourcesHigh level support & resourcesHigh level support & resourcesHigh level support & resources

• Practical EOM methodologies/toolsPractical EOM methodologies/toolsPractical EOM methodologies/toolsPractical EOM methodologies/tools

• Funding/support for EOM from commercial Funding/support for EOM from commercial Funding/support for EOM from commercial Funding/support for EOM from commercial

interestsinterestsinterestsinterests

• Actual/perceived lack of access to Actual/perceived lack of access to Actual/perceived lack of access to Actual/perceived lack of access to ““““the datathe datathe datathe data””””

• ““““Rigor mortisRigor mortisRigor mortisRigor mortis”””” and and and and ““““perfection paralysisperfection paralysisperfection paralysisperfection paralysis””””

• Fear of the resultsFear of the resultsFear of the resultsFear of the results

Page 24: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Current EOM Methodologies & Tools

• FollowFollowFollowFollow----up surveysup surveysup surveysup surveys

• Immediate preImmediate preImmediate preImmediate pre---- and postand postand postand post----activity surveys/testsactivity surveys/testsactivity surveys/testsactivity surveys/tests

• Control group surveysControl group surveysControl group surveysControl group surveys

• Focus groupsFocus groupsFocus groupsFocus groups

• Clinical assertionsClinical assertionsClinical assertionsClinical assertions

• Clinical/case vignettesClinical/case vignettesClinical/case vignettesClinical/case vignettes

• Commitment to changeCommitment to changeCommitment to changeCommitment to change

• PrePrePrePre---- and postand postand postand post----activity prescription measurementactivity prescription measurementactivity prescription measurementactivity prescription measurement

• Chart auditsChart auditsChart auditsChart audits

• QA dataQA dataQA dataQA data

Page 25: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Pre/Post Clinical Assertion with Control

“’“’“’“’Early aggressive therapyEarly aggressive therapyEarly aggressive therapyEarly aggressive therapy’’’’ is treatment initiated within is treatment initiated within is treatment initiated within is treatment initiated within

the first 6 months of RA onset.the first 6 months of RA onset.the first 6 months of RA onset.the first 6 months of RA onset.””””

4.914.914.914.91

5.435.435.435.43

4.914.914.914.91

1111 2222 3333 4444 5555 6666

Pre-ActivityPre-ActivityPre-ActivityPre-Activity

Post-ActivityPost-ActivityPost-ActivityPost-Activity

ControlControlControlControl

Strongly Strongly Strongly Strongly

DisagreeDisagreeDisagreeDisagreeStrongly Strongly Strongly Strongly

AgreeAgreeAgreeAgree

n=117n=117n=117n=117

n=129n=129n=129n=129

n=33n=33n=33n=33

Significant at Significant at Significant at Significant at P P P P < 0.01< 0.01< 0.01< 0.01

Data courtesy of Veritas Institute for Medical Education, Inc.

Page 26: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Pre/Post Case Vignette: Patient With AS

• How would you treat a 28How would you treat a 28How would you treat a 28How would you treat a 28----yearyearyearyear----old male old male old male old male

patient exhibiting the following symptoms:patient exhibiting the following symptoms:patient exhibiting the following symptoms:patient exhibiting the following symptoms:

– Morning stiffness

– X-ray shows bilateral (sacroiliitis)

– Swollen, painful knee

– ESR 10

– CBC normal

Page 27: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Case: AS PatientPercent of Pre-/Post-Discussion Responses

““““How would you treat?How would you treat?How would you treat?How would you treat?””””

35.835.835.835.8

22.522.522.522.5

58.858.858.858.8

4.84.84.84.8

5.25.25.25.2

8.78.78.78.7

10.610.610.610.6

12.412.412.412.4

21.121.121.121.1

20.220.220.220.2

Pre-discussion (n=111)Pre-discussion (n=111)Pre-discussion (n=111)Pre-discussion (n=111)

Post-discussion (n=106)Post-discussion (n=106)Post-discussion (n=106)Post-discussion (n=106)

Combination

Anti-TNF

MTX

SSZ

NSAIDs

Percent of Responses

Data courtesy of Veritas Institute for Medical Education, Inc.

Page 28: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Case: AS PatientPercent Change in Pre-/Post-Discussion Response

““““How would you treat?How would you treat?How would you treat?How would you treat?””””

Combination

Anti-TNF

MTX

SSZ

NSAIDs

-15.4-15.4-15.4-15.4

-5.3-5.3-5.3-5.3

-27.1-27.1-27.1-27.1

+10.2+10.2+10.2+10.2

+37.6+37.6+37.6+37.6

-40-40-40-40 -30-30-30-30 -20-20-20-20 -10-10-10-10 0000 10101010 20202020 30303030 40404040 50505050Percent Change in Responses

Data courtesy of Veritas Institute for Medical Education, Inc.

Page 29: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Case Vignette: Post-activity with Control

Outcomes, Inc. CME Metrics Report, July 25, 2005. Challenging Cases in Gastroenterology—Then & Now, Jobson Education Group. Satellite symposium at Digestive Disease Week, May 15, 2005.

N=46 for both groups P=0.05

Page 30: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Only patients with severe and/or refractory cases of Only patients with severe and/or refractory cases of Only patients with severe and/or refractory cases of Only patients with severe and/or refractory cases of urticaria should be managed with steroids.urticaria should be managed with steroids.urticaria should be managed with steroids.urticaria should be managed with steroids.

Pre/Post/Follow-up Clinical Assertions with Control

4.664.664.664.66 4.814.814.814.81

5.645.645.645.64 5.725.725.725.725.335.335.335.33

5.755.755.755.75

4.744.744.744.74

1111

2222

3333

4444

5555

6666

BaselineBaselineBaselineBaseline Live PreLive PreLive PreLive Pre Live PostLive PostLive PostLive Post Live + EMLive + EMLive + EMLive + EM

PostPostPostPost

Live OnlyLive OnlyLive OnlyLive Only

FUFUFUFU

Live + EMLive + EMLive + EMLive + EM

FUFUFUFU

BaselineBaselineBaselineBaseline

FUFUFUFU

Strongly agree

Strongly disagree

****PPPP < 0.001< 0.001< 0.001< 0.001

n=390 n=1,753 n=1,374n=335 n=229 n=67 n=372FU = Follow-up survey

EM = Enduring material

Mean RatingLive Live Live Live onlyonlyonlyonly participants do participants do participants do participants do

not maintain learningnot maintain learningnot maintain learningnot maintain learning

Live + EM participants maintain Live + EM participants maintain Live + EM participants maintain Live + EM participants maintain

better learning through follow upbetter learning through follow upbetter learning through follow upbetter learning through follow up

Dietze DT, Magazine HI. Enhancing and Sustaining Learning: Report of a Comprehensive CME Initiative and Educational Outcomes Measurement Plan (forum). Alliance for CME 31st Annual Conference; January 24, 2006; San Francisco, California.

Page 31: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Performance Improvement Project

Purpose:Purpose:Purpose:Purpose:

Decrease the risk of recurrent vascular events in

patients with AMI and CAD by initiating therapy with

lipid-lowering agents during hospitalization.

Measurement Point:Measurement Point:Measurement Point:Measurement Point:

Percentage of defined AMI and CAD patients who

received a lipid lowering agent charge during stay.

Multidisciplinary Team on PI Project:Multidisciplinary Team on PI Project:Multidisciplinary Team on PI Project:Multidisciplinary Team on PI Project:

Cardiology Physicians, Telemetry Nursing Staff, Quality

Services, CME Department, Pharmacy Decision

Support (data specialist)

Information courtesy of Susan G. McAlexander, MPA, CHES, CME Coordinator, Providence St. Peter Hospital, Olympia, WA, February 2006.

Page 32: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Measurement, Data Collection & Analysis

• Lipid use rates compared monthly and quarterly

for changes after interventions.• Education – CME and other

• Forms adjustments

• In-services

• Monthly project task force meetings to project planning,

discuss interventions and results.

• Aggregate and specific rate collection performed

in the Quality Department utilizing charge data

and chart review.

• Assessment of knowledge post CME single activity

intervention performed by CME department.

• Monthly tends reported on graphs.

Page 33: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Results and Application to Future CME

• Goal percentage attained in 3rd quarter for AMI patients and 4th quarter for ICS patients.

• Greatest change noted in the cardiology and internal medicine sections. Family medicine N too low to have statistical significance.

• Final report to CIC included a recommendation to tie

CME into future PI projects for maximal impact.

• CME was seen to have a positive and reinforcing effect

on physician learning and practice change.

Page 34: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Measurement Is in Your Future!

Page 35: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

From the CEO of the ACCME (Dec. 2005)

““““The Board wants to make it clear that the The Board wants to make it clear that the The Board wants to make it clear that the The Board wants to make it clear that the

ACCME values an accreditation system that ACCME values an accreditation system that ACCME values an accreditation system that ACCME values an accreditation system that

rewards providersrewards providersrewards providersrewards providers

a) for establishing a mission that identifies

improvements in physician performance and/or

patient or population health as organizational

goals, and

b) for having a process in place to measure and

increase its effectiveness in meeting this mission.”

Page 36: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Updated Accreditation Criteria (9/5/06)

Selected Level 1 Criteria:Selected Level 1 Criteria:Selected Level 1 Criteria:Selected Level 1 Criteria:

““““The provider has a CME mission that includes all of the The provider has a CME mission that includes all of the The provider has a CME mission that includes all of the The provider has a CME mission that includes all of the

basic componentsbasic componentsbasic componentsbasic components…………with expected results articulated in with expected results articulated in with expected results articulated in with expected results articulated in

terms of changes in competence, performance, or patient terms of changes in competence, performance, or patient terms of changes in competence, performance, or patient terms of changes in competence, performance, or patient

outcomes that will be the result of the program. (2.1)outcomes that will be the result of the program. (2.1)outcomes that will be the result of the program. (2.1)outcomes that will be the result of the program. (2.1)””””

““““The provider analyzes changes in learners (competence, The provider analyzes changes in learners (competence, The provider analyzes changes in learners (competence, The provider analyzes changes in learners (competence,

performance, or patient outcomes) achieved as a result of performance, or patient outcomes) achieved as a result of performance, or patient outcomes) achieved as a result of performance, or patient outcomes) achieved as a result of

the overall programthe overall programthe overall programthe overall program’’’’s activities/educational interventions. s activities/educational interventions. s activities/educational interventions. s activities/educational interventions.

(2.4)(2.4)(2.4)(2.4)””””

Page 37: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = I7 = I7 = I7 = I’’’’m not familiar with the new criteriam not familiar with the new criteriam not familiar with the new criteriam not familiar with the new criteria

7. To what extent do you agree with the following statement?The ACCME’s “Updated Accreditation

Criteria” are a very positive step forward

for the CME community.

Page 38: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

8. To what extent do you agree with the following statement? Compared to other organizations, my

organization is doing a great job of EOM.

Page 39: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Audience Opinion Poll

1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree1 = Strongly disagree

2 2 2 2

3333

4444

5555

6 = Strongly agree6 = Strongly agree6 = Strongly agree6 = Strongly agree

7 = Not applicable7 = Not applicable7 = Not applicable7 = Not applicable

9. To what extent do you agree with the following statement? Implementing EOM is essential to the

long-term success of my organization.

Page 40: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Vision for EOM

• EOM standards and benchmarksEOM standards and benchmarksEOM standards and benchmarksEOM standards and benchmarks

• EOM best practicesEOM best practicesEOM best practicesEOM best practices

• Proven methodologies and toolsProven methodologies and toolsProven methodologies and toolsProven methodologies and tools

• More efficient and effective CMEMore efficient and effective CMEMore efficient and effective CMEMore efficient and effective CME

• Improved patient careImproved patient careImproved patient careImproved patient care

• Improved patient outcomesImproved patient outcomesImproved patient outcomesImproved patient outcomes

• Improved population healthImproved population healthImproved population healthImproved population health

Page 41: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Question & Answer Period

To ask a question simply type your question

into the “Questions” box on the lower left-

hand corner of your screen. Click “send” to

submit your question.

Page 42: If you need technical assistance please email us at acme ...media01.commpartners.com/acme_eo2_docs/060913_Slides_Color.pdf33..3. Describe the potential impact of EOM on the 3. Describe

Thank You for Your Participation!

Derek T. Dietze, MADerek T. Dietze, MADerek T. Dietze, MADerek T. Dietze, MA

President

[email protected]

Queen Creek, Arizona