chapter 2 assessing professionalism
TRANSCRIPT
12/9/2013
1
Chapter 2
Assessing Professionalism It matters. So, once we’ve defined professionalism, how do we teach it, observe it, and measure it?
Chapter 1: Professionalism. How can we evolve an optimal environment for teaching and learning about professionalism?
12/9/2013
2
Disclosure statement
Dr. Williams has nothing to disclose.
“Bad Doctors or Bad Drugs” Patients Speak
http://youtu.be/hZzjH04ye48
12/9/2013
3
“I knew in my gut
when that kid was a
student that there would
be problems later…”
“…but nothing big
ever happened in
front of me…”
Former professor of “Doctor X”
who was recently charged with
professional misconduct
Imagine
your
picture
here
12/9/2013
4
Professionalism. How can we evolve an optimal environment
for teaching and learning about professionalism?
Objectives
1. Define “professionalism” as applied to medicine and the health
professions (see example:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1769526/ )
2. Describe 3 fundamental principles of medical professionalism,
and
3. List and describe 10 professional responsibilities in the physician
charter for medical professionalism (see
http://www.abimfoundation.org/Professionalism/Physician-
Charter.aspx )
4. Reflect on professionalism challenges, feed-forward and
feedback approaches suited to the optimal learning environment
(see example: Sullivan and Benner in Am Jnl Critical Care
http://ajcc.aacnjournals.org/content/14/1/78.full)
5. Describe 3 key factors for creating an optimal environment for
teaching and learning about professionalism
12/9/2013
5
Assessing Professionalism: Objectives
1. Quick overview of the landscape for professionalism
in medicine and health care
2. Discuss 3 factors to create an optimal environment
for teaching, learning and professionalism
assessment
3. Practice assessment: building a toolkit using the
OUHSC policy and PCR*: Student Professional
Behavior in an Academic Program
*Professionalism Concerns Report
Source: http://www.adea.org/Pages/Professionalism.aspx
The landscape for
professionalism
assessment…Dentistry
Obj. 1: Quick overview of the landscape …
12/9/2013
6
The landscape for professionalism assessment…in Science
Professionalism in science denotes a
pattern of behavior identified with
scientific integrity, that in turn provides
certain privileges.
The landscape for professionalism assessment… Allied Health (which exclusive of physicians and nurses includes over 85 professions)
See Minnesota Please Care: http://www.minnesotamedicine.com/Default.aspx?tabid=2242
12/9/2013
7
The landscape for professionalism assessment…Nursing
Source:
http://www.aacp.org/resources/studentaffairspersonnel/stud
entaffairspolicies/Pages/professionalism.aspx
Source Toolkit:
http://www.aacp.org/resources/studentaffairspersonnel/stud
entaffairspolicies/Documents/Version_2%200_Pharmacy_P
rofessionalism_Toolkit_for_Students_and_Faculty.pdf
The landscape for professionalism assessment…Pharmacy
12/9/2013
8
The landscape for professionalism assessment… Public Health
See Table: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2431097/table/F1/
ACGME Core Competencies Definitions
Accreditation Council for Graduate Medical Education (ACGME) core competencies.
Patient Care: Identify, respect, and care about patients' differences, values, preferences, and expressed needs; listen to, clearly inform, communicate with and educate patients; share decision making and management; and continuously advocate disease prevention, wellness, and promotion of healthy lifestyles, including a focus on population health.
Medical Knowledge: Established and evolving biomedical, clinical, and cognate (e.g. epidemiological and social behavioral) sciences and the application of knowledge to patient care.
Practice-Based Learning and Improvement: Involves investigation and evaluation of one's own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care.
Interpersonal and Communication Skills: That result in effective information exchange and teaming with
patients, their families and other health professionals.
Professionalism: Commitment to carrying out professional responsibilities, adherence to ethical principles and sensitivity to a diverse patient population. Systems-Based Practice: Actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.
12/9/2013
9
A Physician Charter, ABIM
Source: http://www.abimfoundation.org/Professionalism/Physician-Charter.aspx
The landscape for professionalism assessment…Medicine
…Essential to this contract with society is public
trust in clinicians and scientists which depends
on the integrity of both the individual and the
whole profession to which the individual belongs.
P R O F E S S I O N A L I S M
IN THE AHC: A PROFESSIONAL CHARTER
For a common framework we could easily adapt the ABIM charter to say,
12/9/2013
10
Observable Behaviors
See handout- List your 10 personal favorites.
Briefly discuss with neighbor – What commonalities?
Differences?
12/9/2013
11
3 key factors for creating an
optimal environment for
teaching and learning about
professionalism
1. Communicate about expectations
2. Check the rearview mirror
-- sharing experiences for shared learning?
3. Practice what you/we teach
Obj. 2: Three factors for creating the optimal…
12/9/2013
12
Interprofessional competencies for
professionalism
IPEC General competency statement
“Work with individuals of other professions
to maintain a climate of mutual respect and
shared values.”
“Although there are several measures of attitudes
towards aspects of professionalism in medicine, there
is little evidence to indicate measures that are effective
in assessing attitudes towards professionalism in
medicine as a whole. Few studies have reported measures
that may be used longitudinally throughout the
curriculum. There is little evidence of interventions
that influence attitude change over a period of time…”
Jha, et al. Medical Education 2007: 41: 822–829
12/9/2013
13
So how should
professionalism be
assessed?
Obj. 3: Building a toolkit …
“….Our data indicate that future research should
operationalise generic definitions of professionalism
instead of using attributes and ⁄ or proxy measures. This
may involve establishing a widely agreed construct of
professionalism, for example, through professional
consensus.”
Jha et al., Medical Education 2007: 41: 822–829
Emphasis added
12/9/2013
14
OUHSC Student professional behavior in
an academic program policy
The University of Oklahoma Health Sciences
Center (OUHSC) strives to attract, matriculate,
and train health professions and public health,
biomedical, and pharmaceutical sciences
graduate students …who not only possess the
intellectual capacity for health professions and
graduate study but also have a high capacity for
ethical and professional behavior.
See: OUHSC Faculty Handbook 2012: https://www.ouhsc.edu/provost/
> Faculty Handbook Section 4.2.0 and APPENDIX C
Learners can demonstrate the adoption of behaviors and values inherent in the HSC disciplines. This is a shared interest across OUHSC colleges. Clarity about expected core behaviors and values is necessary to assessment.
The Professional Behavior in an Academic Program policy provides a framework for any observer of an HSC student to use this shared standard of expectations and concrete approach for follow-up and correction as deemed necessary.
1: Clear Expectations
In policy and on
Professionalism Concerns Report (PCR)
2: Observation & Feedback
Faculty and others can more easily compare an observed behavior to the
expected behavior
3: Notation and Follow-up
Where expectations do not match behavior
the student can be clearly counseled and
follow-up action specified
12/9/2013
15
Some agreed upon
Characteristics of professionalism
At OUHSC professional behaviors expected should be observable in six (6)
categories. Specifics follow each of these in the PCR* rubric.
Integrity & Honesty
Patient Centered Care & Patient Safety
Respect
Service & Working within the Team
Responsibility
Responsiveness, Adaptability & Self-Improvement
PCR = Professionalism Concerns Report
Kirkpatrick Four Levels of Learning Outcomes
Four Levels Learning
Outcomes
Conditions Assessment Methods
Reaction Level 1
Learner
satisfaction
Satisfied with
learning experience
Event and self-assessment. Personal
objectives pre- and post-assessment
Learning Level 2
Learner attitudes Desire to change
Pre-test; post-test
retrospective- post-test; exam at end of
learning event
High fidelity simulation Knowledge
acquisition
Knows what to do;
Knows how to do it
Behavior Level 3
Behavioral
Change
Work climate is right
for new behavior to
be demonstrated
Supervisor does NOT prevent;
discourage; ignore (neutralize)
supervisor DOES encourage or
require learning transfer
Results Level 4
Changes in
[clinical] practice
Work environment
rewards the
behavior change
Measure improvements (e.g., increased
quality; decreased cost; reduced
turnover or errors; improved
morale/engagement metrics)
Benefits to
patients
Measure improved patient outcomes;
improved patient satisfaction;
improved metrics on reported measures
of ptnt care
12/9/2013
16
Norcini et al. 2011. Med Teach
Criteria Practice Points
Achieving good assessment
1. validity or coherence
2. reproducibility or
consistency
3. equivalence
4. feasibility
5. educational effect
6. catalytic effect, and
7. acceptability
Consider:
1. perspectives of patients and the public
2. the intimate relationship between assessment, feedback, and continued learning
3. systems of assessment, and
4. accreditation systems
Source: Criteria for good assessment: Medical Teacher, 2011; 33: 206–214
12/9/2013
17
Getting started
Eric S. Holmboe, MD, PhD says:
Assess your current tools.
What competency does the tool assess?
Is it formative or summative or both?
Is it appropriate for the purpose and competency?
Are you satisfied with the tool?
If YES – keep using
If NO
Improve the tool, OR
Identify a new tool
Example see:
Society of Teachers of
Family Medicine
“Tools to Measure
Professionalism” http://www.stfm.org/RCtoolkit/
AssessmentMethods.cfm
Assessment methods and tools
Self-Assessment
Written Exam
Simulation
Learner/Faculty
Discussion
Portfolio
Direct Clinical Observation
Medical Record/Chart Audit
Multisource Feedback Norcini
12/9/2013
18
Observation: preparation see handout
Observation: formative
12/9/2013
19
Ob
se
rva
tio
n: fo
rma
tive
JHU example see handout
12/9/2013
20
Use high fidelity simulation
Gisondi et al. 2004. Acad Emergc Med
Use high fidelity simulation
12/9/2013
21
Use high fidelity simulation Ethical dilemma 1: Patient Confidentiality
Ethical dilemma 2: Informed consent
Ethical dilemma 3: Withdrawal of Care
Ethical dilemma 4: Practicing Procedures on the Dead
Case Example:
Patient Confidentiality
A critically ill patient asks the resident to keep the cause of his
illness a secret. After stabilization of the patient, a very
concerned, simulated employer enters the treatment area and
asks the resident, ‘‘What happened to my friend?’’ The resident
has several options: withhold all information, withhold only the
information that the patient asked to keep private, or return to
the patient and ask whether the case could be discussed.
We defined professional competency in ‘‘Patient
Confidentiality’’ as the resident’s not discussing clinical or
private information about a patient with others.
12/9/2013
22
Key players in professionalism assessment
Students (every level)
Faculty (classroom/laboratory)
Faculty Attending (clinical)
Residents
Independent Observers*
Program Directors
Associate/Assistant Dean Student Affairs
Student progress committee
Curriculum Committee
Dean
Vice Provost Academic Affairs
OUHSC Professionalism Concerns Report
12/9/2013
23
PCR details
PCR follow-up action
• Includes
observed
behavior
• Specifies
correction
needed
• Affords
student
comment
option
• Signed by
student
and faculty
(front)
12/9/2013
24
Other professionalism assessment instruments
ABIM Scale – Professional Attitude & Behaviors
Barry Challenges to Professionalism
Questionnaire
JHU Professionalism Questionnaire
UC Davis Professionalism Instrument
Musick 360-degree assessment
Wake Forest Physician Trust Scale
What should influence your
selection of an assessment tool?
12/9/2013
25
Principle 1 competency based objectives for
the learner
Principle 2 continuous improvement of the
educational experience
Principle 3 continuous improvement of learner
performance
Principle 4 continuous improvement of
educational program performance
Based on: McMillan JH. Essential assessment concepts for teachers and administrators.
Thousand Oaks, CA: Corwin Press, Inc. 2001.
How will you approach …
Under “Professionalism” what are your…
“Surfacing undiscussables” What’s an “undiscussable”?
An issue in an organization that is not engaged in order to “avoid surprise, embarrassment or threat”
Argyris, C. 1991. “Skilled incompetence” Managing with People in Mind. Harvard Business Review Press no. 90085.
12/9/2013
26
pro.fes.sion.al.ism
1. professional character, spirit or methods. 2. the
standing practice, or methods of a professional, as
distinguished from an amateur. [1855-60]
Random House Unabridged Dictionary (Second Edition)
What distinctions do you expect to
consistently observe that should also be
readily observable by others (e.g., peers,
colleagues, patients and society)?
When we build a working consensus we have a route to effective
assessment and measures, selecting appropriate assessment tools,
and reinforcing what distinguishes the professions from non-
professions
References & Resources
Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an Expert panel. Washington, D.C.: Interprofessional Education Collaborative.
Medical Professionalism in the New Millennium: A Physician Charter. Ann Intern Med 2002;136: 243-246
Feng, FC, Steen RG, Casadevall, A. Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences. http://www.pnas.org/content/early/2012/09/27/1212247109.abstract?sid=72cdd6f1-283c-4f0b-aab5-20a83247aba0
Accreditation Council for Graduate Medical Education. General competencies Chicago: ACGME 1999. Available at http://www.acgme.org/outcome/comp/compFull.asp#5; accessed August 16, 2006.
American Board of Internal Medicine Foundation. American College of Physicians–American Society of Internal Medicine Foundation. European Federation of Internal Medicine Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136(3):243–246.
Papadakis MA, Osborn EH, Cooke M, Healy K. 1999. A strategy for the detection and evaluation of unprofessional behaviour in medical students. Acad Med 74:980–990.
Papadakis MA, Teherani A, Banach MA, Knettler TR, Rattner SL, Stern DT, Veloski JJ, Hodgson CS. Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. 2005;353(25):2673–2682.
Hickson GB, Pichert JW, Webb LE, Gabbe SG. 2007b. A complementary approach to promoting professionalism: Identifying, measuring and addressing unprofessional behaviors. Acad Med 82:1040–1048.
Benner P. From Novice to Expert: Excellence and Power in Clinical Nursing Practice. Menlo Park, CA: Addison Wesley; 1984. (Benner’s theory from novice to expert)
OUHSC Student Behavior in a Professional Program Policy. OUHSC Faculty Handbook (2012)
12/9/2013
27
Education Grand Rounds
The University of Oklahoma Health Sciences Center
June 21, 2013
Chapter 2
Assessing Professionalism It matters. So, once we’ve defined professionalism, how do we teach it, observe it, and measure it?
Valerie N. Williams, Ph.D., M.P.A. E-mail: [email protected]
Professionalism How can we evolve an optimal
environment for teaching and learning
about professionalism?
Valerie N. Williams, Ph.D., M.P.A.
E-mail: [email protected]
Solomon Papper Lecture Internal Medicine Grand Rounds College of Medicine . The University of Oklahoma Health Sciences Center
January 9, 2013
Chapter 1