evidence-based practice aupha annual meeting, june 20, 2013, monterey if doctors can do it managers...
TRANSCRIPT
Evidence-Based Practice
AUPHA Annual Meeting, June 20, 2013, Monterey
If doctors can do it managers can do it?
Evidence?
outcome of scientific research,
organizational facts & data,
benchmarking, best practices,
collective experience, personal
experience, intuition
Many managers pay little
or no attention to the
quality of the evidence they
base their decisions on
Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence
What is the added value of evidence- based
practice for managers within the field of
hospital care?
Proof of concept
Phase 1: Training managers in the principles of EBP
Phase 2: Examination of the current decision making
processes that managers are using
Phase 3: Evaluation of 4 completed projects from an
EB perspective (retrospective)
Phase 4: Making EB recommendations for 4 new
projects (prospective)
Phase 5: Evaluation
Evidence based pilot
Decision making process
Focus on procedures instead of evidence
Internal politics and power struggles
No critical appraisal of the evidence at hand
Relying on anecdotal evidence (workshops,
best practices, popular management books,
consultants)
One option (sometimes two)
Bias: Outcome, Halo, Confirmation, etc.
Evidence-based perspective
NOT: Did we made the right decision?
BUT: Is there evidence from scientific research to
support (or call into question) the approach
taken?
Best available evidence?
leadership training: dm process
No problem definition
No organizational evidence consulted
Selection of training companies based on
experience, recommendation or reputation
No explicit selection criteria / procedure
‘Best’ presentation has won: one size fits all
leadership training: scientific evidence
15 meta analyses, 5 relevant
37 (‘systematic’) reviews, 2 relevant
Lots of relevant primary studies
leadership training: scientific evidence
Long history (30 yrs): moderate effect sizes
Senior & middle managers tend to benefit more
than managers at the supervisory level
Effect on ‘poor’ leaders is limited.
Leadership trainings that focus on
interpersonal / social skills show higher effect
sizes than those based on a specific leadership
‘model’
Questions / projects
360 degree feedback
Financial incentives
Lean Six Sigma
Hand Hygiene
Goal setting
Value Based Health Care
Downsizing
Evidence-based perspective
NOT: What works?
BUT: What are, given the target group, the problem and
the context involved, the main factors determining the
success or failure of the project that need to be taken into
account?Best available scientific
evidence?
Multi Source Feedback: background
IFMS: based on multi source feedback
Regulating bodies and insurance companies
(KPI’s – prices/ revenue)
Based on CANMEDS, no standard method
New market: consulting firms
Process
Scoping session: inventory of the aspects
relevant to the question
Session with leading academic
Search in relevant databases
Critical Appraisal
Summary / research synthesis
Recommendations / guidelines
Multi Source Feedback: scientific evidence
223 primary studies on MSF, 42 relevant
6 meta analyses or systematic reviews on
MSF, 3 relevant
18 meta analyses or systematic reviews on
‘feedback’ or ‘performance appraisal’,
5 relevant
Content of the feedback (neg vs pos)
Way of the delivery of the feedback
Interpretation of the feedbackPersonality of the
rateeFeedback orientation
of the ratee
Type and number of ratersSelection of
raters
Rater reliablity (patients, nurses,
colleagues)Type of
response scale
Development vs perfomance appraisal
Organizational culture
Perceived procedural justice
Multi Source Feedback: main factors
Lessons learned I
New approach
Recalibrates the power dynamics (accountability!)
The profit is in the process
Different (better?) decisions were made
Doctors love it!
Lessons learned II
Hard for individual managers
It starts with the senior management team
It’s all about accountability
Support system
EBP > Planning & Control