quality forum new technologies (sessiond7)

Learning and Performance Improvement

Dr Dean Jenkins

Medical Education Consultant, BMJ Learning

Declarations of interest

• Employed by BMJ learning• Does have services and products to sell in relation

to this talk

An example project

• Blended learning programme in diabetes called the ‘Diabetes Academy’• Broad needs assessment in four countries• Communities of Practice theoretical framework• Formative evaluation of the project

Diabetes Academy

• Pilot project in 2010

• 4 collaborators– BMJ Learning (UK), AxDev (Canada), EIMSED (Austria), QUAIME (Germany)

• Supported by Merck

Diabetes Academy - purpose

• Diabetes in primary care• General / family practitioners in the four countries (60 in total)• Designed as a pilot project to evaluate the processes and feasibility of

this approach

Diabetes Academy - gaps

• Analysis of performance gaps / barriers– Review of English and German literature– Qualitative and quantitative evaluation data from

national data of four collaborators

• e.g. included outcomes from DNAT– Diabetes Needs Assessment Tool. An online

knowledge assessment tool in diabetes designed to prioritise reading for primary care physicians.

– Schroter S, Jenkins D, Playle R, et al. Evaluation of an online Diabetes Needs Assessment Tool (DNAT) for health professionals: a randomised controlled trial. Trials. 2009;10:63.

Diabetes Academy – PI Rx

• Collaborators met and prioritised gaps for intervention. Criteria:– common to at least three countries– amenable to the intervention proposed

• 4 priority gaps / barriers identified– Assessment of treatment targets– Lack of knowledge of diagnostic criteria– Drugs’ mode of action– Suboptimal team integration

Diabetes Academy - CoP

• “Communities of Practice” based intervention designed.

• Communities of Practice are groups of individuals who share and interest, a concern, a problem or a passion. They interact on an ongoing basis. They develop knowledge and expertise in their particular practice.

• Chosen primarily because of the complexity of the gaps and barriers identified and that it had a robust theoretical basis that could help frame the evaluation.

Diabetes Academy - the blend

• Face to face orientation meeting• Case-based discussion on-line• Diabetes knowledge testing on-line• Reading materials on-line• Virtual classrooms on-line• Face to face seminars• Diabetes experts supporting many of the

above interventions

Diabetes Academy - outcomes

• Formative evaluation of the pilot• Despite short timescale (3 months) Communities of

Practice were starting to form• Varied use of interventions – many lessons learnt on

‘pull through’ of participants• Some examples of self-reported performance

improvement• Currently being prepared for submission for

publication

Diabetes Academy - perspective

• Performance improvement interventions are complex• Cost is high• Expertise is required for the design of educational

interventions• Motivation of participants and ‘pull through’ tactics

need to be considered• Evaluation of outcomes is challenging• Community of Practice is a useful theoretical

framework

Why learn? How to learn and link to performance?

Key factors

Educational Interventions – why?

• Teaching stuff (so they know) < --- > improving (so they do their work better)

• Teach individuals < --- > empower individuals and teams• Train health professionals < --- > improve patient health

Measure performance

• Consider performance improvement as the specific target of educational interventions.– PI-CME– Education / learning associated with Clinical Audit,

Clinical Governance, Staff development

• Provide learners with:– Data, data, data– Feedback, feedback, feedback

Learning outcomesMoore’s “Expanded Outcomes Framework”. JCEHP 2009;29(1):1-15

e.g. Measuring performance (level 5) outcomes

• Objective:Observation of performance in patient care setting; patient charts; administrative databases.

• Subjective:self-report of performance

Health Professional education

PI-CME / clinical governance

Summary

• Performance and Quality Improvement should be the target of educational interventions. Aim high.– Numbers through a training programme is not as

important as better patient health

• Education needs formative evaluation (to improve) as well as summative evaluation (to prove it works).

• Education should be learner-centred, social and work-based.

E- learning

Dr Helen Morant • BMedSci, MBBS, MSc (Science Media Production)• Editor, Online Learning, BMJ Learning

Outline

• The book is closed on whether “e-learning” works

• Definitions restrict

• We know the principles of education that are effective

• How technology can serve those?

Motivations for learning

• Getting the job– Professional qualifications

• Keeping the job– Certification– Appraisal– Development

• Helping people• Curiosity

The book is closed on whether e-learning works

• It is “as good as traditional methods”• The really important research questions are

about– Medium– Context– Objectives– Cost effectiveness

• We lived on farms, then we lived in cities, and now we're going to live on the internet!

Define: definition

• Computer?• Social network?• Podcast?• The Cloud?

• Think broadly to innovate…

We know what principles of education are effective:

• Deep

• Mixed

• Personal

• Active

• Repetitive

• Competitive

Why do we use computers?

• Linking

• Structuring

• Storing

• Delivering

• Repurposing

Applications

• Blogs• Wikis• Microblogs• Social networks• Bookmarking• Videos• Games• 2nd life

Who’s driving this car?

“we don’t even know what this thing is yet”– Defining how people should use Twitter is a losing battle -

conventions and application (either discrete technology of functionalities) grow organically from the users.

– This is good.

“he didn’t need to tell us. Our litigators are capable of doing a Google search”– In 10 years time, you will not be able to protect intellectual

property rights. Discuss.– You will neither able to claim credit for nor be liable for what

you put the internet. Discuss.

Tell me a story….

• Case studies

• Interaction with patients in real life

• Video and audio - Patient Voices

• Branched learning - virtual patients

Life is messy

Time and movement• Technology allows things that previously had to be synchronous to be asynchronous and vice versa

– Synchronous: lectures– Asynchronous: online lectures

– Asynchronous: letters to the editor– Synchronous: instant messaging

• Portability

References and reading

• Virtual Patient design (RVC slides) http://www.slideshare.net/ctrace/virtual-patients-novice-2011-7103730 • Instructional design variations in internet-based learning for health professions education: a systematic

review and meta-analysis. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Acad Med. 2010 May;85(5):909-22.

• Exploring the Value of Social Media and Medical Meetings http://cmeadvocate.com/2011/03/24/social-tuesday-at-7th-meetings-management-forum.aspx

• Internet-based learning in the health professions: a meta-analysis. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. JAMA. 2008 Sep 10;300(10):1181-96

• The failure of e-learning research to inform educational practice, and what we can do about it David A. Cook Medical Teacher 2009, Vol. 31, No. 2 : Pages 158-162

• Internet-based medical education: a realist review of what works, for whom and in what circumstances Geoff Wong, Trisha Greenhalgh, and Ray Pawson. BMC Med Educ. 2010; 10: 12.

• The impact of E-learning in medical education. Ruiz JG, Mintzer MJ, Leipzig RM. Acad Med. 2006 Mar;81(3):207-12.

• Learning Effectiveness: What the Research Tells Us - Karen Swan in Elements of Quality Online Education: Practice and Direction Edited by John Bourne & Janet C. Moore Copyright ©2003 by Sloan-C™

Outline

• Review evidence for new technologies in quality improvement

• A “Communities of practice” example• Performance improvement learning• How technology drives and challenges

learning• Questions