the bigger picture · the bigger picture optimizing electronic audit and feedback through ... audit...
TRANSCRIPT
![Page 1: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/1.jpg)
The Bigger Picture Optimizing electronic audit and feedback through unobtrusive quantitative process evaluations
Wouter Gude
@WouterGudeAcademic Medical Center / University of Amsterdam
Department of Medical Informatics
![Page 2: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/2.jpg)
103
Effect on health outcomes
Audit andfeedback intervention
![Page 3: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/3.jpg)
▪ Aim to increase our understanding of how interventions
achieve effects, by assessing:
– Quality of implementation, causal mechanisms, and contextual factors associated with outcome variation
▪ Often use qualitative research methods
– Interviews, focus groups, observations
Process evaluations
104
![Page 4: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/4.jpg)
Qualitative evaluations
105
![Page 5: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/5.jpg)
Qualitative evaluations
106
![Page 6: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/6.jpg)
Qualitative evaluations
107
What’s the bigger picture?
![Page 7: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/7.jpg)
▪ The challenge is to understand and optimize A&F interventions
▪ We suggest the mechanism through which A&F brings about change
can be quantified
▪ The mechanism can be studied by harnessing data that are
routinely collected as a by-product of using digital interventions in real-life
Optimizing A&F
108
![Page 8: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/8.jpg)
109
Section Divider:Title goes here
![Page 9: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/9.jpg)
The information value chain
110
• There are fewer events as we move down the chain but each event becomes increasingly more useful
Coiera 2015: Guide to Health Informatics 3rd Ed.
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 10: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/10.jpg)
The information value chain
111
A GP prescribing non-selective beta blockers in a patient with asthma is alerted by a CDS system that this may cause exacerbations.
Example: clinical decision support
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 11: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/11.jpg)
The information value chain
112
A GP prescribing non-selective beta blockers in a patient with asthma is alerted by a CDS system that this may cause exacerbations. When the GP notices the alert…
Example: clinical decision support
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 12: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/12.jpg)
The information value chain
113
A GP prescribing non-selective beta blockers in a patient with asthma is alerted by a CDS system that this may cause exacerbations. When the GP notices the alert and decides to cancel the prescription…
Example: clinical decision support
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 13: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/13.jpg)
The information value chain
114
A GP prescribing non-selective beta blockers in a patient with asthma is alerted by a CDS system that this may cause exacerbations. When the GP notices the alert and decides to cancel the prescription this will affect thepatient’smedication regimen…
Example: clinical decision support
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 14: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/14.jpg)
The information value chain
115
A GP prescribing non-selective beta blockers in a patient with asthma is alerted by a CDS system that this may cause exacerbations. When the GP notices the alert and decides to cancel the prescription this will affect thepatient’s medication regimen and can reduce the risk of asthmaexacerbations.
Example: clinical decision support
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 15: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/15.jpg)
The information value chain
116
• RCTs often already report the final stages (one or both)
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 16: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/16.jpg)
The information value chain
117
• RCTs often already report the final stages (one or both)
• Earlier stages could explain the (lack of) events in the final stages i.e. theinterventions’ observed effectiveness
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 17: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/17.jpg)
The information value chain
118
• RCTs often already report the final stages (one or both)
• Earlier stages could explain the (lack of) events in the final stages i.e. theinterventions’ observed effectiveness
• Assessing how success in one stage actually translates into good resultsat the next stage is of key importance
InteractionInformation
receivedDecisionchanged
Care processaltered
Outcomechanged
![Page 18: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/18.jpg)
▪ Provide novel opportunities to observe the entire information value chain because they can log everything that happens, such as:
▪ Whether, which, and how users interacted with the system– E.g., logins, mouse gestures, key strokes, page views (stage 1)
▪ Which and under what circumstances information was displayed– E.g., content of a medication alert for specific patients (stage 2)
▪ Clinical decisions
– E.g., changes to medication prescriptions (stage 3)
Digital interventions
119
![Page 19: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/19.jpg)
Example: CARDSS Online
120
![Page 20: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/20.jpg)
Example: CARDSS Online
121
50 feedback sessions
614 indicator valueswere flagged as
below the benchmark
379 indicator valueswere targeted with
an action
31 actions werecompleted
No indicatorsshowed significant
improvement
Interaction Information Decision Process Outcome
Significant loss of events down the chain! *
Question: what drives clinicians to act upon feedback? **
** Gude et al. BMJ Qual Saf 2016 How does audit and feedback influence intentions of health professionals to improve practice? A laboratory experiment and field study in cardiac rehabilitation.
* Gude et al. Implement Sci 2016 Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation.
![Page 21: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/21.jpg)
122
Section Divider:Title goes here
![Page 22: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/22.jpg)
123
Analysis of the information valuechain can reveal obstructions in the specific parts of the processand explain variations in observedeffects
This increases our understanding of health informatics interventions, and guides topics for further investigation (e.g. qualitative)
![Page 23: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/23.jpg)
What’s the bigger picture?
![Page 24: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/24.jpg)
![Page 25: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/25.jpg)
Thank you
Academic Medical Center / University of Amsterdam
Department of Medical Informatics
Wouter Gude
![Page 26: The Bigger Picture · The Bigger Picture Optimizing electronic audit and feedback through ... Audit and feedback intervention Aim to increase our understanding of how interventions](https://reader033.vdocument.in/reader033/viewer/2022051813/6031b7e99c179051f2704f24/html5/thumbnails/26.jpg)
THANK YOU.Wouter Gude, PhD candidate
Academic Medical Center, University of Amsterdam, NL
@WouterGude