resident research using a logic model to design, implement and evaluate a program mary jo knobloch,...
TRANSCRIPT
Resident Research Using a Logic Model to Design, Implement and Evaluate a
Program
Mary Jo Knobloch, MPH, CPHWREN Conference 2009
Association of American Medical Colleges Study
• Primary healthcare professionals need more training in basic research skills and increased opportunities to become involved in research
• Research education in the early years of medical training associated with continued interest in scholarly activity
Objectives
• To demonstrate use of a logic model as a framework for program development, implementation and evaluation
• To present preliminary results of 2008 and 2009 assessment of first year residents
Current Setting
• Marshfield Clinic Residency Program• Each program different in research learning approach
– 5 Dermatology – 11 General Surgery – 19 Internal Medicine – 8 Medicine/Pediatric– 11 Pediatric – 2 Pharmacy
Logic Model Framework
• Basic Framework – A living document
• Inputs
• Outputs – Activities and Participation
• Outcomes-Impact– Short – Medium – Long
UWEX Logic Model Site
• http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Short-term Outcome Measures Using Resident Research Database
THE NUMBERS -
• Number of resident projects
• Completed resident projects
• Abstracts, posters, manuscripts
• Number of active research mentors
• Diversity of project study designs– QI and public health or advocacy projects
Resident Research DatabaseTracking the Short-Term Outcomes
Medium and Long-term Outcomes
Using Assessment Tool
• Increase in knowledge and skill level related to basic research methods and processes
• Increase in knowledge related to identification of study design and statistical methods
• Increase in skill level related to literature searches
• Change in confidence level related to understanding and implementation of research
Medium-Long Term Outcomes
• Continued interest in scholarly activity
• Use of evidence to guide practice
• Ongoing provision of multiple training opportunities
Results to Date
• Projects (the numbers)
• Assessment results (the learning)
Projects
• Incidence of Eosinophilic Esophagitis in MESA• Relationship of Urine Cotinine Levels and
Wound Implications in Patients Undergoing Reduction Mammoplasty
• Effects of HMR Weight Loss Intervention Program on Cardiovascular Disease Risk Factors
• Effect of Hyperglycemia on Outcomes in Acute Exacerbation of COPD
Projects • CNS Blastomycosis in Marshfield Clinic System: A Case Series
• A Pilot Project to Promote Early Literacy
• Cardiovascular Morbidity and Mortality among SLE Patients in a Rural Population-based Cohort – ICTR project
• Severely Suppressed Bone Turnover and Atypical Skeletal Fragility• Maja Visekruna, MD, Medicine/Pediatrics
• Faculty and Resident Perspectives on the Dermatology Integrative Competency Curriculum
• Modeling Clearance of Warfarin
Projects
• Implementation of a Resident Well-Being Survey
• Hamman-Rich Syndrome (Idiopathic Acute Interstitial Pneumonitis)
• Evaluating Clinical Compliance with IDSA Guidelines for Diagnosis and Management of Lyme Disease
• Incidence and Description of Anaplasmosis
• Rapunzel Syndrome in a 5 Year-Old (Case Report)
• Risk Factors Related to Recurrence of Umbilical Hernia
Projects
• Renal Cell Carcinoma – Incidence Rates in Wisconsin
• A Retrospective Review of Patients Diagnosed with Stenosing Tenosynovitis (Trigger Finger and Thumb)
• Comparison of Wounds Before and After Sharp Debridement Using 16S DNA Sequencing
• Identification of Drug Therapy Problems through Medication Reconciliation in Heart Failure Patients
• The Role of Pharmacist Evaluation in Activation of Adult Asthma Patients (ICTR project).
• A Public Awareness Campaign about Insulin Dependent Diabetes
Projects• Natural History of Idiopathic Neonatal Hypertension and the Long
Term End Organ Effects in Children with Neonatal Hypertension
• Do Clinicians Document the AAP Recommended Clinical Information Used to Diagnose and Manage Otitis Media with Effusion?
• Defining Services and Referral Patterns for Infants with Positional Plagiocephaly within the Marshfield Clinic System
• Implementing a Pharmacy Continuing Professional Development (CPD) Project at Marshfield Clinic
• Factors Associated with Successful Matching and Residency Matriculation by Non-traditional Applicants to Dermatology Residency Programs
• Effects of Metabolic Syndrome on Disease State and Long-Term Outcomes in Patients with Colorectal Cancer
Projects
• Are Lytic Bacteriophages against Staph Aureus Found in the Anterior Nares of Humans?
• A Rare Presentation of Painless Aortic Dissection as Seizures
• Analysis of Dermatology Resident Self-Reported Successful Learning Styles and Implications for Core Competency Curriculum Development
• Caustic Ingestion in a 2-year old (Case Report)
• Assessment of Medical Residents’ Knowledge of Drug Interactions after an Educational Intervention
Combined Results for 2008 and 2009 First-Year Residents(the learning)
• Assessment of attitude, confidence level and knowledge– 33 residents
Attitude Questions
• To be an intelligent reader of the literature, it is necessary to know something about statistics– 100% agree or strongly agree
Attitude Questions
• I often use statistical information in forming opinions or making decisions– 51% disagree or strongly disagree
• Given the chance, I would like to learn more about biostatistics– 75% agree or strongly agree
Level of Confidence Questions
• Finding clinical practice guidelines for patient care questions– 57% say they have fair or little amount of
confidence
– Exploring evidence using Cochrane Database• 39% say little or no confidence
Confidence Questions
• Identifying factors that influence a study’s statistical power– 78% say fair, little or no confidence
Confidence Questions
• Interpreting P-value for a given result– 70% say fair, little or no confidence– 30% say a lot or complete confidence
Knowledge Questions
• Which of the following is true regarding p-values?– 27% correct– 73% incorrect
• Which of the following is NOT considered a vulnerable population?– 38% correct– 62% incorrect
Knowledge Questions
• Which of the following is an example of a testable null hypothesis – 40% correct– 60% incorrect
Knowledge Questions
• Which of the following is true with regard to the differences between efficacy and effectiveness– 33% correct– 66% incorrect
• Which one of the following measures of central tendency is least affected by outliers? – 37% correct – 63% incorrect
Knowledge Questions
• Which statement is not true about confidence intervals?– 36% correct– 63% incorrect
• The purpose of a double-blind or double-masked study– 97% correct– 3% incorrect
Knowledge
• These 2008/2009 first year residents had difficulty answering questions related to
– Study design – Identifying discrete versus continuous variables– Differences between efficacy and effectiveness– Definition of bias – Differences between mean, median, mode– Sensitivity and specificity– Definition of prevalence
Conclusions
• A program planning tool usually used for public health or intervention planning or grant planning can be used to set up and evaluate a resident research program
• Tracking systems must be in place
• Preliminary assessment of resident attitude and knowledge of basic epidemiology and biostatistics demonstrates the need for continued work in teaching research methods and skills
Questions?