seeing where we are going: why evaluating the revised bih model is important and bih mcah mis update...
TRANSCRIPT
Seeing Where We are Going: Why Evaluating
the Revised BIH Model Is Importantand
BIH MCAH MIS Update
David Dodds, PhD, MPHNichole Sturmfels, MPH
BIH Statewide Meeting—March 15, 2011
• What is the foundation for the BIH program evaluation?
• Why is evaluation important?• What are the roles of BIH staff in evaluation?• What are the plans for data collection and the
BIH MCAH MIS?
Key Questions
BIH Statewide Meeting—March 15, 2011
Tools for Evaluation
BIH MCAH MISClient Data BookConceptual
Framework
BIH Statewide Meeting—March 15, 2011
DRAFT
To improve African American infant and maternal health in
California anddecrease Black:White health disparities and social inequities for women and infants
BIH Activities
Ultimate Goal
INDIVIDUAL•Increased health knowledge•Increased healthy behaviors•Increased empowerment through improved life skills and coping skills•Increased receipt of quality medical, social & mental health services•Decreased unplanned pregnancy•Increased social support both for & among the women•Decreased stress by mobilizing resources and services•Improved parenting•Increased bonding between mother & infant•Improved infants’ developmental milestones
COMMUNITY•Increased community and provider knowledge & cultural competence•Increased community partnerships & linkages among service agencies•Decreased stressors in the community through community & provider/agency action•Improved understanding among community and providers of influence of social inequities on health
INDIVIDUALServices provided to African-American women, infants, and their families & partners include:• Referrals to medical, social & mental health services• Health education• Social and group support• Identifying resources• Self-advocacy
COMMUNITY•Promote community and provider engagement and advocacy•Educate community and providers about influence of social inequities on health
GUIDING PRINCIPLES1. Comprehensive and integrated: Address multiple risk factors and use multiple strategies 2. Multi-level: Address individual, community, service systems and societal levels, with empowerment focus3. Collaborative: Partner with community providers and agencies with similar activities4. Community-driven: Developed, implemented and evaluated by local communities5. Evidence-based: Developed from proven or promising strategies; impact is measurable6. Culturally competent: Designed & implemented in a culturally-competent manner7. Staff training and professional development: Conducted to ensure the BIH activities are provided effectively
Problem
• Poor birth outcomes• Social isolation• Lack of health knowledge
• Lack of access to quality health care
• Poverty• Racism• Environmental stressors
• Maximizing impact of the program (numbers served & effect size)
• Lack of cultural awareness and skills among providers
Conceptual Framework
Intermediate Outcomes
Outcomes Measured in the BIH Revised Model are Based on Scientific Literature
BIH Statewide Meeting—March 15, 2011
Why are intermediate outcomes important?
• Intermediate outcomes are on the critical pathway toward the ultimate goal of improving African American infant and maternal health in California and decreasing Black:White disparities.
• Measuring these outcomes is important for quantifying the good work being done at sites—the previous BIH model did not adequately measure these outcomes.
BIH Statewide Meeting—March 15, 2011
Transactional Model of Stress and Coping*
Key Concepts•Evaluation of stressors•Coping efforts:
• Information Seeking• Optimism• Social Support• Positive Psychology
*Lazarus, Folkman, and Moskowitz
BIH Curriculum Topics and ActivitiesUnderstanding and handling stressRelaxation techniquesGratitude circleStrengths and virtues
Data Book Tool for Measuring OutcomesAssessment Scale C in Prenatal and Postpartum Assessments 1 and 3 BRIEF RESILIENCE SCALE: Resilience refers to one's ability to bounce back or recover from stress
Example: Scientific Concepts in theBIH Curriculum and Data Book
BIH Statewide Meeting—March 15, 2011
Why is evaluation important?
• To measure program progress toward the ultimate goals of BIH.
• To document the effectiveness of the revised program model. – Provide the best possible intervention for clients
within program resources.– Demonstrate the program’s usefulness to federal
agencies, state legislators, county superintendents, and other policy makers.
– Improve accountability for program funding.
BIH Statewide Meeting—March 15, 2011
What is the BIH Staff role in the evaluation?
• Program Fidelity– Ensure that all BIH sites are conducting the same standardized
model. – Clearly document any modifications to the curriculum or case
management process.• Data Quality
– Collect all data in client data book.– Ensure quality of data collection (paper data book) – Ensure quality of data entry (BIH MCAH MIS)
• Program Feedback– Group Sessions: Complete Facilitator Feedback & Summary after
every session– Case Management: Complete FHA Feedback Form monthly
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS – Timeline 2009
• BIH Data and Evaluation Sub-Committee– 6 initial conference calls with 22 participants
(BIH, MCAH, UCSF)• Report at June 2009 Statewide BIH Meeting
concluded:“The current BIH MIS will need to be extensively
revised or a new MIS adopted in order to improve usability and to measure outcomes of the redesigned BIH program.”
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS – Timeline 2010
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS – Timeline 2011 (Working Plan Subject to Change)
• Winter 2010/2011: Data base structure– Programmers build database where Data Book information
will be stored. • Spring 2011: Beta testing and training
– Beta testing by MCAH, Branagh Information Group, and some BIH users
– Training by Branagh Information Group • Summer 2011: MIS Release 1
– Data collection and some standard reports• Winter 2011/2012: MIS Release 2
– Case management and more standard reports• TBD 2013: MIS Release 3
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS - Features
• Release 1—Data Entry– Centralized, secure database hosted on a state server– Users will login through an internet browser– Data entry screens are modeled after Data Book forms– 7 standard reports
• Release 2—Case Management– Additional standard reports and user functionality
• Release 3—Analysis – All standard reports complete (approximately 30)– Ad hoc queries for any information in the database– Data export to Excel
BIH Statewide Meeting—March 15, 2011
Data Entry
Site
BIH Clients
Statewide (ALL)AlamedaBerkeleyContra CostaFresno
Family Health Advocate (FHA)
All FHADanielleKatieJane Jackie
Choose site
Choose data entry mode
Choose FHA
Planned Navigation
BIH Statewide Meeting—March 15, 2011
BIH Clients
All clientsDnielleKatieJane Jackie
First Name
PNA1 PNA2 PNA3 Birth Outcome
PPA 1
Danielle
Vicky
Sara
Jackie
Kim
FHA all enrolled clients view: form status
Choose all enrolled clientsAll clients
All enrolled clientsDanielleVickySaraJackieKim
Navigate to specific client form:White/empty = no data entered into formYellow = form data entered but not validated Red = form data has not passed validationGreen = form data validated and saved (finalized)
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS Screenshot
BIH Statewide Meeting—March 15, 2011
BIH MCAH MIS Screenshot
BIH Statewide Meeting—March 15, 2011
Computing Requirements for BIH MCAH MIS
• Policy Alert # 2010/11-01 (Effective May 1, 2011)• System requirement details:
– High-speed internet connection (DSL, etc.)– Silverlight (Free Microsoft operating system update)– .NET (Free Microsoft operating system update)– Internet explorer 7.0+– 2GHz+ CPU, 2GB + RAM, 1024 x 768 or better display
• May need your local IT to allow software updates• Branagh Information Group tech support:
BIH Statewide Meeting—March 15, 2011
Data submission to MCAH beforethe BIH MCAH MIS is online
• Policy Alert 2010/11-02—Group 1 sites– Forms completed on or before February 28th, 2011
• Client Recruitment• Prenatal Assessment 1—Sections 1 & 2• Postpartum Assessment 1 (for postpartum entry)—
Sections 1 & 2• Group Sessions: Facilitator Summary & Feedback
• Policy Alert 2010/11-03—Group 2 sites– Forms completed on or before February 28th, 2011– Prenatal Supplemental Form
BIH Statewide Meeting—March 15, 2011
BIH MIS – Use and Plans
• How to use the BIH MIS now:– For clients under the “old” model, use Greenbook
and BIH MIS– When implementing the new model, use the hard
copy Data Book and eventually the new BIH MCAH MIS
– If your client is transitioning from old to new• Close out the client using the Greenbook and BIH MIS• Enroll the client again using the Data Book
• BIH MIS will eventually be phased out.
BIH Statewide Meeting—March 15, 2011
Summary• The evaluation will measure progress towards the ultimate
goals of the BIH program .– LHJs, MCAH, and UCSF each have a role in the BIH Program
evaluation. • The conceptual framework, Data Book, and BIH MCAH MIS
are all tools for evaluation.– These tools were developed with input from LHJs .– These tools are based on scientific literature.
• All LHJs prepare for the BIH MCAH MIS by ensuring compliance with system requirements.
• All LHJs submit some initial data to CDPH/MCAH for data entry.
BIH Statewide Meeting—March 15, 2011
Questions?
For questions about the Data Book, submitting data to MCAH, and the BIH MCAH MIS, contact:
Nichole [email protected]
For Help Desk support for the BIH MIS or BIH MCAH MIS, contact:
Branagh Information Group [email protected]
BIH Statewide Meeting—March 15, 2011