measuring organizational readiness for quality improvement elizabeth yano, phd va greater los...
TRANSCRIPT
Measuring Organizational Measuring Organizational Readiness for Quality ImprovementReadiness for Quality Improvement
Elizabeth Yano, PhDElizabeth Yano, PhDVA Greater Los Angeles HSR&D Center for the VA Greater Los Angeles HSR&D Center for the
Study of Healthcare Provider BehaviorStudy of Healthcare Provider Behavior
UCLA School of Public HealthUCLA School of Public Health
PerspectivesPerspectives
Measuring organizational readiness for change Measuring organizational readiness for change – Traditional approaches (attitudes, beliefs, culture)Traditional approaches (attitudes, beliefs, culture)– Expanding to clinical structure and care processesExpanding to clinical structure and care processes
Examples of studies of organizational influences Examples of studies of organizational influences on qualityon quality– National VA reorganization (emphasizing PC)National VA reorganization (emphasizing PC)– VA quality improvement interventions (QUERI)VA quality improvement interventions (QUERI)
Pre-implementation organizational assessmentPre-implementation organizational assessment
Post-implementation predictors of sustainability/spreadPost-implementation predictors of sustainability/spread
Review key informant surveys as one methodReview key informant surveys as one method
Organizational ReadinessOrganizational Readiness
““Readiness Readiness represents a state of mind about the represents a state of mind about the need for an innovation and the capacity to need for an innovation and the capacity to undertake change...”undertake change...”““Readiness Readiness consists of people’s beliefs, consists of people’s beliefs, attitudes, and intentions about the desirability of attitudes, and intentions about the desirability of changes, and perceptions about the ability of changes, and perceptions about the ability of individuals and the organization to successfully individuals and the organization to successfully make those changes.”make those changes.”““ReadinessReadiness represents the predisposition to represents the predisposition to unfreeze established patterns of behavior.”unfreeze established patterns of behavior.”
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Organizational ReadinessOrganizational Readiness
Commonly measured as:Commonly measured as:– Culture (innovativeness, flexibility, climate)Culture (innovativeness, flexibility, climate)– Definition of rolesDefinition of roles– Investment of resources (including competition)Investment of resources (including competition)– Past contracts or agreements (can limit Past contracts or agreements (can limit
changes in behavior)changes in behavior)– Threats to power—control over decisions, Threats to power—control over decisions,
resourcesresources
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Contribution of Organizational Contribution of Organizational Readiness Measures to VA QualityReadiness Measures to VA Quality
Culture (staff surveys)Culture (staff surveys)– QI orientation associated with QI orientation associated with tobacco counseling rates tobacco counseling rates
Investments of resources/competition (PC leader survey)Investments of resources/competition (PC leader survey)– Sufficiency of clinical support arrangements accounts for Sufficiency of clinical support arrangements accounts for
substantial variation in prevention performancesubstantial variation in prevention performance
Past contracts/agreements (PC leader survey)Past contracts/agreements (PC leader survey)– Stringent PC patient assignment associated with lower breast & Stringent PC patient assignment associated with lower breast &
cervical cancer screeningcervical cancer screening
Control over decisions, resources (PC leader survey)Control over decisions, resources (PC leader survey)– PC practice autonomy over internal operations associated with PC practice autonomy over internal operations associated with
higher CRC screening rates and better diabetic control higher CRC screening rates and better diabetic control
Sources: Yano et al., JGIM, 2002; Soban & Yano, JACM, 2005; Yano HCOC, 2005; Goldzweig et al, AJMC, 2004; Jackson, et al., AJMC, 2005.
Organizational Readiness:Organizational Readiness: Expanding Beyond Attitudes, Beliefs, Culture Expanding Beyond Attitudes, Beliefs, Culture
Interventions associated with changes in qualityInterventions associated with changes in quality– Organizational change (Organizational change (biggest effectsbiggest effects))– RemindersReminders– Audit and feedback/profilingAudit and feedback/profiling– Incentives Incentives
Organizational change interventions focus on Organizational change interventions focus on structures and processes of care (org “enablers”)structures and processes of care (org “enablers”)– Not attitudes, beliefs, culture – hard to changeNot attitudes, beliefs, culture – hard to change– Clinical processes of care, management strategies, Clinical processes of care, management strategies,
clinic structure, provider training, decision supportclinic structure, provider training, decision support
Source: Stone, et al., Ann Intern Med, 2002.
Major Reorganization of VA Health Major Reorganization of VA Health Care System (1996-present)Care System (1996-present)
Historically, VA was…Historically, VA was…– Individually-managed hospitals focused on specialty careIndividually-managed hospitals focused on specialty care– Funded through prior-year cFunded through prior-year cost reimbursement ost reimbursement – Extremely poor quality of care reputationExtremely poor quality of care reputation
Congressionally approved VA reorganization…Congressionally approved VA reorganization…– Veterans’ integrated service networks (VISNs) (n=22)Veterans’ integrated service networks (VISNs) (n=22)– Network-level budget control and performance Network-level budget control and performance
agreementsagreements– Incentivized audit-and-feedback on quality/satisfactionIncentivized audit-and-feedback on quality/satisfaction– Funded by population served (capitated)Funded by population served (capitated)– Eligibility reform changed VA to health plan and payor Eligibility reform changed VA to health plan and payor – Computerized patient record system (CPRS) w/decision supportComputerized patient record system (CPRS) w/decision support– Primary care as platform for restructuring care deliveryPrimary care as platform for restructuring care delivery
VA Health Care System as the VA Health Care System as the Organizational ContextOrganizational Context
VA health care system largest in U.S.VA health care system largest in U.S.– 163 hospitals, >750 freestanding community-based 163 hospitals, >750 freestanding community-based
outpatient clinics, >130 nursing homes, >200 vet ctrsoutpatient clinics, >130 nursing homes, >200 vet ctrs– Served over 5 million patients in past yearServed over 5 million patients in past year
Affiliated with over 1,200 academic institutions – Including 107 medical schools, 55 dental schools– 110,000 students and trainees in >45 disciplines/year
> ½ of all U.S. MDs received part of their training in VA
VAVA market penetration growing rapidlymarket penetration growing rapidly– 20% of those <65 and 29% of those >6520% of those <65 and 29% of those >65
VA health care budget $25.9 billion (2003)VA health care budget $25.9 billion (2003)
VA Improves 12 of 13 Leading Quality VA Improves 12 of 13 Leading Quality Indicators (1995-2000)Indicators (1995-2000)
0
1020
3040
5060
7080
90100
HbA1c Flu shots CRC Screen ASA MI
19951996199819992000
Jha NEJM 348:22 2003
VA Beats Medicare 12-1 in 2000VA Beats Medicare 12-1 in 2000
0
10
20
30
40
50
60
70
80
90
100
Mammo Flu shots Pneumovac HbA1c DM eye DM lipid
MedicareVA
Jha NEJM 348:22 2003
Primary Care Organizational ChangesPrimary Care Organizational Changes
0
10
20
30
40
50
60
70
80
90
100
1993 1996 1999
PC budget
PC-based QI
Pt-PCP assignment
PC teams
PC Teams
Pt assignment
PC-based QI
Separate PC budget
Percent of VAs w/PC Program
Changes in PC Physician Volume/MixChanges in PC Physician Volume/Mix
0
1
2
3
4
5
6
7
8
9
10
1993 1996 1999
GIM MDs
Geriatrics MDs
IM Subspecialists
Psychiatrists
Mean FTEEs
Staff Alignment to Primary CareStaff Alignment to Primary Care
0
10
20
30
40
50
60
70
80
90
1993 1996 1999
MDs
NPs
PAs
MSWs
RNs
Percent of VAs with PC Staff Reporting Only to Primary Care
Changes in PC Resource SufficiencyChanges in PC Resource Sufficiency
0
10
20
30
40
50
60
70
80
90
Adminoffices
ExaminingRooms
Treatmentrooms
PCs Patienteducation
space
1993
1996
Percent of VAs Reporting Always/Mostly Sufficient
*
* * *
*
Organizational Readiness:Organizational Readiness:Implications for QI ResearchImplications for QI Research
BasicScience
ClinicalResearch
Health ServicesResearch
RoutineCare
TRANSLATION“new discoveries”
EFFICACY toEFFECTIVENESS“new treatments”
“new cures”“improved access”
“better quality of care”
IMPLEMENTATION“barriers”“barriers”“barriers”
TRIP over theproverbial“brick wall”
VA intramuralresearch program
Organizational Readiness:Organizational Readiness:Implications for QI ResearchImplications for QI Research
Health ServicesResearch
RoutineCare
• Measure organizational readiness• Use to select promising sites• Locally tailor QI intervention(s)
• Fixed characteristics (eg, urban/rural)• Mutable characteristics (change/adapt)
Organizational Readiness for QI:Organizational Readiness for QI:Preparing to Implement Depression Collaborative CarePreparing to Implement Depression Collaborative Care
Depression collaborative care model in 7 VAsDepression collaborative care model in 7 VAs– Depression care managerDepression care manager– PC-MH collaborationPC-MH collaboration– Informatics/registry (screening, reminders)Informatics/registry (screening, reminders)– Leadership supportLeadership support
Pre-intervention semi-structured telephone Pre-intervention semi-structured telephone interviews of all PC and MH leadersinterviews of all PC and MH leaders– Assess current screening and referral processesAssess current screening and referral processes– Assess local barriers (eg, turf, staffing gaps, history)Assess local barriers (eg, turf, staffing gaps, history)– Fed back into planning calls, adapted protocolsFed back into planning calls, adapted protocols
Sources: Parker LE, Yano EM, Rubenstein LV, 2003; Ficket et al, in prep.
Organizational Readiness for QI:Organizational Readiness for QI:Preparing to Implement HIV QI InterventionPreparing to Implement HIV QI Intervention
HIV quality improvement intervention trial HIV quality improvement intervention trial (16 VAs) (16 VAs) (Asch et al)(Asch et al)
– Group-based QI, audit-and-feedback, remindersGroup-based QI, audit-and-feedback, reminders
Used national HIV provider survey (n=118 VAs) Used national HIV provider survey (n=118 VAs) to examine how organizational factors affected to examine how organizational factors affected adoption of HIV QI activities adoption of HIV QI activities beforebefore starting trial starting trial – Assessed local QI activities, HIV guideline useAssessed local QI activities, HIV guideline use– Measured attitudes toward proposed QI modalities Measured attitudes toward proposed QI modalities – Evaluated regional, facility and practice variationsEvaluated regional, facility and practice variations– Fed back to team (site selection, adapted protocols)Fed back to team (site selection, adapted protocols)
Sources: Anaya, Am J Med Qual 2004; Korthuis et al, JAIDS, 2003; Yano, et al., Mil Med 2005.
Organizational Readiness for QI:Organizational Readiness for QI:Implementing/Sustaining Depression Collaborative CareImplementing/Sustaining Depression Collaborative Care
Onsite in-person stakeholder interviews Onsite in-person stakeholder interviews – Network, medical center, clinic siteNetwork, medical center, clinic site– PC and MH leaders, PC and MH providers, nurses, PC and MH leaders, PC and MH providers, nurses,
care managers, patients, consumer reps (n=106)care managers, patients, consumer reps (n=106)
Semi-structured interviews exploring Semi-structured interviews exploring implementation of each care model componentimplementation of each care model component– Leadership support/opinion leadersLeadership support/opinion leaders– Depression care manager interaction/contactsDepression care manager interaction/contacts– Provider interactions and ongoing education needsProvider interactions and ongoing education needs
Fed back to implementation/spread teams and Fed back to implementation/spread teams and developing “diffusion” toolsdeveloping “diffusion” tools
Different Measurement ApproachesDifferent Measurement Approaches
Knowledge/Evidence Base
QualitativeInterviews
Key InformantSurveys
ProviderSurveys
HIGH
LOW
Informs survey design
• know domains/items?• who has knowledge?• can you get to them?• will they cooperate?
• if variation important• if QI intervention requires their change• AND all of above
• telephone or in-person• different levels of interview structure• different levels of stakeholders/informants
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #1Step #1: Translate ideas into survey domains: Translate ideas into survey domainsExample: Example: Translate HIV QI strategic plan into domains:Translate HIV QI strategic plan into domains:– HIV screening policies and protocolsHIV screening policies and protocols– Practice arrangements for management of HIV diseasePractice arrangements for management of HIV disease– Provider ratings of effectiveness of diff QI interventionsProvider ratings of effectiveness of diff QI interventions– Potential barriers to adoption of HIV guidelinesPotential barriers to adoption of HIV guidelines
Example: Example: Evaluate PC organizational predictors of qualityEvaluate PC organizational predictors of quality– Institute of Medicine primary care domains (access, continuity…)Institute of Medicine primary care domains (access, continuity…)– Primary care strategic plansPrimary care strategic plans– PC practice managers (observation and interview)PC practice managers (observation and interview)
ExampleExample: Depression collaborative care implementation: Depression collaborative care implementation– Disaggregate care model components—explicitly open “black box”Disaggregate care model components—explicitly open “black box”
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #2Step #2: Select measures allowing benchmarking : Select measures allowing benchmarking against other health care organizationsagainst other health care organizationsExampleExample: VA QUERI HIV & HCSUS: VA QUERI HIV & HCSUS
ExampleExample: VA, NCQA PSAS, & Kaiser: VA, NCQA PSAS, & Kaiser
ExampleExample: VA & DHHS Office of Women’s Health COEs: VA & DHHS Office of Women’s Health COEs
Step #3Step #3: Develop new measures that match : Develop new measures that match structure-process or -outcome model or QI goalsstructure-process or -outcome model or QI goalsLiterature review, expert panel methods—build on evidenceLiterature review, expert panel methods—build on evidence
Talk to “real people” who live in world you are studyingTalk to “real people” who live in world you are studying
Begin with qualitative interviews or focus groupsBegin with qualitative interviews or focus groups
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #4Step #4: Test and adapt survey to target health : Test and adapt survey to target health care settingscare settingsCognitive interviews with sample respondentsCognitive interviews with sample respondents
Vary types of organizations included (big/small)Vary types of organizations included (big/small)
Develop multiple modules if neededDevelop multiple modules if needed– By setting (hospitals, freestanding outpatient clinics)By setting (hospitals, freestanding outpatient clinics)– By respondent type (hosp director, PC chief, lab tech)By respondent type (hosp director, PC chief, lab tech)
Step #5Step #5: Identify key informants: Identify key informantsDesired knowledge base, incentives to participateDesired knowledge base, incentives to participate
Social desirability and need for validation, politicsSocial desirability and need for validation, politics
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #6Step #6: Sampling organizations: Sampling organizationsWhat’s in it for the organization?What’s in it for the organization?
Sampling to represent what?Sampling to represent what?– Types of organizations, units w/in Types of organizations, units w/in
organizations, different size and complexityorganizations, different size and complexity– Regions, urban/rural locations, other area Regions, urban/rural locations, other area
characteristicscharacteristics
Obtaining their cooperation…Obtaining their cooperation…– Leadership support, uses of data, IRB, HIPAALeadership support, uses of data, IRB, HIPAA– Funding to compensate for administrative timeFunding to compensate for administrative time
Where doyou getthis kind ofinformation?
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #7Step #7: Field preparation : Field preparation Identify and market to venues common to respondents Identify and market to venues common to respondents
Determine regular communication options Determine regular communication options – Management meetings, conference calls, broadcast Management meetings, conference calls, broadcast
fax, advance mass mailingfax, advance mass mailing
Involve senior leaders/opinion leaders – spokespersons Involve senior leaders/opinion leaders – spokespersons
Market value of participation Market value of participation – Demonstrate previous uses of data (“good works”)Demonstrate previous uses of data (“good works”)– Offer incentive (eg, summary of survey results, Offer incentive (eg, summary of survey results,
publications order form, financial)publications order form, financial)
Contact organizations and talk to support staffContact organizations and talk to support staff
Measuring Organizational Characteristics Measuring Organizational Characteristics Using Key Informant SurveysUsing Key Informant Surveys
Step #8Step #8: Administer surveys: Administer surveysHardcopy express-mail with prepaid returnsHardcopy express-mail with prepaid returns
Web-based with varying email introductions and Web-based with varying email introductions and remindersreminders
Quality review of survey content with active follow-up of Quality review of survey content with active follow-up of missing data and non-respondentsmissing data and non-respondents– 2-week second wave mailouts2-week second wave mailouts– 4-week telephone follow-up4-week telephone follow-up
Continual data entry (if hardcopy), quality checkingContinual data entry (if hardcopy), quality checking
Follow-up postcards and repeat announcements in Follow-up postcards and repeat announcements in original venuesoriginal venues
Example Key Informant SurveysExample Key Informant Surveys
Primary care practice organizationPrimary care practice organization (93, 96, 99)(93, 96, 99)
– Care arrangements, teams, staffing, authority, Care arrangements, teams, staffing, authority, resources, QI, decision support, care coordination, resources, QI, decision support, care coordination, profiling, incentives, management structureprofiling, incentives, management structure
HIV practice structure/delivery modelsHIV practice structure/delivery models (00)(00)
– Screening, PC vs. specialty management, HIV clinics, Screening, PC vs. specialty management, HIV clinics, staffing, provider experience, HIV case management, staffing, provider experience, HIV case management, HIV guideline use, barriers, provider preferencesHIV guideline use, barriers, provider preferences
Women’s health care delivery organization Women’s health care delivery organization (01)(01)
– Clinic structures, service availability, referral Clinic structures, service availability, referral arrangements, decision support, QI, leadership, arrangements, decision support, QI, leadership, policies, staffing, authority, provider trainingpolicies, staffing, authority, provider training
93%-100%
83%
82%-100%
Example Key Informant SurveysExample Key Informant Surveys
Clinical practice organizational survey Clinical practice organizational survey ((05)05)
– 3 modules:3 modules:Network directors (n=21) (Network directors (n=21) (~$1 billion each)~$1 billion each)Chiefs of staff (aka medical directors) (n=160)Chiefs of staff (aka medical directors) (n=160)Primary care clinic directors (n=259)Primary care clinic directors (n=259)
– Mapped to prior VA surveys Mapped to prior VA surveys time trends time trends– Mapped to NCQA PSASMapped to NCQA PSAS© © and Kaiser IT surveys and Kaiser IT surveys
benchmarkingbenchmarking
Women’s primary care organizational survey Women’s primary care organizational survey ((06)06)
– Senior WH clinician or medical directorSenior WH clinician or medical director– Classify every VA by organizational taxonomyClassify every VA by organizational taxonomy– Evaluate quality of care within different typesEvaluate quality of care within different types
