VIReC Database & Methods Cyberseminar Series

A Practical Guide to Using Mental Health Assistant Data:

The CDW Mental Health Domain

September 12, 2016

Eric Hawkins, PhD

Carol Malte, MSW

Seattle Center of Excellence in Substance Abuse Treatment and Education (CESATE), HSR&D Seattle Center of Innovation for Veteran Centered and Value-Driven Care,

VA Puget Sound Health Care System, University of Washington

Agenda • What is the Mental Health Domain? – Source of data

• What data are available in the Mental Health domain? – Included measures – Available data elements – Data extraction issues

• How can the data be used? – Data limitations and challenges

• Research example • Helpful resources

09/2016 2

Poll Question #1

What best describes your role as a user of the CDW Mental Health Domain? – Researcher

– Clinician

– Administrator/Policymaker

– Quality improvement

– Other (including non-VA)

09/2016 3

Presentation Focus

Information presented is relevant to:

Research Users Project-specific access

Operational Users Full access

Clinicians Site-specific access

09/2016 4

Poll Question # 2

Have you ever used the CDW Mental Health Domain data?

– Yes

– No

09/2016 5

WHAT IS THE CORPORATE DATA WAREHOUSE (CDW) MENTAL HEALTH DOMAIN?

09/2016 6

CDW Mental Health Domain

• Information related to mental health assessments and

assessment instruments

– Typically completed as part of standard clinical care

• Collected via the Mental Health Assistant

• VINCI documentation indicates

– Entry of information began October 2007 by some regions

– Information from all 4 regions occurred in March 2011

– Earlier data collected via health factors in some cases (PHQ-2,

PC PTSD Screen, AUDIT-C)

09/2016 7

Mental Health Assistant Software

809/2016

Mental Health Assistant Software • Source: information collected via the Mental

Health Assistant in CPRS – Released in 2002

– Graphical user interface for the Mental Health testing package

– Provides secure computer environment for patient administered testing

• Information also collected via clinical reminders and dialogue templates – Includes instruments associated with annual screening in VA

• AUDIT-C, PHQ-2, PC-PTSD

09/2016 9

Mental Health Assistant Administrator

09/201610

Clinical Reminder

09/201611

WHAT DATA ARE IN THE CDW MENTAL HEALTH DOMAIN?

09/201612

Tables in the CDW

Provides names

and details about

all available survey

measures

Provides scores for

each survey

administration

Provides link

between question

text and item

responses

Provides item

responses for non-

multiple choice

questions (date,

integer, etc.)

Provides text and type of each

question on the measure

Link between each measure and the

questions, provides question order

Link between

measure and

scoring details

Provides details on which items make up measurement

scales

Provides details on

how the measure

is scored

multiple choice Provides order for

multiple choice response items

MH.SurveyAdministration

Provides date/time,

location, and ordering

provider of each survey

administration by

patient

Dim.Survey

MH.SurveyResult

MH.SurveyAnswer

MH.SurveyAnswerText

Dim.SurveyQuestion Dim.SurveyContent Dim.SurveyScaleGroup Dim.SurveyScale Dim.SurveyScoringKey

Dim.SurveyChoice

Provides text for

Dim.SurveyChoiceType

13

https://vaww.cdw.va.gov/Metadata/Reports/Images/ Mental%20Health_5414.jpg

Mental Health Domainresponse items

Available Measures Query the Dim.Survey

domain table for list of

measures

• May be multiple versions

of the same measure

Over 200 measures

listed

• Administration data

available on 97 measures

09/201614

Measure Administrations • Some measures rarely

administered or used by few sites

• Others frequently used by all sites PTSD Screen, PHQ-2, AUDIT-C

o AUDIT-C: most administered, all sites using

o Brief Addiction Monitor:

• fewer administrations

• not all sites using

• need to account for different versions of the measure

15

1

Administrations of Selected Measures Survey

Title 2015 2014 2013 2012 2011 2010 2009 2008 Name

Alcohol Use Disorders AUDC 5,237,680 5,100,471 4,835,281 4,854,579 4,877,203 4,808,024 4,580,434 3,766,025

Identification Test-C

Patient Health PHQ-2 4,217,074 4,175,717 4,041,059 4,025,075 4,008,732 3,947,290 3,738,477 2,831,044

Questionnaire-2

PC PTSD Primary Care PTSD Screen 1,779,721 1,840,999 1,960,029 2,397,325 1,901,788 1,587,516 1,417,361 1,023,717

PCLC PTSD Checklist C 184,361 201,072 209,724 186,831 157,005 132,803 56,651 44,792

PCLM PTSD Checklist M 38,231 53,185 72,703 59,599 45,014 32,863 21,409 8,483

Clinical Institute Withdrawal CIWA-AR 90,088 52,155 24,190 7,604 1 1

Assessment - Alcohol Revised

Generalized Anxiety Disorder, GAD-7 71,429 36,321 23,407 5,983 9 1

7 items

BAM Brief Addiction Monitor 70,603 74,584 78,514 27,257 193 1

Brief Addiction Monitor - BAM-R 7,867 4,313 66

Revised

• The dates of data availability are measure dependent • Consistent data for long-standing measures (AUD-C, PHQ2,

PTSD Screen) starting in 2008 • Other measures available for fewer years/select cohorts

09/2016

Know the Data!

QUESTIONS TO CONSIDER

Why was the data collected?

Associated with performance

measures/metrics?

How was the data collected?

Was it systematic?

National clinical remindersor local initiatives?

Is the timing of the data important?

Did VA quality initiatives influence data collection?

Did changes to data

collection occur over time?

09/201617

Main Tables Mental Health Domain

18

SurveyAdministration joins to other tables in the MH Domain and other CDW Domains – Patient details, location, ordering provider

09/2016

Main Tables Continued Joining SurveyAdministration and SurveyResult tables

19

provides administration details and raw scores ▫ ͰΒ Οή ΒΪήΒή Ο ΠϪήΠϼ Ϫή ΪΒΒϚ ήϫϿϿ run

through an example)

▫ Survey Answer table joins to item level responses

Example of scoring challenges

AUDIT CONSUMPTION QUESTIONNAIRE (AUDIT-C)

2009/2016

AUDIT Consumption Questionnaire (AUDIT-C)

• Alcohol Use Disorders Identification Test (AUDIT)

– 10-item questionnaire developed by World Health

Organization

– Alcohol consumption, drinking behaviors and alcohol-

related problems

• AUDIT-C represents the alcohol consumption

questions of the AUDIT

– Valid screener for alcohol misuse

Bush et al., Arch. Inter. Med. 1998; Bradley et al., Arch. Inter. Med. 09/2016 2003; Bradley et al., Alc. Clin. Exp. Res. 2007

21

AUDIT-C 1. Frequency: How often did you have a drink containing alcohol

in the past year? (0-4 points)

2. Quantity: How many drinks did you have on a typical day when you were drinking in the past year? (0-4 points)

3. Heavy Drinking Episodes: How often did you have 6 or more drinks on one occasion in the past year?(0-4 points)

Scoring: Total AUDIT-C score 0-12

Bush et al., Arch. Inter. Med. 1998; Bradley et al., Arch. Inter. Med. 2003

09/2016 22

Prior to 2008

Adopted in 2004 for alcohol screening

Used to screen drinkers only (2004-2008) AUDIT-C data not reliably

available for non-drinkers

2008 to present

All patients asked question 1 of

AUDIT-C

If patient reports Ϫήήϫ ήϭ ϰ Responses to questions

2-3 automatically entered

Clinical reminder forAUDIT-C

standardized across VA

History of the AUDIT-C in VA

09/201623

AUDIT-C Data Collection

AUDIT-C systematically collected via

a national clinical

reminder

24

5

10

15

20

25

30

Relying on the Raw Score

Relying on the raw score in the

SurveyResult Table may result in

invalid total scores

• Appears to be more of a problem

with older measures

• May be caused by incorrect

questions/response options at

some sites

Range on

the AUDIT-C

is 0 to 12

SurveyName RawScore

AUDC 0

AUDC 1

AUDC 2

AUDC 3

AUDC 4

AUDC

AUDC 6

AUDC 7

AUDC 8

AUDC 9

AUDC

AUDC 11

AUDC 12

AUDC 13

AUDC 14

AUDC

AUDC 16

AUDC 17

AUDC 18

AUDC 19

AUDC

AUDC 22

AUDC 24

AUDC

AUDC 28

AUDC 29

AUDC

AUDC 32

AUDC 35

AUDC 36

AUDC 39

AUDC 57 1

Relying on the Raw Score

Invalid entries will have a score in the SurveyResult table • Need to pull item level responses to determine validity

Raw Question 1 Question 2 Question 3 Score Frequency Any Use Quantity Frequency 6+ Drinks

Not asked (due to responses Not asked (due to responses 0 Never on on

other questions) other questions)

Not asked (due to responses Not asked (due to responses 4 or more

4 on on times a week

other questions) other questions)

2 to 3 times 4 3 or 4 Skipped

per week

3 Skipped 3 or 4 Monthly

4 or more 7 7 to 9 Never

times a week

5 Never 3 or 4 Daily or almost daily

OK

Should

not be

scored

May want to exclude

09/2016 26

Scoring Recommendations

Provides link

between question

text and item

responses

Provides item

responses for non-

multiple choice

questions (date,

integer, etc.)

Provides text and type of each

question on the measure

Provides text for multiple choice response items

MH.SurveyAnswer

MH.SurveyAnswerText

Dim.SurveyQuestion

Dim.SurveyChoice

• Reliability of the raw score varies by

measure, date of administration, and site

– Older measures tend to have

more noise

• Recommend using responses to individual

items of measures to calculate total scores

– ϫϿϿ ή Ϫή SurveyAnswer table to join:

1. question text

2. multiple choice answer text

3. free answer text

(ήϫϿϿ ϪϠϪ Β ήΒϿή ΟήϿ

09/201627

Obtaining item level responses

BRIEF ADDICTION MONITOR (BAM)

2809/2016

29

History of the BAM in VA • Initially implemented in

2012

• VA specialty-care addiction programs encouraged but not required to administer at baseline and follow-up

• Designed to facilitate measurement-based care for substance use disorders

– Measurement-based care in Mental Health Initiative (2016)

09/2016

Measure Details • 17-item questionnaire developed by Philadelphia VA CESATE

• Assesses areas relevant to substance use and recovery

– Responses to individual items, rather than a total score, may be

of interest

• Four versions of BAM

– B!Ͱϧ ΠΒήϠϭΠΒϿ ήή ήϨϠϨϥ ϮήΠήϿϿήϯ Ϯϯ

– BAM-R: categorical and continuous responses (e.g., 1-30)

– BAM-IOP: 7-day assessment window

– BAM-C: consumption-related questions only

Cacciola, et al., JSAT, 2012 09/2016 30

BAM-R (Items 1-6)

31 09/2016

32

Measure Response Type • Dim.SurveyQuestion provides information about the response

type for each measure item ▫ The BAM-R has date, multiple choice and integer response types

33

Obtaining Item Level Responses • Join MH.SurveyAnswer and

Dim.SurveyChoice tables for multiple choice responses

• Join MH.SurveyAnswer and MH.SurveyAnswerText tables for date, integer and text responses

select distinct SA.SurveyName, SA.PatientSID, SA.SurveyGivenDateTime, SQ.SurveyQuestionText, SQ.SurveyResponseType, SC.SurveyChoiceText, ST.SurveyAnswerText from [CDWWork].[MH].[SurveyAnswer] SA join [CDWWork].[Dim].[SurveyQuestion] SQ on SA.SurveyQuestionSID=SQ.SurveyQuestionSID join [CDWWork].[Dim].[SurveyChoice] SC on SA.SurveyChoiceSID=SC.SurveyChoiceSID left join [CDWWork].[MH].[SurveyAnswerText] ST on SA.SurveyAnswerSID=ST.SurveyAnswerSID where SA.SurveyName like 'BAM-R'

Provides link

between question

text and item

responses

Provides item

responses for non-

multiple choice

questions (date,

integer, etc.)

Provides text for multiple choice response items

MH.SurveyAnswer

MH.SurveyAnswerText

Dim.SurveyChoice

09/2016

Survey Survey Choice Text Survey Answer Text

Survey Question Text Response Type (Dim.SurveyChoice) (MH.SurveyAnswerText)

Date of administration: DATE *Missing* 1/1/2016

In the past 30 days, how would you say your physical health has been? MCHOICE Excellent NULL

Ϫή Β 3ͼ ΪΒϥ Ϫ Β ϭϠϪ ΪϭΪ ϪΒή ΟϿή θΒϿϿϭϠ ΒϿήήϩϝ INTEGER *Missing* 0

In the past 30 days, how many days have you felt depressed/anxious/angryϩϝ INTEGER *Missing* 0

In the past 30 days, how many days did you drink ANY alcohol? INTEGER *Missing* 3

In the past 30 days, how many days did you have at least 5 drinks...? INTEGER *Missing* 2

Ϫή Β 3ͼ ΪΒϥ Ϫ Β ΪΒ ΪϭΪ ή Β ϭϿϿήϠΒϿ ήή ΪϠ ϩ INTEGER *Missing* 3

Marijuana (cannabis, pot, weed)? INTEGER *Missing* 3

Sedatives and/or Tranquilizers? INTEGER *Missing* 0

Cocaine and/or Crack? INTEGER *Missing* 0

Other Stimulants (amphetamine, methamphetamine...)? INTEGER *Missing* 0

Opiates (Heroin, Morphine, Dilaudid...)? INTEGER *Missing* 0

Inhalants (glues, adhesives, nail polish remover, paint thinner, etc.)? INTEGER *Missing* 0

Other drugs (steroids, non-ήΠϭϭ Ͽήή ΒΪ Ϊϭή ϭϿϿϩϝ INTEGER *Missing* 0

In the past 30 days, how much were you bothered by cravings or urges...? MCHOICE Not at all NULL

How confident are you that you will NOT use alcohol/drugs next 30 days? MCHOICE Extremely NULL

In the past 30 days, how many days did you attend self-help meetings/AA...? INTEGER *Missing* 0

In the past 30 days, how many days in a situation that increased risk...? INTEGER *Missing* 30

Does your religion or spirituality help support your recovery? MCHOICE Not at all NULL

In the past 30 days, how many days ϩ Β work, school, [volunteering]? INTEGER *Missing* 20

Do you have enough income (from legal sources) to pay for necessities...? MCHOICE Yes NULL

In the past 30 days, how ϩΟϪήήΪ Ο ΟϿή ϭϪ θΒϭϿϼθϭήΪϝ MCHOICE Not at all NULL

In the past 30 days, how many ΪΒ ϩ with supportive family/friends]? INTEGER *Missing* 20

How satisfied are you with your progress toward ϩήΠή goals? MCHOICE Considerably NULL 34

Output from Item Level Query

What is the prevalence of alcohol misuse among OEF/OIF Veterans with and without mild traumatic brain injury?

RESEARCH EXAMPLE WITH AUDIT-C

3509/2016

Brief Background

• High rates of alcohol misuse among OEF/OIF Veterans – Rates higher among men than women

– Rates higher among younger Veterans

• 10%-15% of OEF/OIF Veterans have sustained at least 1 TBI during deployment

• Many negative outcomes associated with alcohol use among those with TBI – May impair recovery from TBI

– Increased risk of recurrent TBI and development of alcohol use disorder

• Early detection and intervention is important

Grossbard et al., JSAT, 2013; Hawkins et al., DAD, 2010; Hoge et al., NEJM, 2008; Heltemes et al., Inj., 2012; Winqvist et al., Brain Inj., 2008; Polusny et al., Arch. Gen. Psych., 2011 09/2016 36

37

What is Alcohol Misuse?

Spectrum ranging from:

Drinking above recommended drinking limits

Meeting diagnostic criteria for alcohol use disorders

Saitz, NEJH, 2005 09/2016

AUDIT-C Alcohol Misuse Screening Groups

AUDIT-C Score Interpretation Action

0 Non-Drinker • No action required

1-2 Women Low level use • No action required

1-3 Men

3-4 Women At risk of alcohol • No action required 4 Men misuse • Brief intervention may be helpful

• Associated with alcohol misuse • VA requires brief intervention and/or

5-7 Moderate misuse referral to specialty treatment for Πή ≥5

• Associated with use disorders 8+ Severe misuse • Referral to specialty treatment

recommended

09/2016 38

Using AUDIT-C Data for Research Steps we performed

Step 1: Identify study population

Step 2: Look for outliers in the data

Step 3: Examine individual items

Step 4: Link AUDIT-C data to other VA data

09/201639

Step 1: Identify Study population

• All OEF/OIF Veterans enrolled in the VA with an AUDIT-C screen in 2012

09/201640

Step 2: Look for outliers in the data

• Sites with below average number of patients with AUDIT-Cs

• Consider use of health factors to identify non-drinkers?

– Especially relevant for measures that existed prior to MHA

– PHQ-2, PC-PTSD, AUDIT-C

09/201641

Step 3: Examine individual items closely and decide how to address

• Missing data – If AUDIT-C question 1 =

0, then questions 2 & 3 can be scored as 0

– Decide how to address missing questions (Discard?)

• Responses to items that do not make sense

Raw Score Q1

Response Q2

Response Q3

Response

0 Never Not asked (due to

responses on other questions)

Not asked (due to responses on other

questions)

4 4 or more

times a week

Not asked (due to responses on other

questions)

Not asked (due to responses on other

questions)

4 2 to 3 times

per week 3 or 4 Skipped

3 Skipped 3 or 4 Monthly

8 4 or more

times a week 10 or more Never

5 Never 3 or 4 Daily or almost

daily

09/2016 42

Considerations for Step 3 how decisions potentially impact research

questions

• Duplicate AUDIT-Cs

– How to address AUDIT-Cs documented on same day

– What if the scores are different?

• More than 1 AUDIT-C in time frame of interest

– Use the earliest, latest

09/201643

Step 4: Link AUDIT-C data to other VA

demographic and clinical measures – TBI diagnoses

– Gender, age, marital status, race, service

connection disability, ethnicity

– Substance use and mental health diagnoses

09/201644

Results Age-specific prevalence of heavy episodic drinking and alcohol misuse among OEF/OIF patients with and without TBI Women Men

(N =47,411) (N =310,736) TBI No TBI TBI No TBI (N = 1,836) (N = 45,575) (N = 28,361) (N = 282,375) % (95% CI) % (95% CI) % (95% CI) % (95% CI)

Heavy episodic

drinking

<30 10.1 (7.7-13.1) 8.0 (7.5-8.5) 28.5 (27.6-29.4) 23.1 (22.8-23.4)

30 -39 8.9 (6.5-12.4) 5.9 (5.5-6.4) 22.3 (21.3-23.4) 17.7 (17.4-18.0)

>40 6.9 (4.3-10.8) 5.0 (4.6-5.6) 17.8 (16.6-19.0) 11.5 (11.2-11.7)

AUDIT-C >5

<30 8.3 (6.5-10.5) 7.2 (6.8-7.6) 25.4 (24.6-26.1) 22.6 (22.3-22.8)

30 -39 6.0 (4.4-8.1) 5.1 (4.8-5.5) 18.2 (17.4-19.0) 16.3 (16.1-16.6)

>40 5.9 (4.0-8.4) 4.1 (3.8-4.4) 13.7 (12.9-14.6) 10.4 (10.2-10.6)

Grossbard et al., Psych. Serv., 2016

09/2016 45

Helpful Resources

• VIReC Factbook Corporate Data Warehouse (CDW) Mental Health 1.0 Domain November, 2014 – http://vaww.virec.research.va.gov/CDW/Factbook/FB-CDW-Mental-

Health-Domain.pdf (VA Intranet Only)

• VIReC Clinical Informatics Cyberseminar: Mental Health Assistant presented by Drs. Katie Lysell and Rick Owens, November 20, 2007 – http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/arc

hives/video_archive.cfm?SessionID=116

09/201646

47

Contact Information

Eric Hawkins, PhD Eric.Hawkins@va.gov

Carol Malte, MSW Carol.Malte@va.gov

09/2016

QUESTIONS?

4809/2016

References • Bradley, K.A., et al., 1998. Alcohol screening questionnaires in women: a critical review. JAMA 280, 166-171. • Bradley, K.A., et al., 2003. Two brief alcohol-screening tests From the Alcohol Use Disorders Identification Test (AUDIT): • validation in a female Veterans Affairs patient population. Arch Intern Med 163, 821-829. • Bradley, K.A., et al., 2007. AUDIT-C as a brief screen for alcohol misuse in primary care. Alc Clin Exp Res 31, 1208-1217. • Bush, K., et al., 1998. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem

drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch.Intern. Med. 158, 1789-1795.

• Cacciola, J.S., et al., 2012. Development and initial evaluation of the Brief Addiction Monitor (BAM). The Journal of Substance Abuse Treatment, 44(3): 256263.

• Grossbard JR, et al., Follow-up care for alcohol misuse among OEF/OIF veterans with and without alcohol use disorders and posttraumatic stress disorder. Journal of Substance Abuse Treatment, 45: 409-415.

• Grossbard J.R., et al., 2016. Prevalence of alcohol misuse and follow-up care in a national sample of OEF/OIF VA patients with and without TBI. Psychiatric Services. Aug 1. [Epub ahead of print]

• Hawkins, E.H., et al., 2010. Recognition and management of alcohol misuse in OEF/OIF and other veterans in the VA: a cross-sectional study. Drug and Alcohol Dependence, 109-147-153.

• Heltemes, K.J., et al., 2012. Blast-related mild traumatic injury is associated with a decline in self-rated health amongst US military personnel. Injury, 43:1990-1995.

• Hoge, C.W., et al., 2 008. Mild traumatic brain injury in US soldiers returning from Iraq. New England Journal of Medicine, 358: 453-463.

• Polusny, M.A., et al., 2011. Longitudinal effects of mild traumatic brain injury and posttramatic stress disorder comorbidity on postdelplyment outcomes in National Guard soldiers deployed to Iraq. Archives of General Psychiatry, 68: 79-89.

• Saitz, R. 2005. Clinical practice. Unhealthy alcohol use. New England Journal of Medicine, 352: 596-607. • Winqvist, S. et al., 2008. Recurrent traumatic brain injury is predicted by the index injury occurring under the influence of

alcohol. Brain Injury, 22: 780-785.

49