1

Using Qualitative Methods for Improving Cardiovascular Care

Elizabeth H. Bradley, PhD

Professor of Public Health

Yale School of Medicine

American Heart Association

Pre-Conference Workshop

May 19, 2010

Disclosure

Research used in this presentation were funded in part by the Agency for Healthcare Research and Quality, the National Heart, Lung, and Blood Institute, the Commonwealth Fund, and the Donaghue Medical Research Foundation

No conflicts of interest to disclose

2

3

Objectives of Workshop

Develop competencies to:

Define qualitative methods and know when to use them

Critically evaluate the methodology of qualitative studies (study design and sampling, data collection, analysis)

Workshop Outline

1:00 -2:00 PM What are qualitative methods and when should we use them?

(Examples of Results from Qualitative Studies)

2:00-2:15 PM Break

 

2:15-3:15 PM Sampling and Data Collection with Examples from Studies

3:15-3:30 PM Break

3:30-4:30 PM Data Analysis with Examples from Studies

 

4:30-5:00 PM Standards of Rigor and Addressing Limitations of Qualitative Studies

4

5

Research cycle we know from quantitative reseach

Pick Topic

Define study design + sample

Focus research objectives

Collect Data

Analyze Data

Report Results

Act

6

What are qualitative methods?Qualitative methods are a set of research

approaches used with the following objectives:

To describe a phenomenon

To generate hypotheses

To develop grounded theory

7

Quantitative versus qualitative objectives

Estimate prevalence/incidence of phenomenon, y (rather than describe phenomenon)

Test (rather than generate) hypothesis concerning predictors, correlates, or consequences of y

(for instance: x y or y y1)

Test (rather than develop) a theory

8

Products of qualitative research

Objective Output

Describe phenomenon, yKey domains of y

Taxonomy of y

Generate hypotheses Recurrent themes

Testable hypotheses

Develop theory Conceptual model/theory (boxes w/ arrows)

9

Describing a phenomenon

Not everything that can be counted counts, and not everything that counts can be counted.

- Einstein

10

When is a phenomenon best described with qualitative methods?When the phenomenon that is multifaceted and

complex to understand and interpret

When the interest is not just the objective event but also how the event is experienced

When social interaction and context are important

Examples?

11

Why is describing phenomenon useful?

Distilling a complex phenomenon to its key dimensions or component parts can help develop taxonomy

Taxonomy help us compare apples to apples in evaluation (QI interventions, HMOs, etc.)

Lays groundwork for valid measurement (improves instrument design and fielding)

12

Example: Taxonomy for Quality Improvement Efforts

Domain Dimensions Exemplar concepts or range

Goals Content High quality; low cost, market shareSpecificity Very specific to nonspecific Challenge Very challenging to “easy” goalsSharedness Widely shared to poorly shared

________________________________________________________________

AdministrativeSupport Philosophy Innovation first; safety net provider; etc.

Resources Human, capital, technical_________________________________________________________________

PI initiatives Type Clinical pathway; standing orders; Style of impl. Top down; participatory; blameless

_________________________________________________________________

Bradley et al., JAMA, 2002

13

Generating hypothesesIdentify causal links that seem to be at work

What causes what? What is the consequence? Under what conditions?

Can be from what one observes directly in sequencing or from how participants talk about their experiences

Remember that these are hypotheses (only)

14

Example: Recurrent themes/hypotheses in D2B improvement

1. Explicit organizational goal-setting

2. Visible senior management support

3. Innovative, standardized protocols

4. Flexibility in implementation

5. Uncompromising clinical leaders

6. Collaborative, interdisciplinary teams

7. Specific data feedback

8. Non-blaming culture

PARADOX Bradley et al., Circ, 2006

15

Developing grounded theory

Theory: a set of premises or hypotheses about how the world works; a conceptual model

Grounded versus axiomatic theory

16

How is grounded theory useful?

Can be interesting in its own right

And sets up hypotheses for testingIdentifies x and y variables; direction of effectsIdentifies mediating effectsGuides statistical model Helps avoid “fishing” exercise

Helps in making sense of observed results

17

Example: Conceptual model for long-term care use

Enabling Factors

Predisposing

Need

Attitudes Toward

Services

Social Norms About Caregiving

Perceived Control

Intended Use of Long-term

Care

Actual Use of

Long-term Care

Bradley et al., HSR 2002

18

When to use qualitative methods?When your research objective matches what

qualitative methods can accomplish- Describe a phenomenon- Generate (not test) hypotheses- Develop grounded theory

When the phenomenon is complex and difficult to measure with existing quantitative approaches

When literature and hypotheses are lacking

19

When not to use qualitative methods

When you really want to know and publish how often something occurs

Because you think qualitative methods will be easier, cheaper, or faster

Because “there is no literature in the area” still has to be a topic that requires qualitative inquiry

Can mixed methods help?

Mixed methods:

The integration of qualitative and quantitative methods to improve understanding

Can be simultaneous or sequential

(qualitative quantitative; quantitative qualitative

20

21

Positive deviance approach is a mixed methods approach (qualitative leads to quantitative)

A “positive deviance” approach

1. Identify top performing hospitals

2. Study them qualitatively

3. Generate hypotheses about top performance

4. Test hypotheses quantitatively; random sample

5. Disseminate evidence in national campaign

Bradley et al., Impl Sci 2009

Summary

Qualitative methods provide an approach to understanding what may not have been previously examined and what defies quantitative measurement

Qualitative methods must fit with the research objectives to be valid and useful

22

Break

23

Sampling and data collection

24

25

Common qualitative study designs

In-depth interviewing

Focus groups

Participant or non-participant observation

Case study; ethnography

Hybrids

26

Sampling techniques in quantitative research

What are key concepts that govern sampling?- Representative-ness of population (valid)- Big enough (precise)

What are different sampling strategies?

How does one determine sample size?

27

Sampling techniques in qualitative research

What are key concepts that govern sampling?

- Participants have the experience under inquiry

- Diversity; find all dimensions that might matter

- If you heard it once, you have heard it

28

Sampling strategy in qualitative research

Purposeful sampling (sometimes called purposive) Theoretical sampling as the highest standard

May have a random component, which is augmented with purposeful selections

Inclusion criteria (defining “key informants”)- Must have the experience in question- Must be willing to talk about it

Looking for broad representation, but not in the proportions that are in the population

29

Sample size in qualitative research

Concept of “theoretical saturation”

When no new concepts emerge from successive interviews

Judgment call to some extent

Typically small samples

30

Sampling for focus groups

Typically 6-10 participants per group

Usually 2-5 groups per “strata”

Must share some common experience or trait

Homogeneity v heterogeneity issues

Avoid power differential within group if you can

31

Principles in data collection for quantitative research

Closed-ended measures

No room for interpretation by investigator

Consistency and reproducibility is goal

Premise that abstract can capture truth (example of “age” or “race”)

32

Principles of data collection for qualitative research

Open-ended questions

Investigator interpretation is expected (disclosure)

Depth, validity is goal TRUST IS #1 CONCERN

Premise that truth is in rich detail, not abstraction

Always need consent (usually oral is fine, per IRB)

33

Spend time establishing trust and safety

Body language, facial expressions

Describe goals, consent process, data integrity

Do not judge anything, no matter how small, ensure that there are no right or wrong answers

Important distinction between professional researcher and colleague or friend

34

Techniques in depth interviewing

Discussion guide versus survey instrumentFew questions (5), all open-ended, with probes

Open-ended interviewing Be authentically curious

Practice passive listening

Be alert for jargon, unclear links, new concepts

Delve into things that do not make sense

Keep your own views to yourself, interrupt judiciously

Practice the 5-second pause; let silence happen

35

Typical questions

Grand tour question: Tell me about your experience with… (whatever your inquiry is)

Can you tell me more about that? (Think back)

What was that like for you? What happened next?

You used the term “physician champion,” can you tell me more about that? What did you mean when you used that term?

36

More approaches to interviewing

Wait the participant out…eyebrow raises, etc.

Seek examples, but be careful how you ask this. It can cause defensiveness; you want vignettes, stories, etc.

You do not want their packaging (abstracting) of concept but rather the rich detail so you can interpret

Summary question: is their anything I should have asked you that would help me understand xyz?

37

Special concerns in focus groups: establishing ground rules

1. Expect differences of opinions

2. Interested in positive and negative comments3. Want to hear from everyone (“if you are talking a lot, I

may asked you to give others a chance, and if you are not talking, I may call on you”)

4. Speak one at a time

5. Respect confidentiality of members

6. Use first names only (no identifying information)

7. We will stick to time limit (1 hour – 90 minutes)

38

Recording data

Multiple approaches

For in-depth interviews or focus groups, audio-taping is often successful (helps reliability) and forgotten by participants soon after you begin

Informal note-taker other than the lead interviewer

In field work (ethnography, observation studies), one often just has field notes and maybe archival data

39

In-depth interviewing and focus group moderator skills

Strong interviewing skills

Keen observational skills

Ability to control and guide discussion

Ability to suppress own personal views

Has and projects authentic respect for participants

40

How to develop skills

Watch someone experienced

Do some yourself and have your work critiqued

Practice (toss early attempts); you learn as you go and analyze your own work

Recognize that it is not for everyone!

Summary

Sampling and data collection rules of thumb in qualitative methods are almost the opposite from these rules in quantitative methods

Authentic, open curiosity is key element – of any good scientist, especially one in qualitative research

Self-awareness and practice improves skills

41

Break

42

Data analysis

43

44

Open-ended data

Quantitative analysis of open-ended data- Counting frequency of different statements, ideas, etc.

- Report: “20% said x is a problem”

- “Content analysis”

Qualitative analysis of open-ended data - Develop concepts and themes and models

- Report themes and illustrative quotations

- Populate the “x-axis”

45

Steps in implementing the qualitative analysis

1. Prepare the data

2. Read the data for general understanding

3. Code the data

4. Integrate the data

5. Develop taxonomy, themes, and theory

Bradley et al., HSR 2008

46

Codes for organizing qualitative data

Codes are tags or labels for assigning meaning to descriptive information

Coding is the process of organizing the data into “chunks” that are alike, moving from words and sentences to “incidents” that depict a particular concept

47

Approaches to developing codes

1. Provisional “start list” of codes

2. Purely inductive, or grounded, codes

3. Between “start list” and inductive approaches to coding

48

Major types of codesConcepts of importance

- Could be key variables emerging- Could be reasons why or how something works

Characteristics of participant or setting- Often emerge as correlates or medicating factors

Potentially non-causal links among concepts- Explicit or implicit evidence of inferences about how

different coded concepts may interrelate

49

Beginning coding

Read the transcripts or notes for overall understanding (best to occur with team)

Note in words the key concepts (each separately), write memos for the file as needed, mark up margins of transcripts or use software to note

Come together in group to review transcripts line-by-line, using a constant comparative method

50

Negotiating codes

The group will not agree and that is the beginning of the analysis process…negotiate, talk out the concepts, fleshing out their properties

Develop code list from those meetings

Read a few more, same process

Refine code list, recoding as needed

51

Interplay of data analysis and data collection

These happen simultaneously in qualitative research; data analysis informs data collection

Can change the discussion guide

Can sample in new ways theoretical sampling!

52

Saturation

As coding continues, code sheet is becoming solidified with the properties of each code clarified for whole team

Final code sheet (once saturated)

Now, APPLY the final coding structure

Examples from studies

53

Standards of rigor and addressing limitations in qualitative studies

54

55

Evaluating methods of quantitative research

Reliability: degree to which findings can be reproduced if conducted on same population with same protocol, even if on average wrong

Validity: degree to which findings are on average “true,” even if imprecise

Generalizability: degree to which findings in sample reflect the truth for the population about which you want to make some conclusion

56

Improving reliability in qualitative research

1. Tape record (unless intimidating or non-consented)2. Use multiple interviewers (unless intimidating)3. Send transcript to interviewee for checking (debated) 4. Independent professional transcription5. Check accuracy of transcription against the tape 6. Apply code structure in systematic way7. Check inter-rater reliability of coding (debated)8. Retain audit trail to document code development and

analytic decisions

57

Improving validity in qualitative research

1. Multiple coders with different backgrounds

2. Multiple analysts with different backgrounds using negotiated consensus methods

3. Multiple sources: interviews, observations, written documents, etc., if possible

4. Feed back results to participants (debated)

58

Improving generalizability (aka “transferability”) of qualitative research

Use theoretical sampling to ensure participants reflect diversity of opinions and experiences

Seek disconfirming evidence

Be sure to saturate before ending data collection

59

The most common criticism: small sample size so not generalizable

Generalizability is a moot issue with qualitative research because one’s objective is to generate hypotheses, NOT make statistical inferences about a population

However, findings from qualitative studies should be applicable (“transferable”) to many as long as the sample is sufficiently diverse and theoretical saturation is achieved

60

Concluding remarks

The research method, qualitative or quantitative, must match the research objective

Qualitative work is not fast, easy, or cheap

Qualitative studies do have subjective components, BUT…..

61

Concluding remarks

So do quantitative studies!

We can get lulled into the false sense of security that quantitative research is more objective because there are numbers and statistics

Good qualitative research employs rigorous and specific techniques that can enhance reliability, validity, and transferability of findings

62

Thank you

ReferencesBradley EH, Holmboe E, Mattera J, Roumanis S, Radford MJ, Krumholz HK. A qualitative study of increasing beta-blocker use after

myocardial infarction: why do some hospitals succeed? Journal of American Medical Association 2001; 285:2604-2611.

 

Bradley EH, Curry LA, Webster TR, Mattera JA, Roumanis SA, Radford MJ, McNamara RL, Barton BA, Berg DN, Krumholz HM. Achieving rapid door-to-balloon times: How top hospitals improve complex clinical systems. Circulation 2006; 113:1079-1085.

Bradley EH, McGraw SA, Curry LA, Buckser DA, King KK, Andersen R. Expanding the Andersen model: the role of psychosocial factors in long-term care use. Health Services Research 2002; 37:1221-1242.

Bradley EH, Curry LA, Devers K. Qualitative data analysis for health services research: Developing taxonomy, themes and theory. Health Services Research 2007; 42:1758-1772.

Curry LA, Nembhard IM, Bradley EH. Qualitative and mixed methods provide unique contributions to outcomes research. Circulation 2009;119:1442-1452.

Patton, M (1999). Enhancing the quality and credibility of qualitative analysis. Health Services Research 34(5): 1189-1208.

Patton, M. Q. (2002). Qualitative Research and Evaluation Methods, 3d ed. Thousand Oaks, CA: Sage Publications.

Pope, C., S. Ziebland and N. Mays. (2000).Qualitative Research in Health Care. Analysing Qualitative Data. British Medical Journal 320 (7227): 114-6

Popay J, Rogers A, and Williams G. (1998). Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research 8(3):341-351.

63