AN OVERVIEW OF QUALITATIVE RESEARCH
September 5, 2014Elyse R. Park, Ph.D., M.P.H
Associate Professor in PsychiatryMassachusetts General Hospital/Harvard
Medical School
QUALITATIVE PAPERS IN PubMed
Year PubMed Qualitative research Ratio Pubs Pubs
(per 1,000 articles)
1990 399,836 38 0.101994 423,431 86 0.201998 461,188 150 0.332000 521,038 217 0.422004 622,978 922 1.48 2008 817,189 1920 2.352010 930,796 2676 2.87
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Features of Qualitative Research
Researcher’s role Main task is to determine how people understand & behave
Little standardized
instrumentation Most analysis is done with
words
Types of Qualitative Research Questions
• Descriptive– What happened?
• Interpretative– What’s the meaning?
• Theoretical– Why did things happen?
Anticipating Clinical Integration of Genetically-Tailored Tobacco Treatment:
Perspectives of PCPsOBJECTIVES
• Explore physicians’ attitudes toward treatment strategies that matched patients according to genotype
• Understand patient-based & physician-based barriers
• Identify concerns about external factors that would need to be addressed prior to clinical integration of a genetic test to tailor smoking cessation treatment.
Park et al., Nicotine & Tobacco Research, 2007
Cultural Attitudes About Breast Health
Study Purpose
• To explore minority women’s cultural attitudes toward screening that might influence their return for screening– Communication– Barriers– Health Beliefs– Improving mammography rates in
communityMoy, Park et al, Psychooncology, 2006
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
What Type of Question Is It?
What (exploratory) Qualitative
What (how many, how much)
Quantitative
Who & Where Quantitative
Why & How (explanatory) Qualitative
Question Strategy
Qualitative vs. Quantitative
When generalizability of results & comparison across populations is needed
For studying selected issues, cases or events in detail
When repeatability of measurements is important
When flexibility of approach is needed
When detailed numerical description of a representative sample is required
When meaning rather than frequencies are sought
When measurements are knownWhen relevant concepts are unknown or unclear
Subject matter clearly definedSubject matter is unfamiliar
QuantitativeQualitative
Mixed Methods
• Collecting quantitative data
• Collecting qualitative data
• Using both sets of data to draw conclusions
Mixed Methods
1. Interactive or independent– Interactive: mixing of study questions, data
collection and/or analysis– Independent: mixing during interpretation
2. Priority of each study
3. Timing of each study– Concurrent– Sequential– Multiphase/combined
– Creswell & Clark, 2011
Exploring The Role of Mood on Postpartum Relapse to Smoking
• Purpose: To examine how anxiety & depression impact postpartum relapse to smoking
• Design: Pilot study (n=65)– Smokers who quit during pregnancy– Repeated surveys during 6 months
postpartum– In-depth interviews for participants 1)
who relapse or 2) report symptoms of anxiety/depression
(Park et al., Nicotine & Tobacco Research, 2009; Park et al, Addictive Behaviors, 20009)
Racial Disparities in Coping with Lung Cancer
• Use CanCORS dataset to 1) compare rates of depression in non-Hispanic Black (NHB) and White (NHW) lung cancer patients and 2) identify rates of mental healthcare utilization) Trager & Park, JCO 2014
• Conduct in-depth interviews with patients recruited from the Dana-Farber Harvard Cancer Center to characterize Phase 1 findings (n=20)
Developing A Qualitative Research Question
Dangers
– Too Diffuse– Too Focused– Contains unexamined assumptions– Reflects research questions, not interview
questions
Research Questions & Hypotheses
Research Questions
• Quantitative: HYPOTHESES
• Qualitative: PROPOSITIONS
Purposes of Qualitative Research
• Stand alone
• To inform survey development
• To elucidate survey findings
• To inform program development (curriculum, interventions)
Linking Qualitative Data With Surveys
Qualitative data…
Contribute to the creation of survey items by:
1) Capturing survey domains
2) Determining dimensions
3) Item wordings (cultural idioms)
• Augment testing of an instrument• Help generate hypotheses• Explain aspects of findings that are puzzling
Black and White adults' perspectives on the genetics of nicotine
addiction susceptibility
• To explore attitudes and beliefs of black and white participants regarding the role of genetics in addiction to nicotine
• To explore attitudes of black and white participants about use of genetic testing to be matched to optimal smoking cessation treatment
Park et al., Addictive Behaviors, 2011
Using Qualitative Data to Inform Survey Development
SURVEY GOAL: To develop a national survey to assess residents’ attitudes about cross-cultural care:
1) Perceptions of preparedness to deliver care to diverse populations
2) Educational experiences3) Educational climate
Weissman et al., JAMA, 2005
QUALITATIVE STUDY GOAL: To elucidate the 3 survey domains
Park et al., Academic Medicine, 2005
Mixed Methods Design
1) Convergent: separate quantitative and qualitative data collection, separate analyses & comparison of the two
2) Explanatory: quantitative data collected first; qualitative data collection is subsequent
3) Exploratory: qualitative data collection explores a topic and analysis results are used to build a quantitative data collection procedure
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Qualitative Data Collection
· Individual in depth interviews· Narratives· Focus group interviews
Consensus methods· Participant observation· Case studies· Document analysis
Patient and Physician’s Attitudes Regarding the Disclosure of Medical Errors.
Design: 13 focus groups. Physician-only, patient-only, & physician/patient groups.
Patient questions: defining patient safety & medical errors
Physician questions: what they would disclose to patients
Themes of results:• What errors to disclose• Disclose near misses?• Role of apology
Gallagher T et al., JAMA, 2003
Focus Groups versus Individual Interviews
FOCUS GROUPS • Observe interaction/captures interpersonal
dynamics• Allows for spontaneity• Secure environment if participants feel
empowered to speak among peers
INDIVIDUAL INTERVIEWS• Interviewer control• More information per informant• Useful for evoking personal experiences/
perspectives
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Qualitative Sampling
Purposeful/purposive sampling
Particular settings, persons, or events are deliberately selected according to 1) research purpose and 2) theoretical development
Stratification: deliberate selection of participants
based on certain criteria
(Patton, 1990; Miles & Huberman, 1994)
Purposive Sampling
Particular settings, persons, or events are deliberately selected based on a characteristic.
ΔHomogeneous Δ Stratified Purposeful Δ SnowballΔ CriterionΔ Random Purposeful
Patton MQ, 1990
Saturation
• Saturation occurs when no new information
(redundancy) is obtained (Lincoln & Guba, 1985; Patton, 1990)
• Through the constant comparison of data (Glaser & Strauss, 1967, Cutcliffe & McKenna, 2002) - researcher states s/he has saturation based on comparison of data analyzed to additional data
• Recurrent patterns and themes are seen
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Determining the Structure of Interviews
• 3 Levels: Un—Semi—Highly
• Structure determines the:–Content–The number of questions–Sequencing
Developing The Interview Guide
Unstructured
• 1-2 broad topics
Structured
• 4-5 distinct topics or questions
• distinct probes for each topic
• specific order
• order from general to specific
• begin with most important questions
• builds in transitions among topics
Considerations when using Qualitative Methods in Cross Cultural Research
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Pilot Testing The Guide
• Are the questions phrased in a way that will elicit the information that you’re seeking?
• Are the questions clear?
• Are the questions biased?
• Are any of them closed ended?
• Do any need expansion?
• DID YOU use ANY double negatives?
Considerations when using Qualitative Methods
• Developing a research question• Selecting data collection modality• Recruitment & sampling• Developing the interview guide• Piloting the interview guide• Qualitative analysis• Determining quality of research
Data analysis
Inductive: categories and themes emerge (grounded theory)
Deductive: processes of “fitting” the data into categories and themes (framework analysis)
Steps to Analysis
1) Familiarization2) Transcribing3) Organizing4) Coding
Reducing data Displaying data• Helps organize data• See areas where analysis is complete• See patterns & themes• See how data “fits” theory
5) Drawing & Verifying Conclusions
Components of Data Analysis:Interactive Model
Data Collection
DataReduction
DataDisplay
Conclusions:Drawing/Verifying
Miles and Huberman, 1994
Characteristics of Qualitative Data Analysis
• Systematic and verifiable
• Theory driven
• Iterative process
• Flexible to alternate views
• Includes a system to handle incongruencies
between coders
Audit Trail
Auditability: clear account of the research process so that that reader can judge the dependability of the study (aka decision trail)
Lincoln & Guba, 1985
Keeping Track Of The Analysis Process
• Theories that influenced the data analysis
• How data were organized
• Multiple coder strategy
• How you conducted data reduction
• How conclusions were drawn & verified
What Are You Coding?
• The Word
• Internal Consistency
• Frequency
• Extensiveness
• Intensity
• What Was Not Said
Initial Coding Framework
COVERAGE CHARACTERISTICS AND SATISFACTION• Access satisfaction
– Satisfied– Not satisfied
• Problems with access– Physician not in network– Plan accepted by a limited number of physicians– Distance to providers
DIFFICULTIES AND GAPS IN COVERAGE• Ever had a period that was not covered?
– Yes• Reason for gap
– Affordability– Change in job/job status– Change in dependent status– Other
POSSIBLE ATTRIBUTES• Insured (Y/N)• Gender• Cancer Recurrence (Y/N)
Advantages of Computing
• Reducing time needed for manual handling tasks
• Increased flexibility in handling large amounts of data
• More rigorous analysis of data
• More visible data analysis audit trailSt John & Johnson, 2000
Disadvantages of Computing
• Focus on quantity vs. meaning
• Data can become too reduced/too much focus on coding & retrieval
• Time spent learning use
• Pressures to use/commercialism
St John & Johnson, 2000
Experimentalistics try to control it.
Statisticians count it.
Interviewers ask questions about it.
Observers watch it.
Participant observers do it.
Qualitative inquirers find meaning in it.
-From Halcolm’s Laws of Inquiry
Interactive Exercise
1) Write down an idea for a study that you may want to conduct qualitatively.
2) Explain your idea/question and discuss what you bring to the study: your background, beliefs, clinical training, & patient population.
3) Discuss the answers to the following questions:a. I want to ask this question because…b. One thing I’m sure about what’s going on is…c. I would be really surprised if I learned…
Disclosure: Quantitative
Physicians’ intentions to disclose findings of a somatic alteration were influenced by both the availability of an FDA-approved medication and by the prognosis conferred. The majority of participants (n=xx; xx%) responded that they would “definitely disclose” this finding if there were an agent that was currently being studied in a Phase II trial that was accessible to patients. More than half of physicians (n=xx, xx%) would “definitely disclose” if the alteration conferred a favorable prognosis or if the alteration was in a pathway for which a FDA-approved agent was available for a different cancer. However, only xx% (n=xx) would “definitely disclose” an alteration found that conferred an unfavorable prognosis
Disclosure: Qualitative
The decision to disclose was predicated upon the belief that is the patient’s right to know, as well as a strong sense of expressed identity and obligation in enrolling patients in clinical trials (“This is what we do.”). Physicians expressed intentions to disclose to enable exploration of different treatment options, with the clarification that disclosing doesn’t necessarily mean treating. Physicians’ willingness to disclose would also be influenced by a patient’s perceived treatment access (e.g. insurance coverage), potential trial candidate status (e.g., eligibility, performance status, preclinical data), toxicity of the medications, and a patient’s ability to benefit from the treatment. However, if a patient is not deemed a good trial candidate, there was still a sentiment expressed that the patient had a right to know about the findings; there was expressed tension about what to do when a patient was not a viable candidate yet had right to know.
Ways To Enhance Validity
• Respondent validation
• Triangulation: Using multiple investigators, theories,
data, or methods to gather data, within a single study, to
provide confirmation and completeness of data
• Attending to cases that deviate from theory
• Supervision and peer review of analysis
Consistency & Dependability
Assuring that information collected is accurate &
consistent.• Pretesting & standardizing instruments• Team interviewing• Tape recording• Debriefing• Participant feedback• Multiple coders/establishing inter-rater
reliability• Coding with content analysis• Keeping an audit trail