taking account of what can’t be counted: the place of qualitative ... › ... ›...
TRANSCRIPT
On evidence in HTA: Bringing patient, caregiver and community experience in Taking account of what can’t be counted: The place of qualitative evidence in HTA
Fiona A. Miller, PhD @FionaAliceMill
Associate Professor, IHPME, THETA
CADTH Symposium
April 11, 2016, Ottawa
2
What is evidence?
• Origins (Upshur, 2001)
– Middle English: via Old French from Latin evidentia, from evident- 'obvious to the eye or mind' (Oxford)
• Definitions (Upshur, 2001)
– something that makes plain or clear; an indication or sign (Dictionary.com)
– something that furnishes proof (Merriam-Webster)
• Facts alone are not evidence (Madjar & Walton, 2001)
– “facts can only become evidence in response to some particular question” (Chandler et al, 1994)
3
What questions might HTA need to answer?
Describing/ Characterizing
• Patients
– The meaning and experience of illness
• Technologies
– The meaning and experience of current and proposed technologies
• Services/ Systems
– The arrangement and operation of current and proposed services
Interrogating/ Explaining • Patients
– How social, cultural and economic contexts condition illness
• Technologies – How social, cultural and
economic contexts condition the effects of technology
• Services/ Systems – How services and systems
condition experiences and outcomes
– What makes some interventions work better than others
4
Answering “other” questions: The need for “other” evidence
• From and about persons
– ‘knowledge in the possession of people’ (JBI, 2014)
– Physically, socially, culturally embodied and embedded
• Only partially captured in the clinical and economic evidence traditionally used in HTA
5
Sources of “other” evidence
Research-based
• Often “qualitative”
– Interviews, focus groups
– Ethnographic observation
– Questionnaires, documents
• Descriptive
• Interpretive
• Theory
Opinion-based
• Patient experts - Expert opinion
– Input
– Participation
– Deliberative engagement
• Descriptive
• Interpretive
• Judgment
6
Often seeks to “give voice”
May use research-based methods
ON RELEVANCE AND RIGOUR
The quality of “other” evidence
7
Relevance
• On relevance – Being ‘pertinent to…’, ‘bearing upon…’, ‘connected with…’,
or ‘appropriate to…’, ‘…the matter at hand’, as well as ‘germane’, ‘apropos’, ‘material’, and ‘applicable’ (Dobrow et al, 2015)
• Opinion has a priori relevance – Local, specific
• (And other functions beyond the probative)
• Research has complex relevance – Aggregative function – persistence of themes
– Configurative function – unanticipated connections between themes (Sandelowski et al, 2012)
– Comparative function – the contingency of current practice 8
Rigour
• On rigour
– The quality of being extremely thorough, exhaustive, or accurate … meticulous, thorough, careful, diligent, scrupulous, exact, precise, correct
• A function of act and actor
– Appropriate
– Expert
– Trustworthy
9
Critical appraisal tools
• QARI, JBI
– Critical appraisal of qualitative evidence
• NOTARI, JBI
– Critical appraisal of text and opinion based evidence
10
QARI Critical Appraisal Criteria (JBI, 2014)
• 1. Congruity between the stated philosophical perspective and the research methodology
• 2. Congruity between the research methodology and the research question or objectives
• 3. Congruity between the research methodology and the methods used to collect data
• 4. Congruity between the research methodology and the representation and analysis of data
• 5. There is congruence between the research methodology and the interpretation of results
• 6. Locating the researcher culturally or theoretically • 7. Influence of the researcher on the research, and vice-versa, is
addressed • 8. Representation of participants and their voices • 9. Ethical approval by an appropriate body • 10. Relationship of conclusions to analysis, or interpretation of the
data 11
NOTARI Critical Appraisal Criteria (JBI, 2014)
• 1. Is the source of opinion clearly identified? • 2. Does the source of opinion have standing in the field
of expertise? • 3. Are the interests of patients/clients the central focus
of the opinion? • 4. Is the opinion’s basis in logic/experience clearly
argued? • 5. Is the argument that has been developed analytical?
Is the opinion the result of an analytical process drawing on experience or the literature?
• 6. Is there reference to the extant literature/evidence and any incongruence with it logically defended?
• 7. Is the opinion supported by peers?
12
Rigour
• On rigour
– The quality of being extremely thorough, exhaustive, or accurate … meticulous, thorough, careful, diligent, scrupulous, exact, precise, correct
• A function of evidence systems (Pang et al, 2003)
– Stewardship
– Financing: securing and allocating funds
– Creating and sustaining resources
– Producing, utilizing and synthesizing evidence
13
Health “evidence” systems (Pang et al, 2003)
14
In conclusion …
• There are questions that cannot be answered without “other” evidence – Concerning the embodied and embedded ways in which
persons understand and experience illness and its management, and how services and systems do and could condition these realities
• Such evidence can be generated through research (e.g., qualitative research) and/or expert opinion – Which can be more or less relevant – And more or less robust
• The quality of such evidence is a function – Of acts and actors – And of evidence systems
• If we need this evidence, we need systems to support its quality and use 15
References
• Upshur, RE. (2001). The status of qualitative research as evidence. Morse, Janice M., Janice Swanson, and Anton J. Kuzel. (Eds.) The nature of qualitative evidence. Sage, 2001: 5-26
• Madjar, I & Walton, JA. (2001). What is problematic about evidence? Morse, Janice M., Janice Swanson, and Anton J. Kuzel. (Eds.) The nature of qualitative evidence. Sage, 2001: 5-26
• Chandler, JK., Davidson, AI., Harootunian, HD. (1994) Editor’s Introduction, Questions of evidence: Proof, practice, and persuasion across the disciplines. Chicago: University of Chicago Press, 1-8
• The Joanna Briggs Institute. (2014) Joanna Briggs Institute Reviewers’ Manual: 2014 edition. The Joanna Briggs Institute
16
References
• Sandelowski, M., Voils, C. I., Leeman, J., & Crandell, J. L. (2012). Mapping the mixed methods–mixed research synthesis terrain. Journal of mixed methods research, 6(4), 317-331.
• Pang, T., Sadana, R., Hanney, S., Bhutta, Z. A., Hyder, A. A., & Simon, J. (2003). Knowledge for better health: a conceptual framework and foundation for health research systems. Bulletin of the World Health Organization, 81(11), 815-820.
• Dobrow, MJ., Miller, FA., Frank, C., Brown, AD., (2015) Understanding Relevance of Health Research: Considerations in the Context of Research Impact Assessment, Commissioned by the Ontario SPOR Support Unit, IHPME White Paper Series
17
Questions?
18