![Page 1: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/1.jpg)
Appraisal, Extraction and Pooling of Qualitative Data and Text
- Evidence from qualitative studies, narrative and text.
JBI/CSRTP/2013-14/0003
![Page 2: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/2.jpg)
Introduction• Recap of Introductory
module– Developing a question (PICo)– Inclusion Criteria– Search Strategy– Selecting Studies for Retrieval
• This module considers how to appraise, extract and synthesize evidence from qualitative studies and text.
![Page 3: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/3.jpg)
Program Overview
![Page 4: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/4.jpg)
Program Overview
![Page 5: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/5.jpg)
Session 1: Introduction to Qualitative Evidence Synthesis
![Page 6: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/6.jpg)
Qualitative Research is…• “…a situated activity that locates the observer in the world. It
considers a set of interpretive, material practices that make the world visible…[it] involves an interpretive, naturalistic approach to the world. This means qualitative researchers study things in their natural settings, attempting to make sense of, or to interpret, phenomena in terms of the meanings people bring to them” (Denzin & Lincoln, 2000)
![Page 7: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/7.jpg)
Qualitative research and health care• Health care is a people centered process concerned with
health and healing– Health care derives much of it’s knowledge from a range of
biomedical sciences
• However this does not supply all the knowledge that is necessary to provide holistic care for the patients and clients
• Qualitative researchers use humanistic frameworks to examine ordinary activities of everyday life
![Page 8: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/8.jpg)
Qualitative Research Findings as Evidence for Practice
• Qualitative evidence is of increasing importance in health services policy, planning and delivery.
• It can play a significant role in: – understanding how individuals / communities perceive health,
manage their own health and make decisions related to health service usage;
– increasing our understandings of the culture of communities and of health units;
– Informing planners and providers;– evaluating components and activities of health services that
cannot be measured in quantitative outcomes.
![Page 9: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/9.jpg)
Application of Qualitative Research to Practice
• As with quantitative research, results from a single study only should not be used to guide practice
• The findings of qualitative research should be synthesized in order to develop recommendations for practice
![Page 10: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/10.jpg)
Research Paradigms
• Serve as a model/world view or philosophy that structures knowledge and understanding
• Define what is to be observed and scrutinised• Prioritise questions• Inform the interpretation of results
![Page 11: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/11.jpg)
Dominant Paradigm 1: Positivist/Empirical-Analytical
• Assumptions– There is an external, real world of objects apart from
people, which is comprehensible to us– Researchers can compare their claims against this
objective reality
• Seeks to predict and control
![Page 12: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/12.jpg)
Dominant Paradigm 2: Interpretive• Assumptions
– Reality as we know it is constructed/produced inter-subjectively (between people).
– Meaning and understanding are developed socially and experientially.
– Researchers’ values are inherent in research. – Findings of knowledge claims are created– All interpretations are located in a particular context,
setting and moment.• Seeks to understand
![Page 13: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/13.jpg)
Dominant Paradigm 3: Critical• Assumptions
– Examines issues of power and envisions new opportunities– Main task social critique, whereby the restrictive and alienating
conditions of the status quo are brought to light– Common assumptions with Interpretivism; however, moves past
describing meaning to emancipation and change. • Seeks to critique and emancipate
![Page 14: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/14.jpg)
Qualitative Methodologies
![Page 15: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/15.jpg)
Interpretive Methodoligies
![Page 16: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/16.jpg)
Phenomenology• Assumes that people come to know a phenomena
through their experience of it• Focuses on INDIVIDUAL meaning• Emphasizes a focus on people's subjective
experiences and interpretations of the world. • Understand how the world appears to others.
![Page 17: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/17.jpg)
Ethnography• Study of Culture/specific cultural group.
– What is the way of life of this group of people?– Everyday life is worth of study.
• Focuses on SOCIAL meaning.• Researcher immerses self in cultural group.
– Field Work - primarily participant observation and interview data
• Focuses on the context of communities• Used increasingly in healthcare to explore the relationship
between health and culture.
![Page 18: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/18.jpg)
Grounded theory• Develop theory grounded in real world;• Searches and generates theoretical explanations
from observations of the world (induction);• The constant collection and iterative analysis of data
to enable theory to emerge; and• Data are not constrained by predetermined
theoretical framework, but define boundaries of inquiry.
![Page 19: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/19.jpg)
Critical Methodologies
![Page 20: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/20.jpg)
Action Research• Basis in critical social science;• Researchers interact with the
participants to achieve change;• Often community-based;
– used with implementation studies in healthcare; and
• Treats the individual as an autonomous being, capable of exercising agency
![Page 21: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/21.jpg)
Discourse analysis• Discourse – to talk, converse; hold forth in speech or
writing on a subject;• Discourse as patterns of ways of representing
phenomena in language;• Application of critical thought to social situations and
the unveiling of (hidden) politics within socially dominant or marginalised discourses.
![Page 22: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/22.jpg)
Qualitative data analysis• Is based on principles of thematic analysis for most
methodologies– A process of identifying and presenting meaning in the
form of themes, metaphors or concepts;• there is wide variability in the terms used and how they are
defined;
– Not suited to all methodologies (i.e. discourse analysis)
![Page 23: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/23.jpg)
Qualitative data analysis
• Three strategies:– Data analysis as a separate step following data collection– Data analysis occurs simultaneously with data collection– Data collection and analysis are ‘staged’
![Page 24: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/24.jpg)
Congruity between Paradigm, Methodology and Methods
![Page 25: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/25.jpg)
![Page 26: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/26.jpg)
Group Work 1• In pairs, work through the Group Work 1 task in the
Workbook.• Reporting back
![Page 27: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/27.jpg)
Session 2: The Systematic Review of Qualitative Evidence
Trends and Debates within the International Literature
![Page 28: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/28.jpg)
Approach Purpose Evidence of Interest
Search Strategy
Critical Appraisal
Data Extraction
Method of Synthesis
Outcome Software Available
Narrative Synthesis
To summarise two or more papers in narrative form
Generic Unspecified/Selective
Not specified Not Specified
Unspecified Review article-type report
NO
Meta-ethno-graphy
The generate new knowledge/theory use processes of interpretation
Findings of qualitative research studies
Not comprehen-sive or exhaustive; seeks saturation – theoretical sampling
Opposed; all studies included as each may provide insight into the phenomena of interest
Extraction of key concepts
Refutation-al synthesis;Reciprocal translation;Line of argument synthesis.
Higher order interpretat-ion of study findings
NO(Although QARI can be used)
Meta-synthesis: approaches
![Page 29: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/29.jpg)
Approach Purpose Evidence of Interest
Search Strategy
Critical Appraisal
Data Extraction Method of Synthesis
Outcome Software Available
Realist Synthesis
To develop and test program theories
Generic; but more suited to qualitative research studies
Not comprehen-sive or exhaustive; seeks saturation
Not specified Extracts positive and negative mechanisms/strategies that influence change
Tacit testing of implicit theories; building theory.
A transfer-able theory on “what works, for whom, in which circum-stances”
NO
Thematic analysis
To aggregate findings of 2 or more studies
Findings of qualitative research studies
Not comprehen-sive or exhaustive; seeks saturation – theoretical sampling
Not specified Extracts of major/recurrent themes in literature
Aggregation of themes/metaphors/categories
A summary of findings of primary studies under thematic headings
NO(Although QARI can be used
![Page 30: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/30.jpg)
Approach Purpose Evidence of Interest
Search Strategy
Critical Appraisal
Data Extraction
Method of Synthesis
Outcome Software Available
Content analysis
To analyse then summarise content of papers.;occurrences of each theme counted and tabulated
Generic Usually comprehen-sive with predeterm-ined search strategy
Not specified
Content extracted then coded using extraction tool designed to aid reproducibility
coded data categorised under thematic headings; also counted and tabulated
A summary of findings, and their rate of occurrence, of primary studies under thematic headings
YES
Meta-synthesis/Meta-aggregation
To aggregate the findings of included studies
Findings of qualitative research studies
Comprehen-sive; detailed search strategy at protocol stage required
Required, using standard-ised critical appraisal instrument
Extraction of findings PLUS data that gives rise to finding using data extraction instrument
Aggregation of findings into categories; and of categories into synthesised findings
Synthesised findings that inform practice or policy in the form of a standardised chart
YES
![Page 31: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/31.jpg)
Meta-synthesis: worked examples
Meta aggregationMeta ethnography
![Page 32: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/32.jpg)
Meta aggregation
• A structured and process driven approach to systematic review drawing on the classical understandings and methods associated with systematic review of quantitative literature as practiced by the Cochrane Collaboration
![Page 33: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/33.jpg)
Meta aggregation
• Based on an a-priori protocol– Established, answerable question– Explicit criteria for inclusion– Documented review methods for searching, appraisal,
extraction and synthesis of data
![Page 34: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/34.jpg)
Meta aggregation• Explicitly aligned with:
– Philosophy of pragmatism• Delivers readily useable findings• Informs decision making at the clinical or policy level
– Transcendental phenomenology• Looks for common or “universal” essences of meaning• Attempts to “bracket” pre-understandings of the reviewer
![Page 35: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/35.jpg)
Transcendental Phenomenology
• Based on the philosophic traditions of Husserlian phenomenology:– the intuitive examination of essences that have immediate
validity;– Seeks to avoid undue influence of the reviewer on the text;– Seeks to generate practice level theory that has explanatory
power for policy or practice– Seeks to preserve the intended meaning of text
![Page 36: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/36.jpg)
The needs and experiences of people with a diagnosis of skin cancer: a
systematic review Janet Barker, Arun Kumar, Wendy Stanton and Fiona Bath-Hextall, JBI Library of
Systematic Reviews, 2011; 9(4):104-121
![Page 37: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/37.jpg)
Inclusion Criteria• Participants
– Adults with a diagnosis of skin cancer.
• Phenomena of Interest – The needs and experiences of people with skin cancer.
• Context– Hospital or community-located dwellers.
• Types of studies – All qualitative studies that described or analyzed the needs and experiences
with people who had been diagnosed with skin cancer.
![Page 38: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/38.jpg)
Methods• Each paper was assessed independently by two
reviewers for methodological quality. The internal validity (quality) of research papers was assessed using JBI-QARI.
• There were no disagreements between the two reviewers and therefore a third reviewer was not required.
![Page 39: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/39.jpg)
Results
![Page 40: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/40.jpg)
Results ( cont/…)
• A total of 12 findings were extracted from the 2 included qualitative papers
• These findings were aggregated into 4 categories on the basis of similarity of meaning.
• The categories were synthesized to generate 2 synthesized findings.
![Page 41: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/41.jpg)
Results/cont..
• The 2 studies included in the review used a qualitative approach but neither specified a particular methodological approach.
• Both studies were conducted in the United Kingdom (UK), using semi structured interviews.
![Page 42: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/42.jpg)
![Page 43: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/43.jpg)
Recommendations arising• There is a real need to increase knowledge of skin cancer so
that people do not delay in seeking medical help as early diagnosis can dramatically improve both prognosis and the patient experience since early lesions are treated more simply compared with larger or neglected lesions.
• Health professionals caring for these patients need to understand the psychosocial concerns of this patient group in order to design services appropriately and to provide patients with the support they need and information that they can easily understand.
![Page 44: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/44.jpg)
Meta-Ethnography• Explicitly aligned with philosophy of interpretivism• Searches for new meaning• Focuses on multiple realities
![Page 45: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/45.jpg)
Hermeneutic Interpretivism• Is the basis for meta-ethnography
– Interpretation of text for its inner meaning– Focuses on the reviewers interpretive skills– Seeks to re-interpret the published literature– Seeks to generate new, mid level theoretical
explanations– Seeks engagement between reviewer and text
![Page 46: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/46.jpg)
Meta ethnography
• From within the social sciences to develop theories from existing ethnographic data,
• Iterative development of emic interpretations, • Incorporates 7 phases, these can be aligned to the
systematic review process, but there is no requirement to do so.
![Page 47: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/47.jpg)
Meta ethnography• Identify the research interest• Decide which studies to include• Read the studies• Determine how the studies are related• Translate the studies in to each other• Synthesize the translations• Express the synthesis
![Page 48: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/48.jpg)
Meta ethnography: 3 stages of synthesis
• First order interpretations– Themes, metaphors or concepts identified
• Second order interpretations– The researchers interpretation of how the identified
concepts relate to each other
• Third order interpretations– Seeks to encompass themes within each other
![Page 49: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/49.jpg)
Interpretation Second Order Interpretation:
• Reciprocal– Like interpretations are brought together “this one is like that one..”
• Refutational– Competing discourses; must also have a relationship that can be
explored
Third Order Interpretation:
• Line of argument– Asks what do the parts infer about the whole
![Page 50: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/50.jpg)
Patient adherence to Tuberculosis treatment
Salla A. Munro, Simon A. Lewin, Helen J. Smith, Mark E. Engel1, Atle Fretheim, Jimmy Volmink. ‘Patient Adherence to Tuberculosis Treatment: A Systematic Review of Qualitative Research’.
PLoS Medicine July 2007.
![Page 51: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/51.jpg)
Inclusion criteria
• Patients, carers or health professionals delivering DOTs
• Perceptions of adherence was the phenomena of interest
• Context was patients undergoing DOTs therapy for TB• Qualitative studies on the perception of adherence
![Page 52: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/52.jpg)
Methods
• Studies were assessed using a checklist• Data extracted using a standardized form• Synthesis was reciprocal and line of argument
![Page 53: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/53.jpg)
Results
• Following screening, 44 papers were included• First order interpretations 8• Second order interpretations 6• Third order interpretations 4• Expression of the synthesis: visual model
![Page 54: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/54.jpg)
![Page 55: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/55.jpg)
Recommendations• Increase visibility of TB programs in the community, which may
increase knowledge and improve attitudes towards TB• Provide more information about the disease and treatment to patients
and communities• Increase support from family, peers, and social networks• Minimize costs and unpleasantness related to clinic visits. Increase
flexibility/ patient autonomy
![Page 56: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/56.jpg)
STEP 2: CATEGORIES
STEP 3: SYNTHESISED FINDINGS
FIRST ORDER ANALYSIS
SECOND ORDER INTERPRETATION
THIRD ORDER INTERPRETATION
QARI METAGGREGATION
META ETHNOGRAPHY
![Page 57: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/57.jpg)
Session 3: Critical Appraisal of Qualitative Evidence
![Page 58: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/58.jpg)
Why Critically Appraise?
• Combining results of poor quality research may lead to misleading understandings of issues explored
1004 references
832 referencesScanned Ti/Ab
172 duplicates
117 studiesretrieved
715 do not meetIncl. criteria
82 do not meetIncl. criteria
35 studies forCritical Appraisal
![Page 59: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/59.jpg)
The Critical Appraisal Process• Every JBI review must set out to use an explicit
appraisal process. Essentially:– A good understanding of philosophical perspectives,
methodologies, methods is required in appraisers; and– An agreed instrument/checklist should be used.
![Page 60: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/60.jpg)
Validity, Reliability and Generalizability
• Judging the quality of qualitative research remains deeply contested
![Page 61: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/61.jpg)
Analogous criteria for paradigmatic assumptions
Quantitative Qualitative
Reliability Dependability
Internal Validity Credibility
External Validity Transferability
![Page 62: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/62.jpg)
Reliability/dependability
• Appropriateness of methodology, methods and implementation of the research methods, regardless of paradigm
• The focus of dependability is on achieving consistent quality rather than repeatability.
![Page 63: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/63.jpg)
Internal validity/credibility• Credibility addresses whether a finding has been
represented correctly– Assessment of credibility is multi-dimensional, including
goodness of fit and representativeness, – Credibility is auditable - the process may be based upon
researcher confirmation, member checks, peer checks, second researcher analysis, or observation.
![Page 64: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/64.jpg)
External validity/transferability• Findings are not generalizable in the quantitative sense of
the word – generalization is “narrowly conceived in terms of sampling and
statistical significance.”– “qualitative research is directed toward naturalistic or idiographic
generalizations, or the kind of generalizations made about particulars”
– Schofield (1990) describe qualitative metasynthesis as “cross-case generalizations created from the generalizations made from, and about, individual cases.” Sandelowski et
al(1997)
![Page 65: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/65.jpg)
Critical Appraisal• General acceptance of need for quality• Ongoing debate around the role of appraisal• Particular focus of debate on role of scales and sum
scores in appraisal• In practice, appraisal instruments for qualitative
research tend to focus on establishing the degree to which the evidence applies to practice (transferability) rather than internal validity (credibility)
![Page 66: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/66.jpg)
The premise
• Methods applied during the systematic review of qualitative evidence should firstly be congruent with the universally accepted process of systematic review.
• The characteristics of a systematic review might be debated in terms of the detail, but there is general acceptance of a series of steps, stages or processes
![Page 67: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/67.jpg)
The Joanna Briggs Approach• Establishing credibility in qualitative research requires:
– A standardized approach that is clearly defined, articulated, and applied, and
• Takes in to consideration the relationship between:– Methodology and methods– Methods and data– Data and conclusions
![Page 68: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/68.jpg)
![Page 69: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/69.jpg)
The Qualitative Assessment and Review Instrument (QARI)
![Page 70: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/70.jpg)
• QARI provides a systematic process mirroring that of systematic reviews of quantitative research whilst sensitive to the nature of qualitative data.
• a significant proportion of qualitative work is not designed to address questions or issues of an immediate practical nature
• this is consistent with the traditions of many qualitative methodologies
• JBI focuses on evidence to improve global health, reviews with a clinical or policy focus are encouraged.
![Page 71: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/71.jpg)
![Page 72: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/72.jpg)
![Page 73: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/73.jpg)
![Page 74: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/74.jpg)
![Page 75: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/75.jpg)
Critical Appraisal in JBI-QARI
![Page 76: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/76.jpg)
![Page 77: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/77.jpg)
Critical Appraisal of Interpretive Research
• In interpretive and critical inquiry validity relates to the rigor of the process of inquiry– limiting bias to establish validity in the appraisal of
quantitative studies is antithetical to qualitative approaches to inquiry
• QARI incorporates a checklist to appraise rigor generically
![Page 78: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/78.jpg)
![Page 79: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/79.jpg)
• Read the 2 papers provided and then undertake critical appraisal of the paper using the JBI Critical Appraisal Checklist for Interpretive and Critical Research.
• Report Back• Enter the details into JBI QARI
Group Work 2
![Page 80: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/80.jpg)
Session 4: Data Extraction
![Page 81: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/81.jpg)
Study data and Data Extraction
![Page 82: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/82.jpg)
Data most frequently extracted
1004 references
832 referencesScanned Ti/Ab
172 duplicates
117 studiesretrieved
715 do not meetIncl. criteria
82 do not meetIncl. criteria
35 studies forCritical Appraisal
26 studies incl.in review
![Page 83: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/83.jpg)
Considerations in Data Extraction • Source - citation and contact details• Eligibility - confirm eligibility for review• Methodology - approach to the inquiry• Methods - how data were collected• Phenomena of Interest - the described experience or activity• Setting - specific context of study eg, hospital• Geographical location - region or country• Culture - cultural characteristics of setting or participants• Participants - demographic data such as gender, age etc• Data analysis - analytic approach taken, eg thematic analysis etc
![Page 84: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/84.jpg)
![Page 85: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/85.jpg)
Extracting Findings• The units of extraction in this process are specific findings (and illustrations from the
text that demonstrate the origins of the findings);• In Meta-aggregation a finding is defined as: A conclusion reached by the
researcher(s) and often presented as themes, metaphors, findings, concepts, conclusions etc;
• Some papers may present themes and sub themes (or findings of the qualitative analysis at different levels); for such papers the reviewer will need to decide which of these levels represent authentic findings; however, only findings of one level should be extracted from a specific paper.
• No matter what level is decided, the actual verbatim words of the researcher must be extracted.
![Page 86: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/86.jpg)
Extracting Illustrations• Meta-aggregation requires reviewers to also extract an
illustrative excerpt that the researcher presents in support of that particular finding;
• It is only necessary to extract one such supporting illustration;• The actual verbatim words the researcher uses as the
illustration must be extracted.
![Page 87: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/87.jpg)
![Page 88: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/88.jpg)
Assigning a Level of Plausibility1 to Qualitative Evidence
1 Currently, the term “credibility” is used in QARI; the next version will refer to plausibility
![Page 89: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/89.jpg)
Levels of Plausibility- Qualitative• Unequivocal - relates to evidence beyond reasonable doubt• Plausible - those that are, albeit interpretations, plausible in light of
data and theoretical framework. • Not Supported - when 1 nor 2 apply and when most notably findings
are not supported by the data– Should not be included in synthesis to inform practice
![Page 90: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/90.jpg)
![Page 91: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/91.jpg)
![Page 92: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/92.jpg)
Session 5: Data Analysis and Meta-synthesis
![Page 93: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/93.jpg)
General Analysis - What Can be Reported and How
– What phenomena of interest has been evaluated– The meaningfulness/appropriateness/feasibility of the
experience/activity– Contradictory findings and conflicts– Limitations of study methods– Issues related to study quality– The use of inappropriate definitions– Specific populations excluded from studies– Future research needs
![Page 94: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/94.jpg)
Meta-Synthesis
• Analysis and synthesis of qualitative studies • Based on processed data• Aim of meta-synthesis
– is to assemble findings; – categorize these findings into groups on the basis of similarity in
meaning; – aggregate these to generate a set of statements that adequately
represent that aggregation.
![Page 95: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/95.jpg)
Synthesis of the Results of Qualitative Studies
• Differing methodologies, such as phenomenology, ethnography or grounded theory, can be mixed in a single synthesis of qualitative studies because the synthesis is of findings and not the data.
• This is a critical assumption of meta-aggregation.
![Page 96: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/96.jpg)
As qualitative studies differ from RCT, meta-synthesis utilizes a different approach from that used during meta-analysis.
Both approaches provide an understanding based on populations, settings and circumstances
Unlike meta-analysis, meta-synthesis deals in multiple realities and so provides but one interpretation of the phenomenon.
Meta Synthesis vs Meta Analysis
![Page 97: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/97.jpg)
Executing Meta Synthesis using QARI
![Page 98: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/98.jpg)
Assembling the findings
• Once findings are extracted from all included papers;– The reviewer transitions from a focus on papers to a
focus on assembling all of the findings– This is executed inside JBI-QARI
![Page 99: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/99.jpg)
![Page 100: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/100.jpg)
Creating Categories
• Categorization involves repeated, detailed examination of the assembled findings• The reviewer identifies groups of findings on the basis of similarity in meaning to
create categories– A category “Name” is the element of analysis that appears in the synthesis and
is therefore a full description that conveys the whole, inclusive meaning of the category; that comprehensively represents the meanings embodied in the findings it encompasses
– A Category “Summary” is a brief explanatory statement created by the reviewer to assist interpretation and auditabililty.
![Page 101: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/101.jpg)
![Page 102: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/102.jpg)
![Page 103: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/103.jpg)
Allocating Findings to Categories
![Page 104: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/104.jpg)
![Page 105: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/105.jpg)
![Page 106: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/106.jpg)
Synthesized Findings• In meta-aggregation a synthesized finding is an overarching
description of a group of categorized findings that allow for the generation of recommendations for practice.
![Page 107: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/107.jpg)
Creating Synthesized findings• Meta-aggregation of categories involves repeated, detailed examination of the
assembled categories• The reviewer identifies groups of categories on the basis of similarity in meaning
– A Synthesized Finding “Name” is the element of analysis that conveys the whole, inclusive meaning of the categories; and that comprehensively represents the meanings embodied in the findings it encompasses
– A synthesized finding “Name” must also draw a conclusion or claim and communicate the probability of that conclusion or claim
– A Synthesized Finding “Summary” is a brief explanatory statement created by the reviewer to assist interpretation and auditabililty.
![Page 108: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/108.jpg)
Creating Synthesized Findings– Can be stated propositionally as “if-then” statements
• e.g. “If a patient is awaiting a final diagnosis, their relatives will sometimes feel as if they are not involved”
– Preference for the indicatory form (i.e. suggesting or demonstrating the expedience or advisability of action - emphasizing the probability of the claim)
• “Relatives of patients awaiting a final diagnosis of brain death may feel as if they are not involved if strategies to include them are not pursued”.
![Page 109: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/109.jpg)
![Page 110: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/110.jpg)
Allocating categories to synthesized findings
![Page 111: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/111.jpg)
![Page 112: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/112.jpg)
![Page 113: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/113.jpg)
![Page 114: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/114.jpg)
Session 6: QARI Trial
![Page 115: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/115.jpg)
Day 2
![Page 116: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/116.jpg)
Program Overview
![Page 117: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/117.jpg)
Overview
• This module considers how to appraise, extract and synthesize evidence from qualitative studies and text and opinion papers.
![Page 118: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/118.jpg)
Session 7: The systematic review of text and opinion
![Page 119: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/119.jpg)
Text, Expert Opinion and Discourse– Narrative and text (e.g. review papers, discussion papers and policy
documents) present information that arises out of experience or observation. Much of the resources drawn on to develop policy are narrative and textual in nature rather than the outcomes of formal research
– The opinions of leaders, professional organizations and learned bodies draw on the collective wisdom of experienced experts and are often referred to for guidance in relation to policy making and professional practice
![Page 120: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/120.jpg)
• Expertise is highly regarded and is linked to the ability of an expert to “have to hand” information in a given area and is associated with the possession of large amounts of knowledge and fluency in applying this knowledge
• Discourse refers the verbal interchange of ideas that is grounded in language and in the context within which it occurs. Discourse in the professional and public domains is a source of knowledge that can be used to inform policy and clinical decision making.
![Page 121: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/121.jpg)
Text, Expert Opinion and Discourse as Evidence for Policy and Practice
• Narrative, opinion, expertise and discourse often represent the best available evidence in areas where research is limited, or where the knowledge that is needed is generally generated through policy-making or other processes rather than through formal research
• This kind of knowledge cannot be ignored as legitimate sources of evidence for policy and practice
![Page 122: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/122.jpg)
• While this kind of evidence is not a product of “good” science it is empirically derived and mediated through the cognitive processes of health professionals or policy makers who have typically been trained to be analytical
• The superior quality of evidence derived from research is not denied rather, in its absence, it is not appropriate to discount opinion as non-evidence
![Page 123: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/123.jpg)
The Narrative, Opinion and Text Assessment and Review Instrument (NOTARI)
![Page 124: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/124.jpg)
![Page 125: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/125.jpg)
![Page 126: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/126.jpg)
Critical Appraisal of Evidence arising out of Opinion and Text
![Page 127: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/127.jpg)
![Page 128: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/128.jpg)
Critical Appraisal of Text and Opinion• This focus on limiting bias to establish validity in the
appraisal of quantitative studies is not possible when dealing with text and opinion
• The assessment of validity within the JBI-NOTARI process focuses on:
examining the opinion; identifying the credibility of the source of the opinion; establishing the motives that underlie the opinion; and locating alternative opinions that give credence to it or, conversely,
question it.
![Page 129: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/129.jpg)
![Page 130: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/130.jpg)
Critical Appraisal of Evidence arising out of Text and Opinion
Report Back
![Page 131: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/131.jpg)
Extracting ConclusionsExtracting Conclusions
• The units of extraction in this process are specific
conclusion stated by the author/speaker and the
text that demonstrate the argument or basis of the
conclusion
![Page 132: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/132.jpg)
![Page 133: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/133.jpg)
Meta-SynthesisMeta-Synthesis
![Page 134: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/134.jpg)
• The aim of meta-synthesis is to assemble conclusions; categorise these conclusions into groups on the basis of similarity in meaning; and to aggregate these to generate a set of statements that adequately represent that aggregation. These statements are referred to as synthesised findings - and they can be used as a basis for evidence based practice.
![Page 135: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/135.jpg)
![Page 136: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/136.jpg)
Session 8: NOTARI Trial
![Page 137: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/137.jpg)
Session 9: Protocol Development
![Page 138: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/138.jpg)
Session 10: Protocol Presentations
![Page 139: Appraisal, Extraction and Pooling of Qualitative Data and Text - Evidence from qualitative studies, narrative and text. JBI/CSRTP/2013-14/0003](https://reader035.vdocument.in/reader035/viewer/2022062309/56649edf5503460f94befa09/html5/thumbnails/139.jpg)
Session 11: Assessment