chapter 026
TRANSCRIPT
1Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Chapter 26
Interpreting Research Outcomes
2Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Interpreting Research Outcomes (Cont’d)
Requires abstract thinking Introspection, reasoning, and intuition Usually the final chapter of theses or
dissertations/“Discussion” section of research papers
Aided by discussion with others
3Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Process of Interpreting Research Outcomes
For quantitative, outcomes, and intervention studies Examining study evidence Determining findings Forming conclusions Identifying limitations Generalizing the findings Considering implications for practice Suggesting further studies
4Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Examining Evidence
Identify the limitations of study and consider how these might affect study findings and conclusions
Limitations in a research article are Not flaws, per se, but rather Limitations to the believability of results Limitations to generalization
5Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from the Research Plan
Does study hold up logically? How good are the operationalizations? Did the framework really support understanding of variable
connections? Were study concepts really what were being measured? Was the sample representative? And of what was it
representative? Did extraneous variables intrude? Were statistical analyses appropriate to
Levels of measurement? Distribution of data? Research questions?
6Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from Measurement
Again, how well did operational definitions fit framework?
Were the measurements valid and reliable? Were validity and reliability established for
this study? What was the effect size?
7Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from the Data Collection Process
Subject refusal rate Subject attrition Sample size sufficiency Sample representativeness Intervention fidelity Unforeseen events Consistency of measurement techniques
8Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from the Data Analysis Process
Accuracy of data collection, recording, transcription
Missing data management Accuracy of analyses and mention of
statistical package used, if any Adherence to assumptions of statistical tests Appropriateness of statistical tests
9Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from Data Analysis Results
Most direct evidence of the results Interpretation of results from quasi-
experimental and experimental studies, often based on decision theory
10Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from Data Analysis Results (Cont’d)
Five possible types of results: Significant results in keeping with those predicted
by the researcher Nonsignificant results Significant results that oppose those predicted by
the researcher Mixed results Unexpected results
11Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Significant and Predicted Results
Support logical links developed by researcher among the purpose, framework, questions, variables, and measurement methods
Researcher needs to consider alternative explanations for positive findings
12Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Nonsignificant Results
Double-check statistics, to make sure + and – not reversed
Double-check data entry, to make sure reverse-coding occurred, if needed
Re-calculate power analysis, to reflect actual effect size (of this study)
Report results For This Study, not as a general truth
13Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Significant and not Predicted Results
Double-check statistics, to make sure + and – not reversed
Double-check data entry, to make sure reverse-coding occurred, if needed
Think the logic through again, from concepts to operational definitions
14Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Mixed Results
Most common outcome Double-check statistics Recalculate effect size for each variable Re-estimate needed sample sizes for each
test, to check for type II error Additional research is indicated to examine
mixed study results
15Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Unexpected (Serendipitous) Results
Relationships not in study hypotheses Re-examine theoretical ideas, especially
when explaining Focus on study hypotheses, but include
serendipitous results in report, too Use these findings to develop or refine
theories and to formulate later studies
16Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Evidence from Previous Studies
Comparison of this study with similar studies Is a pattern present? Inconsistencies explained, as far as possible
17Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Determining Findings
Developed by evaluating evidence and translating and interpreting study results
Often organized by objectives or hypotheses Dialogue with colleagues or mentors to clarify
meaning or expand implications of research findings
18Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Forming Conclusions
Synthesis, conjoining, expansion of findings Remember that research never proves
anything Correlation is Never causation Suggestions for subsequent research that
extend findings Descriptive research is generalizable only to
very similar populations
19Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Identifying Limitations
Restrictions or problems in a study that may decrease the generalizability of the findings
Theoretical limitations Methodological limitations
20Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Theoretical Limitations
Framework weaknesses or unsuitability Variable definition weaknesses Framework-to-variable connection shaky
21Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Methodological Limitations
Limit credibility of the findings Weak design Limited control over treatment (intervention)
implementation Instruments with limited reliability and validity Limited control over data collection Improper use of statistical analyses
Restrict generalizability Non-representative sample Single setting
22Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Generalizing the Findings
Extends implications of findings from sample studied to larger population or from situation studied to larger situation
Use caution when choosing extent to generalize
Empirical generalizations Based on accumulated evidence from many
studies Important for verifying or developing theory
23Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Considering Implications
Meanings of conclusions for body of nursing: knowledge, theory, and practice
What will change, because of this study? At Worst: contributes to body of knowledge At Best: contributes to theory, or to actual
clinical practice In The Population From Which Sample Was Drawn
24Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.
Recommending Further Research
Routine assessment, for purposes of certainty or application
Is the evidence now enough for widespread generalization? If not, one of these is next: Replication Extension of study Change in methodology to establish evidence for practice
Regarding current study Better design, measures, variables Avoidance of pitfalls (logical, human)