chapter 026

24
1 Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc. Chapter 26 Interpreting Research Outcomes

Upload: stanbridge

Post on 11-Aug-2015

35 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Chapter 026

1Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Chapter 26

Interpreting Research Outcomes

Page 2: Chapter 026

2Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Interpreting Research Outcomes (Cont’d)

Requires abstract thinking Introspection, reasoning, and intuition Usually the final chapter of theses or

dissertations/“Discussion” section of research papers

Aided by discussion with others

Page 3: Chapter 026

3Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Process of Interpreting Research Outcomes

For quantitative, outcomes, and intervention studies Examining study evidence Determining findings Forming conclusions Identifying limitations Generalizing the findings Considering implications for practice Suggesting further studies

Page 4: Chapter 026

4Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Examining Evidence

Identify the limitations of study and consider how these might affect study findings and conclusions

Limitations in a research article are Not flaws, per se, but rather Limitations to the believability of results Limitations to generalization

Page 5: Chapter 026

5Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from the Research Plan

Does study hold up logically? How good are the operationalizations? Did the framework really support understanding of variable

connections? Were study concepts really what were being measured? Was the sample representative? And of what was it

representative? Did extraneous variables intrude? Were statistical analyses appropriate to

Levels of measurement? Distribution of data? Research questions?

Page 6: Chapter 026

6Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from Measurement

Again, how well did operational definitions fit framework?

Were the measurements valid and reliable? Were validity and reliability established for

this study? What was the effect size?

Page 7: Chapter 026

7Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from the Data Collection Process

Subject refusal rate Subject attrition Sample size sufficiency Sample representativeness Intervention fidelity Unforeseen events Consistency of measurement techniques

Page 8: Chapter 026

8Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from the Data Analysis Process

Accuracy of data collection, recording, transcription

Missing data management Accuracy of analyses and mention of

statistical package used, if any Adherence to assumptions of statistical tests Appropriateness of statistical tests

Page 9: Chapter 026

9Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from Data Analysis Results

Most direct evidence of the results Interpretation of results from quasi-

experimental and experimental studies, often based on decision theory

Page 10: Chapter 026

10Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from Data Analysis Results (Cont’d)

Five possible types of results: Significant results in keeping with those predicted

by the researcher Nonsignificant results Significant results that oppose those predicted by

the researcher Mixed results Unexpected results

Page 11: Chapter 026

11Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Significant and Predicted Results

Support logical links developed by researcher among the purpose, framework, questions, variables, and measurement methods

Researcher needs to consider alternative explanations for positive findings

Page 12: Chapter 026

12Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Nonsignificant Results

Double-check statistics, to make sure + and – not reversed

Double-check data entry, to make sure reverse-coding occurred, if needed

Re-calculate power analysis, to reflect actual effect size (of this study)

Report results For This Study, not as a general truth

Page 13: Chapter 026

13Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Significant and not Predicted Results

Double-check statistics, to make sure + and – not reversed

Double-check data entry, to make sure reverse-coding occurred, if needed

Think the logic through again, from concepts to operational definitions

Page 14: Chapter 026

14Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Mixed Results

Most common outcome Double-check statistics Recalculate effect size for each variable Re-estimate needed sample sizes for each

test, to check for type II error Additional research is indicated to examine

mixed study results

Page 15: Chapter 026

15Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Unexpected (Serendipitous) Results

Relationships not in study hypotheses Re-examine theoretical ideas, especially

when explaining Focus on study hypotheses, but include

serendipitous results in report, too Use these findings to develop or refine

theories and to formulate later studies

Page 16: Chapter 026

16Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence from Previous Studies

Comparison of this study with similar studies Is a pattern present? Inconsistencies explained, as far as possible

Page 17: Chapter 026

17Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Determining Findings

Developed by evaluating evidence and translating and interpreting study results

Often organized by objectives or hypotheses Dialogue with colleagues or mentors to clarify

meaning or expand implications of research findings

Page 18: Chapter 026

18Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Forming Conclusions

Synthesis, conjoining, expansion of findings Remember that research never proves

anything Correlation is Never causation Suggestions for subsequent research that

extend findings Descriptive research is generalizable only to

very similar populations

Page 19: Chapter 026

19Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Identifying Limitations

Restrictions or problems in a study that may decrease the generalizability of the findings

Theoretical limitations Methodological limitations

Page 20: Chapter 026

20Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Theoretical Limitations

Framework weaknesses or unsuitability Variable definition weaknesses Framework-to-variable connection shaky

Page 21: Chapter 026

21Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Methodological Limitations

Limit credibility of the findings Weak design Limited control over treatment (intervention)

implementation Instruments with limited reliability and validity Limited control over data collection Improper use of statistical analyses

Restrict generalizability Non-representative sample Single setting

Page 22: Chapter 026

22Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Generalizing the Findings

Extends implications of findings from sample studied to larger population or from situation studied to larger situation

Use caution when choosing extent to generalize

Empirical generalizations Based on accumulated evidence from many

studies Important for verifying or developing theory

Page 23: Chapter 026

23Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Considering Implications

Meanings of conclusions for body of nursing: knowledge, theory, and practice

What will change, because of this study? At Worst: contributes to body of knowledge At Best: contributes to theory, or to actual

clinical practice In The Population From Which Sample Was Drawn

Page 24: Chapter 026

24Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Recommending Further Research

Routine assessment, for purposes of certainty or application

Is the evidence now enough for widespread generalization? If not, one of these is next: Replication Extension of study Change in methodology to establish evidence for practice

Regarding current study Better design, measures, variables Avoidance of pitfalls (logical, human)