critiquing aresearch articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · critiquing...

5
MAUREEN GIUFFRE, PhD, RN Critiquing a Research Article literature and sometimes there is a separate subtitle. Whether or not it is subtitled or separated from the review of literature, the point of the introduction is to orient you to the problem that is to be investi- gated. In prose fonnat, the author may tell you something like the following (depending on subject matter of course): 1. The American populace has a high incidence of adult onset diabetes. 2. The vast majority of Americans are totally sedentary, getting no regular exercise. 3. Adult onset diabetes may be preventable with regular vigorous exercise. 4. The key to getting Americans exercising and keeping them exercising is unknown. At this point, the author will conclude the introduction with a statement something like. "Therefore it was the intention of this research to evaluate the success of a motivational program in keeping adults compliant with an exercise pro- gram." When critiquing the introduction you should ask yourself if the author has made a good case for the study he/she is about to present. Does the research question logically follow the points made previ- ously? From this section, you should understand what the author's intentions are. Is the author telling you that what he/she is seeking is basic knowledge for the understanding of a problem (so that others may come up with interventions), or is he/she seeking the solution to a clinical problem. The author may present definitions of terms or assumptions used in the research or may wait until the Review of Literature or even the Methods Journal of PeriAnesthesia Nursing. Vol 13, No 2 (April), 1998: pp 104·1 ('f Maureen Giuffre, PhD, RN, is a Clinical Research Consultant in Private Practice, Salisbury, MD. Address correspondence to Maureen Giuffre, PhD, RN, 26361 High Banks Dr, Salisbury, MD 21801. © 1998 by American Society of PeriAnesthesia Nurses. 1089-9472/98/1302 -0005$03. 00/0 Most research articles begin with some sort of an introduction. This section may be labeled "introduc- tion," but most often it is not. Sometimes, the introduction runs directly into the review of the INTRODUCTION 104 It is often the assignment of the perianesthesia nurse to critique a research article to evaluate a suggested procedure or instrument for implementation the patient care area. Although details of what should appear in each section of the research article have appeared in past reports in this series, this article will present an outline for critiquing the entire research manuscript. © 1998 by American Society of Peri Anesthesia Nurses. TITLE AND ABSTRACT It is the title and abstract of an article that draws our attention and peaks our interest. When used in that manner neither require critique, but should the assignment be to critique a research manuscript (for a class), we will need to look at the title and abstract in the same detail that we look at the rest of the manuscript. Unfortunately, we cannot do so until we have read and critiqued the article. So we will return to the title and abstract at the end of this report. This series I have been wntmg on understanding re- search has focused on critiquing one section of the research manuscript at a time. This article will reference the previous manuscripts but will attempt to bring it all together in one place. A LTHOUGH MUCH attention is given to imple- menting research in nursing practice, the

Upload: others

Post on 10-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Critiquing aResearch Articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · Critiquing aResearch Article literature and sometimes there is a separate subtitle. Whether or

MAUREEN GIUFFRE, PhD, RN

Critiquing a Research Article

literature and sometimes there is a separate subtitle.Whether or not it is subtitled or separated from thereview of literature, the point of the introduction isto orient you to the problem that is to be investi-gated. In prose fonnat, the author may tell yousomething like the following (depending on subjectmatter of course):

1. The American populace has a high incidenceof adult onset diabetes.

2. The vast majority of Americans are totallysedentary, getting no regular exercise.

3. Adult onset diabetes may be preventable withregular vigorous exercise.

4. The key to getting Americans exercising andkeeping them exercising is unknown.

At this point, the author will conclude theintroduction with a statement something like."Therefore it was the intention of this research toevaluate the success of a motivational program inkeeping adults compliant with an exercise pro-gram."

When critiquing the introduction you should askyourself if the author has made a good case for thestudy he/she is about to present. Does the researchquestion logically follow the points made previ-ously? From this section, you should understandwhat the author's intentions are. Is the authortelling you that what he/she is seeking is basicknowledge for the understanding of a problem (sothat others may come up with interventions), or ishe/she seeking the solution to a clinical problem.

The author may present definitions of terms orassumptions used in the research or may wait untilthe Review of Literature or even the Methods

Journal of PeriAnesthesia Nursing. Vol 13, No 2 (April), 1998: pp 104·1 ('f

Maureen Giuffre, PhD, RN, is a Clinical Research Consultantin Private Practice, Salisbury, MD.Address correspondence to Maureen Giuffre, PhD, RN, 26361

High Banks Dr, Salisbury, MD 21801.© 1998 by American Society of PeriAnesthesia Nurses.1089-9472/98/1302 -0005$03. 00/0

Most research articles begin with some sort of anintroduction. This section may be labeled "introduc-tion," but most often it is not. Sometimes, theintroduction runs directly into the review of the

INTRODUCTION

104

It is often the assignment of the perianesthesia nurse to critique a researcharticle to evaluate a suggested procedure or instrument for implementationthe patient care area. Although details of what should appear in each sectionof the research article have appeared in past reports in this series, this articlewill present an outline for critiquing the entire research manuscript.© 1998 by American Society of Peri Anesthesia Nurses.

TITLE AND ABSTRACT

It is the title and abstract of an article that drawsour attention and peaks our interest. When used inthat manner neither require critique, but should theassignment be to critique a research manuscript (fora class), we will need to look at the title and abstractin the same detail that we look at the rest of themanuscript. Unfortunately, we cannot do so untilwe have read and critiqued the article. So we willreturn to the title and abstract at the end of thisreport.

Thisseries I have been wntmg on understanding re-search has focused on critiquing one section of theresearch manuscript at a time. This article willreference the previous manuscripts but will attemptto bring it all together in one place.

ALTHOUGH MUCH attention is given to imple-menting research in nursing practice, the

Page 2: Critiquing aResearch Articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · Critiquing aResearch Article literature and sometimes there is a separate subtitle. Whether or

CRITIQUING RESEARCH

section if that is appropriate. There is no perfectplace to put definitions or assumptions, and they arenot always necessary, but if you find yourselfreading and not knowing how the author is using aterm, they obviously have placed their definitionstoo late in the text.

When reading this and every section of themanuscript, ask yourself if it is well written. Peoplewho have something important to say can makethemselves understood without relying on the crutchof jargon. A well-written research report does notnecessarily involve the largest or most obscurewords, nor is there usually a prize for the longest,most convoluted sentence. Having said that, Flor-ence Down, Editor of Nursing Research, once helda contest for the worst opening sentence in aresearch report. The winners were wonderfullyawful. First place prize was given to Donna Diersfor the following sentence. (The original sentencehad 12 reference citations which I have left offbecause I could not bear to type them.)

"The interaction coefficient in dyadic helping relationshipsas occurs between nurses and patients or nurse practitionersand patients or clients (for the distinction is characterizedby mutuality of duality as opposed to consensus ofopportunity) correlates positively with hierarchical but notdominance-performing distribution of rewards (as occursin balanced professional negotiations) and requires investi"gation beginning with taxonomical theoretically qualitativeinvestigation but includes quasi-experimental derived hy-potheses. " 1

That sentence is quite amusing, if you have thatsort of sense of humor, and if you had read so manysimilar sentences that were not intended to befunny. If you find yourself reading that sort ofprose, find something more useful to do with yourtime. Because even if you can stay awake longenough to wade through the manuscript you willhave no idea what was said when you are finished.

THEORETICAL OR CONCEPTUALFRAMEWORK

This section may precede or follow the review ofliterature section, depending on the subject, or maybe entirely absent. When I have my academic capon, I will say that all research can benefit from theuse of a theoretical framework. Studying a questionwithin the context of a theoretical framework willmake the findings more useful in the long haul. Onthe other hand, the author may not be interested infitting his/her question into a larger picture. They

105

may simply be looking for the best widget for aparticular job. This is the author's choice.

Should the author choose to use a theory, theauthor needs to use it, not just drop it off like theQueen of Spades when playing Hearts. If the authoruses a theoretical framework, he/she is (suppos-edly) doing so to (1) test a theory, (2) to understanda process or behavior, or (3) to organize a mass ofwhat may appear to be disparate data so that thereader can understand it better.2 If the author isusing a theory for one of these three reasons, he/shewill be logically required to revisit the theory in thediscussion section.

One final point. Just because we are nurses doesnot necessarily mean that we are required to use"Nursing Theories" to frame our questions orunderstand our answers. If a nurse who happens tobe doing research on a particular group of peoplechooses to use a theory from sociology, and itworks for the nurse and his/her study, this is fine.Requiring the nurse to use a "Nursing Theory" is apolitical issue, not a research or science issue.

THE REVIEW OF LITERATURE

In short, the review of literature section shouldpresent related research, research findings, and thegaps in the research.2 Good reviews of literaturewill discuss the methods ofthe previous studies andtry to make sense out of the failures of the past.This way the reader will know that the author isprepared to undertake the next step in the researchon this subject. The author may say somethingalong the following lines:

1. Tom used the Health Belief model to designhis interventions in a study of exercise, butdiscovered that changing people's understand-ing of the benefits of exercise did not changetheir behavior.

2. Jane used the Self-efficacy theory to frameher interventions. Unfortunately, her 2-day-a-week, 6-week intervention did not show achange in behavior at 6 months.

3. Dick also used the Self-efficacy theory, butinstead of a 2-days-per-week intervention heprovided the subjects with a 4-day-per-weekintervention for 6 weeks. Unfortunately, hismethod of evaluating behavioral change mayhave been insensitive.

4. In conclusion, the past literature suggests thatthe Self-efficacy theory may be useful inframing interventions designed to change

Page 3: Critiquing aResearch Articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · Critiquing aResearch Article literature and sometimes there is a separate subtitle. Whether or

106

exercise behaviors, but the past researchinvestigated interventions or used evaluationtools that may not have been powerful enoughto demonstrate lasting change.

All too often an author will present a series ofarticles that seem to do nothing more than supportwhat appears to be the author's point of view. Insome cases, the author makes such a good case forwhy what he/she is investigating must be true wehave to wonder why the author is doing theresearch at all.

There is almost no place for secondary refer-ences, including review articles and books, in thereview of literature section. The author needs toconfine himselflherself to primary references sothat he/she can evaluate the methods of priorresearch. It is also important to evaluate whetherthe author has been selective in the use of pastresearch so that he/she cites only articles thatsupport hislher view-point. Sometimes this is hardto know. Presumably, if you are evaluating anarticle because it discusses a procedure or instru-ment you are interested in implementing on yourunit, this will not be the only article you read onthat subject. If you find the author is missing a fewkey articles, then you will know that the review ofliterature is not complete.

The beginning section of a research articleshould present the problem, the past literature, andpossibly a theoretical framework in which to studythe problem. This section will close with eitherdetailed research questions, specific aims, or hypoth-eses. Specific aims will look something like this:

This study is designed to:1. Recruit nonexercising men and women be-

tween the ages of 40 and 49 into a diabetes/cardiovascular risk evaluation.

2. Enter 200 people at risk for diabetes, withrecent onset diabetes, or at risk of cardiovas-cular disease who can state they are preparedto commit at least 3 hours a week to anexercise program.

3. Randomly assign half to a passive interven-tion and half to an aggressive intervention.

4. Institute a I-year exercise program that re-tains at least 70% of the subjects in theaggressive intervention group.

5. Compare exercise participation rates of groups.A hypothesis on the same subject will look

something like this. "Adults between the ages of 40and 49 and at risk for adult onset diabetes who

MAUREEN GIUFFRE

receive an aggressive exercise intervention pro-gram will have a higher rate of participation inexercise one year after initiation of the programthan similar adults who are exposed to an educa-tional program only."

Regardless of whether the author chooses to usespecific aims or hypotheses, when evaluating thissection you will need to ask yourself whether thesestatements logically flow from the introductory mate-rial, are they clearly and unambiguously worded,and are they testable. From this point, it will be aneasy transition to the methods section of the article.

METHODSThe methods section usually contains the follow-

ing four subsections: subjects, instruments, proce-dures, and analysis. The first thing you need to askyourself about the entire section is "Does the studydesign fit the question?" For example, if the authorhas made a case for determining the best way torewarm postoperative patients, does he/she then goon to describe a survey designed to describe themost common method of rewarming patients. Themethods must make sense.

Subjects

The subject portion of the methods sectiondescribes who it is that is being studied and howthey were selected. The choice of subjects shouldmake sense in light of what has been said to thispoint. If the researcher has criticized past researchbecause it studied only men or was not ethnicallydiverse, it would not make sense for the currentauthor to use only white men.

Sample refers to the people who are actuallystudied. Population refers to the people to whomthe findings of the research can be generalized. If,for example, a researcher from a particular PACUstated that during the course of 1997, 2,000 patientsreceived care in her PACU. She is interested insome aspect of their care and randomly selects asample of 200 charts from the population of 2,000.She can then generalize to the population and nofurther. It is up to you the reader to decide if herfindings have some relevance to your population inyourPACU.

Few studies contain subjects that are randomlyselected from some population. Most nursing stud-ies use convenience samples. In that case, thesample and the population are the same, and Igenerally refer to both of them as the population.

Page 4: Critiquing aResearch Articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · Critiquing aResearch Article literature and sometimes there is a separate subtitle. Whether or

CRITIQUING RESEARCH

The population should make sense in terms ofwhat it is the researcher is looking for. If the authoris looking for a very clear answer to a verycircumscribed question, then a small homogeneoussample may be appropriate. On the other hand, ifthe author knows how something works in the testtube or in a narrow population, and now wants tosee how it works in the real world, a much broaderpopulation might be called for.

I often see articles criticized because a gender,age, or ethnically diverse population was notincluded in the sample. Requiring the author toexpand the· study population to reflect the popula-tion of the world may be an unnecessary expense.If, for example, the question is "does exercisereduce the rate of bone loss in postmenopausalwomen?" it is probably not necessary to study anethnically diverse population (I am not aware ofany evidence that would suggest that the bones ofdifferent ethnic groups respond differently to exer-cise). On the other hand, if the question concernedaccess to exercise facilities and social support tomaintain an exercise routine, then a diverse groupis a must.

The point is that the study population must fit thequestion. The size of the population must also beappropriate. If the author is looking at a questionthat involves subtle changes or a lot of variables,then a large population will be required. The authorshould tell the reader how he/she came up with theplanned sample size.

Finally, the author should explain what was doneto ensure the subjects' human rights were notviolated, or in the case of nonhuman animals, thatthe animals were treated in a humane manner. Inboth cases, the authors must note that the researchwas approved by the appropriate Human Subjectsor Animal Subjects Research Committee. Thesecomments are sometimes included in the Proceduresection.

Tnstruments

At this point, the author may chose to present amini-review of the literature on the selected instru-ments. It must be clear that the instruments (be theythermometers or questionnaires) are the best toolsto answer the questions. The author should addressquestions of validity, reliability, and sensitivity.

For a tool to be valid, it must measure theconcept under question.3,4 Remember, there may beplenty of evidence that a tool is valid under one

107

circumstance, but the author has to tell the readerwhy it is a valid measure of the concept he/she isinvestigating under the circumstances he/she has set up.

Instrument reliability will need to be substanti-ated. If there is more than one investigator, theauthor needs to show that when the instruments areused by more than one investigator on the samesubjects, the same scores are obtained (interrater-reliability). Just as with validity, the author needs toevaluate the reliability of the tools used in themanner that he/she has used them.

Sensitivity refers to the instrument's ability topick up subtle changes in the phenomenon beingstudied. Should the intervention that is beingstudied result in a 10% reduction in postoperativepain for most people, but the instrument being usedonly picks up reductions of 20% or more, then itwill appear as though there is no improvement. Inthis case, the instrument is insensitive. Thereshould be some indication in the instrument sectionthat the author has considered this.

Procedure

The procedure section should describe to youhow the study was done, What was done? Inreference to the questions that were presented in theearlier part of the manuscript, the author should, atthis point, tell you exactly what an aggressiveexercise regimen consisted of. The author shouldgive you enough detail so that you could repeat thestudy exactly if you wanted to. Duration of interven-tion and evaluation schedule should be clear.

It is at this point that the major risks to internalvalidity will be apparent.5 You should ask yourselfwhen reading this section "Is there anything elsegoing on here that could explain the results, otherthan what the author has done?" Does the re-searcher evaluate change too infrequently? Are thecontrol group and experimental group exposed toinconsistent influences other that the experimentalintervention, that might explain their differences?Is the evaluator blinded to which intervention thesubject received?

AnalysisIn this section, the author will describe how

he/she plans to analyze the data. The author shoulddescribe how missing data will be handled andwhat statistical tests will be used.

Page 5: Critiquing aResearch Articlemis.nurse.cmu.ac.th/mis/download/course/lec_566823... · Critiquing aResearch Article literature and sometimes there is a separate subtitle. Whether or

REFERENCES

108

RESULTS

The results section should not be Greek or someother language that you cannot understand. If theresearcher is publishing a manuscript in the Journalof PeriAnesthesia Nursing, then he/she is undersome obligation to present the findings in a mannerthat the readership will understand.

The results section should begin with a descrip-tion of what was found and then proceed to astatistical analysis if appropriate. It is not accept-able for the author to describe the statistical signifi-cance of a finding without ever telling you whatthat finding was. Remember that statistical testsusually only tell you whether the difference foundin the groups can be attributed to chance or not; itdoes not tell you the size of the difference or if thedifference is an important one.6

The results should logically follow the questionsof the study. If the author presents a series ofquestions to be asked or hypothesis to be tested, theresults should march through those questions one ata time.

DISCUSSION

The discussion section usually begins with abrief synopsis of the study findings. Is this synopsisaccurate?

Does the author interpret or discuss the findingsso that they have meaning to the readers of thejournal in which he/she has presented in the study?The researcher should interpret the findings in lightof other findings-similar and conflicting. Does theauthor suggest implications for the findings withoutexaggerating? Does the author revisit the theoreti-

1. Downs FS: Congratulations! You're the worst! Nurs Res34:70, 1985

2. Giuffre M: Reading research critically: The review of theliterature. J Post Anesth Nurs 9:240-243,1994

3. Giuffre M: Reading research critically: Assessing thevalidity and reliability of research instrumentation-Part 1. J PostAnesth Nurs 10:33-37.1995

4. Giuffre M: Reading research critically: Assessing the

MAUREEN GIUFFRE

cal framework and tell us what the findings mean interms of the theory? The author also should de-scribe limitations of the study without going intothe obvious ones. The author may, but does nothave to, make suggestions for further study basedon his/her findings.

ABSTRACT AND TITLE

Finally, you are in a position to evaluate theabstract and title. The title should be reasonablyshort but descriptive. I always feel annoyed by thetitles that says "The impact of X on Y" but whenyou read the actual manuscript you discover thereis no impact of X on Y. The study found no impact.The title should be accurate and if possible reflectthe population studied, ie, "The lack of impact of Xon Y in little old ladies."

The abstract should briefly describe the question,the sample, the methods, and the findings. Theabstract should not contain so much informationthat it is redundant to read the manuscript. It isinappropriate for the abstract to reflect only the"positive" findings and to conveniently overlookthe "negative" findings. The abstract should con-tain the important words of the study since it isoften used for indexing the study.

When critiquing a research article or a number otrelated articles, the more you read the easier it willget, especially if you combine reading researchwith reading about research. It will also be easierand more enjoyable if you do it with a small group.Different people will bring differing perspectivesand past knowledge to the process.

validity and reliability of research instrumentation-Part 2. J PostAnesth Nurs 10:107-112, 1995

5. Giuffre M: Reading research critically: Threats to internalvalidity. J Post Anesth Nurs 9:303-307,1994

6. Giuffre M: Reading research critically: Statistical signifi-cance. J Post Anesth Nurs 9:371-374,1994

7. Giuffre M: Reading research critically: The discussionsection. J Peri Anesth Nurs 11:417-420, 1996