barbara resnick, phd, aprn, bc, faanp university of maryland, school of nursing
DESCRIPTION
University of Maryland Dissemination and Implementation Program Webinar 2: Reporting Research: Methods. Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing Ann Gruber-Baldini, PhD University of Maryland, School of Medicine. Methods Section. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/1.jpg)
Barbara Resnick, PhD, APRN, BC, FAANPUniversity of Maryland, School of Nursing
Ann Gruber-Baldini, PhDUniversity of Maryland, School of Medicine
1
![Page 2: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/2.jpg)
Participants- thorough description, including demographic information, injury or disorder characteristics, mean time since onset
Measures- include a detailed description of all measures used, including their reliability and validity information
Procedure- Include all procedures in detail, so that someone else could replicate your study exactly; include recruitment, research design, and types of statistical analyses
![Page 3: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/3.jpg)
There is no specific page limit, but a key concept is to keep this section as concise as you possibly can.
People will want to read this material selectively.
The reader may only be interested in one formula or part of a procedure.
Materials and methods may be reported under separate subheadings within this section or can be incorporated together.
![Page 4: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/4.jpg)
The objective is to document all specialized materials and general procedures, so that another individual may use some or all of the methods in another study or judge the scientific merit of your work. ◦ It is not a detailed recipe. ◦ it is not a long winded story.
![Page 5: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/5.jpg)
It is awkward or impossible to use active voice when documenting methods without using first person.
First person writing would focus the reader's attention on the investigator rather than the work.
Use third person passive voice. Use normal prose with complete sentences
– avoid informal lists.
![Page 6: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/6.jpg)
Materials and methods are not a set of instructions.
Omit all explanatory information and background - save it for the discussion.
Omit information that is irrelevant to a third party, such as what color ice bucket you used, or which individual logged in the data.
NO RESULTS should be included in methods
![Page 7: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/7.jpg)
The purpose of the section is to make it possible for interested readers to repeat the author’s experiment …to reproduce results.
Explain exactly what was done. Think of bench research:
◦ What experiments were run and how they were run, what equipment and materials were used and how they were used, how much, how often, what, where, when, and why.
![Page 8: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/8.jpg)
Design◦ RCT◦ Quasi exp◦ Repeated measures◦ Single group ◦ Randomization process◦ Who if anyone was blinded?
Double blinded-interventionist doesn’t know (ie provider in drug trials) and participant doesn’t know
In social sciences what methods were used to preserve blinding
![Page 9: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/9.jpg)
Sample◦ Recruitment process-who did the recruitment?◦ Number available; number contacted; number
approached; number consented; number refused; reason for refusal
◦ Eligibility criteria ◦ Determination of sample size ◦ Stopping rules if relevant ◦ Randomization-who performed this?
![Page 10: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/10.jpg)
Declaration that an institutional review board governing research has determined that the study protocol adheres to ethical principles.
Without such approval, no research project can be conducted nor can it be published in a reputable, peer reviewed journal.
![Page 11: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/11.jpg)
Consolidated Standards for Reporting Trials (CONSORT) Diagram
![Page 12: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/12.jpg)
Location of the study◦ For example: If in institutions as relevant a brief
description of site is helpful Inner city Rural 1,000 bed or 25 bed
![Page 13: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/13.jpg)
Measures◦ The what aspect of measures! ◦ Brief description of what data was collected ◦ Brief evidence of reliability and validity◦ Acknowledgement of no evidence of reliability
and validity but rational why the measure was used
![Page 14: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/14.jpg)
Protocol for data collection◦ Who: Unit nurses, research assistant, family◦ When: baseline, 2, 4 and 6 months post
implementation of the intervention ◦ Where: location such as primary care office;
home setting; nursing home room ◦ How: Face to face interview; paper and pencil
test; internet survey etc.
![Page 15: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/15.jpg)
Describe your intervention in sufficient detail to conceptualize a replication.
Reference prior work or refer to the web for further detail (as appropriate)
Write this in the past tense and third person ….The exercise program was implemented every second Thursday of the month and lasted for 2 hours.
….The specimen was centrifuged for 10 minutes.
![Page 16: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/16.jpg)
Gory details can be placed in a table to save room.
What are the gory details?◦ Intervention dosage ◦ Strength of dose◦ Time intervals of dosing◦ Who is implementing the intervention◦ See Conn article WJNR
http://wjn.sagepub.com/content/34/4/427The online version of this article can be found at:DOI: 10.1177/0193945911434627
![Page 17: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/17.jpg)
Gory details◦ Reference if previously developed ◦ Conceptual framework◦ Intervention components◦ Timing of delivery◦ Dose◦ Mode(s) of delivery (e.g., face to face; internet)◦ Intervention target and recipient (e.g., patient or family)◦ Delivery setting◦ Culturally relevant◦ Intervention variations: men/women ; tailoring rules◦ TREATMENT FIDELITY –plan can be presented here
![Page 18: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/18.jpg)
Content: What was the content and how was it delivered
Provider: who delivered it? Format: What method was used (telephone,
individual) Setting: Where was it done Recipient: To whom was the intervention
delivered? Intensity: How often/how long for each touch? Duration: Over what period of time were there
intervention contacts?
![Page 19: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/19.jpg)
Component Description of the Intervention
Component I Environmental and Policy Assessments
The identified facility champion worked with the research function focused care nurse (FFCN) to complete environment and policy assessments using Environment and Policy/Procedures for Function and Physical Activity Evaluation Forms. The findings were used to identify and recommend appropriate and affordable interventions to alter the environment and policies/procedures so that they would optimize function and physical activity of residents (e.g., make pleasant walking areas inside and outside sites).
Component II Education
Education of nursing staff, other members of the interdisciplinary team (e.g., social work, physical therapy), residents and families was done by the FFCN using previously established materials and adult learning techniques. All staff in the communities were invited to attend the 30 minute educational session. For those who are unable to attend a face-to-face session, a printed powerpoint was available. Hard copy educational materials were given to all residents and available family/proxies.
Component III Establishing FFC Goals
The FFCN worked with the champion and staff DCWs to complete Physical Capability Assessments and Goal Attainment Forms with each resident. Goals were established based on capability assessments, communication with other members of the team and input from residents. In addition, the FFCN identified intrapersonal factors (e.g., cognitive status, pain, fear of falling, drug side effects such as somnolence) and worked with members of the health care team to implement interventions to optimize patient participation in functional and physical activity. The FFCN also worked with the champion to make sure the goals were written into the required Service Plans on each resident.
Component IV Mentoring and Motivating
The FFCN worked to motivate all caregivers to provide function focused care throughout the 12 month period by: (a) observing performance of caregivers and providing one-on-one mentoring to incorporate FFC into routine care; (b) providing caregivers with positive reinforcement for providing FFC; (b) meeting with caregivers to address their beliefs about physical activity and feelings and experiences associated with providing FFC; (c) reinforcing benefits of FFC and strengthen outcome expectations; (d) highlighting role models (other caregivers who successfully provide FFC); (e) identifying change-aides and positive opinion leaders to disseminate and implement FFC and eliminate the influence of negative opinion leaders.
![Page 20: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/20.jpg)
Can include a diagram to demonstrate flow thru the protocol◦ Include any deviations or decisions made at
particular points through the process.
![Page 21: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/21.jpg)
How was nesting/clustering addressed? Randomization by setting or individual?
Adjustments done and how done? Sufficient detail for replication
◦ Reference statistical methods for more detail
![Page 22: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/22.jpg)
State if an intention-to-treat analysis was done
State if any participants were excluded for any reason from the analyses
State if subgroup or additional analyses were performed
State level of significance used and be prepared to address inflated p for multiple tests (often addressed through method of analysis).
![Page 23: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/23.jpg)
Statistical methods should be described in sufficient detail to enable a knowledgeable reader with access to the original data to verify the reported results.
References for statistical methods should be to standard works when possible.
Any computer programs used should be identified. Statistical terms, abbreviations, and symbols should be
defined. Details about randomization, if used, should be given, as well
as concealment of allocation to treatment groups, and the method of masking (blinding).
Losses to observation (such as dropouts from a clinical trial) should be reported.
It is recommended to include the word “considered” in descriptions of statistical significance, such as “a P value of less than 0.05 was considered statistically significant”, since the choice of this cut-off point is arbitrary.
![Page 24: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/24.jpg)
Hitting just the right level of detail is difficult in these sections. ◦ Enough detail for a reader to reconstruct his/her
study, but not so much that the relevant points get buried.
◦ Ask yourself at each place: “Would I need to know this to reproduce this experiment?”
◦ This is tooo much!... “We rolled the patient over and wiped the skin clean
prior to applying the dressing. We did this with every dressing change”.…assume they know something.
![Page 25: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/25.jpg)
Not Enough Information is more commonly the problem ◦ Make sure it is replicable!
![Page 26: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/26.jpg)
DO NOT include results in the methods DO NOT include discussion in the methods
![Page 27: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/27.jpg)
If a plan is in place this can be explained in the methods section◦ Procedures or Measures as relevant
![Page 28: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/28.jpg)
Same information is needed in methods◦ Sample◦ Intervention-not usually relevant ◦ Measures◦ Data Analysis
![Page 29: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/29.jpg)
It is probably the case that convenience sampling is the most frequently used in qualitative studies.
State the size and type of sample used in the reported study.
If an unusual variant of sampling is used, it is useful to acknowledge the nature of it.
Other comments about the sampling process may be helpful-snowballing
SIZE and whether saturation was achieved
![Page 30: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/30.jpg)
It is important to explain why your data set is the most illustrative and useful to answer the question you are posing.
Be careful to describe how you picked your sample. What criteria did you use?
Can you compare the data set to other alternatives and why did you choose this one?
Describe the important variations within the data set (for instance age and gender distributions) so that the reader gets a good picture of it.
![Page 31: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/31.jpg)
Descriptive data included? Most qualitative studies (but not all) the data
collection method is usually the interview method. ◦ How the interviews were carried out◦ Location/timing/by whom/ questions asked
Example of write up:◦ All students were interviewed by the researcher on two
occasions, for between 30 and 45 min. All interviews were recorded, with the permission of the students being interviewed. After the interviews, the recordings were transcribed into computer files. Care was taken by the researcher to assure the respondents that they and the place of their work would not be identifiable in any subsequent report. Once the final research report was written, the tapes from the interviews were destroyed.
![Page 32: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/32.jpg)
Describe how the researcher handled the data ◦ ‘The interviews were recorded and transcribed.
The researcher then sorted those data into a range of categories and these are reported below’….is a bit too brief!
Care should be taken with very general terms such as ‘content analysis’, when reporting data analysis. The term is probably so broad as to have little meaning.
![Page 33: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/33.jpg)
Describe carefully each step in the analysis to make it possible for the reader to believe that your conclusions are correct -- or argue against them.
A good rule is to present the analysis of one observation/item/response in detail.
Describe your interpretations during the analysis in a systematic way, in small identifiable steps.
![Page 34: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/34.jpg)
For example might describe “in vivo” coding which uses the participants own words for the codes.
![Page 35: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/35.jpg)
Data analysis was done using basic content analysis(Crabtree B & Miller W, 1992) and started with the first interview. The analysis began with “in vivo” coding(Strauss & Corbin, 1998), or “grounded” coding (Glaser& Strauss, 1967), which involves using the informants' own words to capture a particu lar idea. The following is an example of “in vivo” coding: The code identified was independence…"They feel more independent, you know, because it seems like most of their ability to do things independently goes away when they come here”. The codes identified were grouped based on similarities and differenc es. For example, a number of codes arose from the data that focused on facilitators of restorative care such as encouragement of the resident, cueing the resident, or asking them to “help you out”. These were combined under the theme of “Facilitators " of restorative care. Coding was completed initially by the principal investigator and a code book established. The second nurse investigator with experience in implementing restorative care programs review the coded data and revised the codes and added new codes based on her review. This two coders then reviewed the data and codes together until consensus was achieved between the two reviewers.
![Page 36: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/36.jpg)
◦ All of the interview transcripts were read by the researcher and coded in the style of a grounded theory approach to data analysis (refs). Eight category headings were generated from the data and under these all of the data were accounted for. Two independent researchers were asked to verify the seeming accuracy of the category system and after discussion with them, minor modifications were made to it. In the grounded theory literature, a good category system is said to have ‘emerged’ from the data (refs). Other commentators have noted that, in the end, it is always the researcher who finds and generates that system (refs).
![Page 37: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/37.jpg)
Carefully describe the reliability and validity/Confirmability of your data analysis process◦ Reliability-recognizability of the
findings/transfirmability ◦ Validity-consensus among a group
![Page 38: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/38.jpg)
Credibility of the data refers to the believability, fit, and applicability of the findings to the phenomena under study (Denzin & Lincoln, 2000; Lincoln & Guba, 1985). The focus groups were done when the 12 month intervention was completed in each of the treatment facilities. Since completion of the intervention by site occurred at different times, this allowed the investigators to use the findings from the first focus groups to confirm or refute codes and emerging themes in the subsequent groups.
Confirmability or auditability of the data refers to the objectivity of the factual aspects of the data (Lincoln & Guba, 1985). Confirmability of the data was considered by having other members of the research team review the findings and provide feedback as to whether these findings logically fit with other settings and experiences. Specifically the findings were reviewed by three different RCNs that had worked in the treatment facilities, as well as the co-investigators on the study including three epidemiologists with experience in long term care research.
![Page 39: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/39.jpg)
This guide for observational studies may be helpful:
von Elm E, Altman DC, Egger M, Pocock SJ, et al. (2007) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies. PLoS Med 4(10): e296.
http://www.strobe-statement.org/
![Page 40: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/40.jpg)
Methods Section◦ Design◦ Sample◦ Intervention as appropriate◦ Measures◦ Data analysis
![Page 41: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/41.jpg)
Homework due Thursday, June 13 Please send to: [email protected]
![Page 42: Barbara Resnick, PhD, APRN, BC, FAANP University of Maryland, School of Nursing](https://reader036.vdocument.in/reader036/viewer/2022062423/56814496550346895db137b3/html5/thumbnails/42.jpg)
We’ll see you back here on Thursday, July 25 at 4:30 p.m.