Download - Methodology and IRB/URR
![Page 1: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/1.jpg)
Methodology and IRB
By Dr. James Lani
![Page 2: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/2.jpg)
Methodology and IRB/URRDr. James Lani
Take Away Message
Research design is a blueprint with several components:
• Research method: Quant/Qual/Mixed• Operational constructs: How
constructs are measured• Sampling strategy and procedure
What is Research
Research follows the scientific method. What’s the scientific method?
• Formulation of testable questions or hypotheses
• It’s organized knowledge: logical (theory) and evidence based (observable)
• Precise constructs• Can be disproven (falsifiable)• Parsimonious (simplest
explanation)
![Page 3: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/3.jpg)
Methodology: The Cookbook Metaphor
Cooking Researching
Making a stew… Examining research questions…
Ingredients: 2 lbs beef, I clove garlic…
Preparation: Cube beef, mince garlic…
Cooking Instructions: Bake at 350 for one hour
Celebrate: You’ve made a replicable) stew!
Ingredients: 20 participants, 15 item questionnaire, semi-structured interview.
Preparation: Administer questionnaire before and after lecture; semi-structure interview with participants for 20 minutes using a tape recorder.
Data analysis plan: Conduct dependent sample t-test; transcribe interviews then thematizeparticipants responses.
Celebrate: You’ve conducted a (replicable) research study!
Where the recipe can be replicated
![Page 4: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/4.jpg)
Methodology Essential Ingredients Restate research
question and hypotheses
Overview
Research Design
Participants
Materials/ Instruments
Data Collection
Procedures
Data Analysis Plan
Sample Size
Limitations
![Page 5: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/5.jpg)
Methods Quantitative Qualitative Mixed
You can count it Not quantitative Both
Goal
Tend to answer “What questions” (What is relationship or differences) or “When questions” (when is theory supported…after intervention
Tend to answer “Why questions” (understand why people feel that way) or “How questions” (explore how they see things)
Both
Research methods/ Strategies of data
collection
Experimental (random assignment) and Non-experimental methods (no control group), Observations (time participant)…
Semi-structured interviews, Archive data, Observations (write down positive and negative feeling words)… Both
Operationalize variables
Define constructs/ instruments
Define constructs/ instruments Both
Sampling strategy Discuss sampling process Discuss sampling process
Both
![Page 6: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/6.jpg)
Theoretical (or Conceptual) Framework
Theory is a systematic explanation of behavior of phenomena.
• Theory guides analyses• List existing theories and how your
research questions relate to those theories
• E.g., Theory of mind: the ability to understand that others have their own beliefs, desires, intentions. Empathy. Tested by Faux Pas Task (ability to recognize a faux pas). Research question: Does alcohol abuse impact empathy?
![Page 7: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/7.jpg)
Population
The population is the group you want to generalize to.
• Describe characteristics of population• Why is population relevant to problem
(look at other peer reviewed study’s justification)
• Distinguish the population from the sample
![Page 8: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/8.jpg)
Sampling Frame and Sample
Universe: theoretical
population to generalize to
Population: largest target population
from universe you have access to or
“Sampling Frame”
Original Sample
Final Sample
Attrition
![Page 9: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/9.jpg)
Survey Item
Old (one’s own SE) Change in item New (perceptions of others’ SE)
1I feel that I have a
number of good qualities.“I have” to “she has” I feel that she has a
number of good qualities.
2I feel I do not have much
to be proud of.“I do not” to “she
does not”I feel she does not have much to be
proud of.
3On the whole, I am
satisfied with myself.“I am” to “she is” and “myself” to “herself”
On the whole, she is satisfied with herself.
…
If you need to amend instrument, use a change matrix; do not create your own instrument!
![Page 10: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/10.jpg)
Materials: Informed Consent
• State purpose of project
• State procedure and how long it will take
• State voluntary nature of participation
• State risks (if any)
• Have them sign or state that by filing our survey they are agreeing to participate.
![Page 11: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/11.jpg)
Constructs vs. Variables
Th
eo
reti
cal
Em
pir
ical
Construct A: Social Environment
Construct B: Personality
Variable A: Birth-Order
Variable B: Introversion/ extroversion
![Page 12: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/12.jpg)
Constructs vs. Variables
Constructs are the invisible abstract things we’re measuring (e.g., personality), while variables are the way we’re assessing (measure/operationalize) those invisible things.
Constructs and variables need to be precise (is personality measured by introversion scale or by conscientiousness scale?)
e.g., Intelligence is a construct, while the number of words remembered is a way of assessing intelligence.
e.g., Personality is a construct, while the scores on an introversion/extroversion test is a way to assess an aspect of personality.
![Page 13: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/13.jpg)
VariablesOperationalize Constructs, Make Distinction between
IV’s/DV’s, and Describe Level of MeasurementExample:
Does Empathy differ by group (alcohol abuse vs. no alcohol abuse)?
Empathy is my dependent variable and Group is my independent variable.
Empathy (my construct) is measured by scores on the Faux Pas task. Alcohol abuse (my construct) is measured by 5 or more drinks in one day.
Empathy is a ratio-level variable measured with scores ranging from 0-25, while my Group variable is a nominal-level (categorical-level) variable because participants re in one of two groups (alcohol abuse group or not in alcohol abuse group).
![Page 14: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/14.jpg)
Reliability and Validity
Valid but not Reliable
Reliable but not Valid
Reliable and Valid
![Page 15: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/15.jpg)
Validity• Internal Validity: IV causes a change in the DV (not time or
other covariates, etc.
• External validity: can be generalized to the population
• Construct validity: does the scale measure the theoretical construct
• Translational validity:
• Face validity (items are reasonable)
• Content validity (items match the domains of interest)
• Criterion related validity: measures behave as theory predicts
• Convergent validity: how close the variable aligns with the construct (use EFA)
• Concurrent validity: construct relates to established instruments
• Predictive validity: measure can predict an outcome (GPA→Income)
![Page 16: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/16.jpg)
Reliability
Internal consistency: Cronbach alpha. Average inter-item correlation
Inter-rater: if interval, correlate; if dichotomous, kappa
Test-retest: administer same test at two times
Split half: divide instrument into 2 parts and calculated totals, then correlate totals.
![Page 17: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/17.jpg)
Brief Review: Units of Analyses
Quantitative (e.g., Age)
Nominal-level (Latin for name). Gender (M/F), Grouped (Old = 65+, middle age = 36-64, young = 35 or younger). Assign any number of groups (old = 1, middle = 2, young = 3).
Ordinal-level is ranked (Latin for showing order). GPA (A-F), or age (group 1 = age 1–15, group 2 = age 16-25, group 3 = age 25-54, group 4 = age 65+)
Interval/Ratio-level (also named scale or continuous; Latin for [equal] space [between numbers]). What is your age today in years? ____ (a number from 1-105)
![Page 18: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/18.jpg)
Types of Methodology Models
Theories explain phenomena,
Models represent Phenomena.
• SEM and Path models• Regression models (linear, logistic, ordinal)• ANOVA models (repeated-measures)• Time-series• Heirarchical Linear Models (HLM)• Correlational Models
![Page 19: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/19.jpg)
Relationship Among Variables
Employee Tenure
Leadership Style
EmployeeSatisfaction
Customer Satisfaction
Mediator
Moderator
![Page 20: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/20.jpg)
Data Collection Method --Overview
• Describe the procedures used to administer the materials to the participants
• Remember to be as detailed as necessary so someone can literally replicate your study
![Page 21: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/21.jpg)
Data Collection
Qualitative • Procedure for accessing participants
• Selection of data collected
• Number and duration of interviews
• How and when data is collected
• How data is recorded (e.g., hand notes, audiotaped)
• Role of researcher-relationship to researcher
Quantitative• Procedure to administer measures
![Page 22: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/22.jpg)
Pilot Test (only if you develop instrument)
• Detect potential issues in the instrument
• Allows you to get feedback and to finalize your survey/interview items
• Makes sure participants understand survey items
• Assess typical responses to survey items or interview questions (were participants comfortable, long-winded, defensive, etc.
• Can you access data?
![Page 23: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/23.jpg)
Data Analysis Plan
• Quantitative: Describe the analysis plan used to test each hypothesis, the assumptions of the statistical analyses, and a justification of the appropriateness of the analysis for each research question.
• Qualitative: Describe how the data will be analyzed (or thematized)
• Phenomenological
• Case Study
• Grounded theory
![Page 24: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/24.jpg)
Sample SizeQuantitative
Varies by type of statistical analysis
1. Research questions in statistical language2. Level of measurement of variables3. Determine statistical analysis4. Conduct power analysis
Qualitative
Varies by which qualitative approach is taken
• Phenomenological• Case Study • Grounded theory
![Page 25: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/25.jpg)
Assumptions, Limitation, Delimitations
Assumptions: Discuss things out of your control about the population and design, then justify assumptions (e.g., participants will answer honestly.)
Limitations: Are out of your control and describe weaknesses in design, threats to validity (e.g., generalizability).
Delimitations: Are in your control and relate to choices you will make to narrow the scope of the study (e.g., variables, research questions).
![Page 26: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/26.jpg)
Ethical Considerations
• Describe informed consent procedures
• State whether your study will be anonymous or confidential with respect to the participants
• Describe considerations for children or vulnerable participants
![Page 27: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/27.jpg)
See yourself graduate in 2014!
1-1 Personalized Dissertation Consulting
877-437-8600
![Page 28: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/28.jpg)
Thank you for your participation
and attention!
Join us for our next webinar on Wednesday, December 17th at 8:30pm
ET.Results and Discussion Chapters,
Editing, and Defense
![Page 29: Methodology and IRB/URR](https://reader033.vdocument.in/reader033/viewer/2022051213/55a27fc11a28ab15408b46fe/html5/thumbnails/29.jpg)
Join us for our 1-day Dissertation Workshop
Dallas/Fortworth, Texas December 6th 9am-5pm
$199/student $89 lodging/night