1 survey research ps 1006 lecture 1 sam cromie. 2 research methods observational research...
Post on 17-Dec-2015
220 Views
Preview:
TRANSCRIPT
2
Research Methods
• Observational research– Unstructured– Structured– Participatory
• Experiments– Group design
• Within-subject• Between subject
– Single-subject design
• Surveys• Qualitative research
– Interviews– Focus groups– Document analysis– Etc.
• Case studies
Survey research
Writing good questions
Data Collection Methods
Research Designs
SamplingMethods
Essential Reading:
Research Methods in Psychology (5th or 6th edition)
Shaughnessy, Zechmeister & Zechmeister
Surveys: Advantage and Disadvantages
• Positives– Easy to sample a
large number of people
– Context likely to retains ecological validity
– Relatively easy to administer
• Negatives– Danger of biased
surveys– People may not
answer truthfully– No true IVs - Cannot
infer causality
5
Types of question
• Open-ended - ‘How do you feel today?’
– Allows wide range or responses - preferable when researcher has little idea of the type or response to expect
– Respondents decide how much information to give
– Lack of structure can make scoring of the responses difficult
6
Types of question
• Closed questions - ‘On a scale of 1-5 how happy do you feel today?’
– A fixed set of responses is provided– Respondent must pick one– Easy to score– May miss responses that some participants
wants to give since the scope for answers is limited
7
Types of question
• Mixed questions– A fixed set of responses is provided– An open-ended choice is given to cover any
alternatives that may have been missed– Still has a limited scope
How do you feel today?A. HappyB. SadC. AngryD. RelaxedE. Other
(specify)_________________________________
Response scales
• Categorical questions – non-directional optionsHow are you happy today?
• Likert Scales - Labeled points on scaleI am Happy today
• PercentagesHow happy are you on a percentage from 1-100
• Continuous scalesPlease mark the line below to express you level of happiness:
Strongly Agree
Agree Disagree Strongly Disagree
Happy Unhappy
Happy Angry Sad Tired
Categorical Questions
• Often used for demographic information:– Job type, ethnic background, etc.
• Dichotomous questions – only two options: Yes/no, Male Female
– No indication of response strength
• Can be used as quasi IVs in data analysis
• Can be analysed with other categorical questions using Chi Squares
10
Likert Scales
• Labeled points on scale• Several possibilities
– Strongly agree, Agree, Disagree, Strongly disagree
– Scale from 1-5 with ends (poles) labeled
– Useful to try and define parts of the scale (anchoring)
• How many values should be put on a scale?– 7±2 options - above this
difficult to use the full range
• When normally distributed can be analysed parametrically
11
Percentages
• Gives participants a wider range of possible responses
• But resposes tend to cluster around commonly used percentages – 10, 20, 25, 50, 75, 90
Continuous scalesHappy Unhappy
• Line is normally 10cm long• Score = distance in mm from start of line
to mark• Get a fuller use of the scale than
percentages• Allows more sensitive measurement than
Likert• More challenging to code
Writing good questions
• A survey lives and dies by its questions– Unclear questions confuse
• Vocabulary should be clear and simple
– Questions should be specific• Should government do something about health
care?• Should government pass a law on health care?• Should government pass a law on health care that
would guarantee health coverage for all?
Writing good questions
• Include conditional information before the key idea in the question– If you were the victim of a violent crime, would
you consider buying a gun?
• Consider varying the polarity of the questions– Questions should be phrased both negatively
and positively• Some people just like to say No (or Yes)!
• Leading questions - suggesting the ‘right’ answer– People often recognize the importance of gun control in
the prevention of crime. What is your opinion of gun control?
• Loaded questions - emotionally charged– Are you in favor of laws that that allow dangerous
convicted felons out of prison early?
• Double-barreled questions - packing too much in – Do you favor life sentences for repeat offenders and
stricter sentences for criminals who used guns?
Avoid when writing Qs
Basic sampling terms
• Population - all members of defined group– Census - ask entire population– Survey - ask representative sample
• Sampling frame - specific list of all the members of the population - not always accurate – e.g., register of currently enrolled students - what
about students who are abroad or taking year out?
• Element - a member of the population
17
Basic sampling terms
• Representativeness - – dictates how generalisable the results are. Clearly,
biased samples = bad.
• Selection bias – – procedures used to select sample which result in non-
representative sample
18
Sampling methods
• Probability sampling - each element has same probability of being included– Simple random sampling– Stratified random sampling - population
divided into strata and random samples are drawn from each e.g, make sure ethnic balance reflects population
• Most sampling is not really random
Sampling methods
• Nonprobability sampling - each element has different probability of being chosen– Accidental sampling - selection on basis of
availability and willingness to respond e.g., television polls. People may participate:
• For money• Because they are interested in topic• Because they have a particular view point
– Purposive sampling - elements included on the basis of their special characteristics
Potential sampling problems
• Flawed surveys often reflect poor samples– Sampling bias can enter in two places
• At selection of sampling frame– Use of phone directories for general population
– List of subscribers to a magazine
• At selection of a sample– Use of mail survey
– Calls made to college students on Friday and Saturday nights
• Any study in which subjects self-select is potentially problematic
21
Survey research designs
• Cross-sectional designs – view of population at one time period only
Time
Example
• Suppose we conduct a survey of the number of left-handers in the population
• Of inds below 60, left-handers approx = 10%• Of Inds above 60, left-handers are much less than 10%
– Conclusion: left-handers die younger– Is this valid
• Need to ensure that timing does not produce a major confound
Successive independent samples
Time
• Can be used to assess change over time– Must ensure that samples are comparable
• Any change detected should be due to time and not sample differences
Longitudinal studies
Time
• Follow a single group of individuals over time– Requires a lot of time and effort– Subject drop out a problem
• Sometimes mortality• Most often people drop out for some reason• Must ensure that there is not a systematic reason for
drop out
Data collection
• There are several ways to conduct surveys– Through the post– In person– Over the phone– Via internet
• Each of these methods has its own advantages and disadvantages
Postal surveys
• Advantages– Minimizes effort needed to
collect data– Good for personal or
embarrassing topics• Respondent’s replies are
anonymous
• May cut down on the amount of socially desirable responding
• Disadvantages – May not be filled out be
intended participant– Respondents cannot ask
for clarification– No follow-up questions can
be asked– No control over the order in
which completed– Big potential for response
bias
Self-selection - postal surveys• Respondents in postal surveys always self-
select• Controlling response bias in postal surveys
– Try to ensure a high return rate• Over 50% is considered very good
• Response bias is less severe with return rates over 50%
– Practical steps to improve response rate• Include a SAE
• Personalize the introduction (at least with name)
• Do follow-up mailings with new copies of the survey to those who do not respond
Personal interviews
• Advantages– High response rate (often over
80%, though less in high-crime areas)
– Can control order of responding
– Control over context of responding
– Ensure the correct participant answers question
– Respondent can ask for clarification
– Interviewer can ask follow-up questions
• Disadvantages– There is a potential for
interviewer bias• May suggest the desired
response (non-verbally)
– People may give socially desirable responses
• They may not want to express their true beliefs to the interviewer
– Difficult to ask about embarrassing topics
• No anonymity for the respondent
Telephone surveys
• Advantages– Better than personal interviews in high-crime
areas• People do answer the phone!
– More anonymity than for personal interviews– Can control order of responding– Respondent can ask for clarification– Interviewer can ask follow-up questions
Telephone surveys - disadvantages
• People may not want to be bothered over the phone– Particularly now that telemarketers often use surveys
as a sales pitch
• No control over surroundings– Respondent may be doing a number of things at once
• There is the potential for interviewer bias– May suggest the desired response through non-
semantic content of dialogue
• People may provide socially acceptable responses– They may not want to express their true beliefs to the
interviewer
Internet surveys
• More and more widely used
• Data can be imported directly into analysis package
• Quick and efficient• Cuts down on paper use• Excellent for international
surveys• Can use audio-visual
content, etc.
• Lack of control of context in which it is filled out
• Identity of participant• Respondents cannot ask
for clarification• No follow-up questions
can be asked• No control over the order
in which completed• Biased against the less
computer literate• Inappropriate for many
populations
top related