sociology 300 spring 2012 survey research. the practice of research in criminology and criminal...

Sociology 300

Spring 2012

Survey Research

The Practice of Research in Criminology and Criminal Justice

Chapter 7: Survey Research

Basic Terms in Survey Research

Survey Research Collection of information from sample of individuals

through their responses to questions Includes all procedures and materials (e.g., questionnaires)

Questionnaire Survey instrument containing the questions in a self-

administered survey Interview schedule

Survey instrument containing the questions asked by the interviewer in an in-person or phone survey

Respondent Person who answers questions on a survey

Response Rate Percentage of persons surveyed who complete a survey Item response rate – percentage of items that are

completed on a survey



Basic Types of Questions

Open-ended Allow respondent to fill

in own answer

May ask for specific information or

May ask for narrative response Respondent says/writes

whatever he/she wants

Closed-ended Respondent chooses from

list of possible responses May also be called

Fixed choice questions Forced choice questions



Examples of Basic Types of Questions

Closed-Ended

What is your annual income? Check the appropriate box:

no income

Less than $5000

$5001 - $20,000

More than $20,000

Open-Ended

What is your date of birth? ___ / ___ / ______

mm / dd / yyyy

Describe the most difficult thing about being a single, working parent

____________________

____________________

12 14 1980

Finding good day care



Advantages/Disadvantages of Each Type of Question

Closed-Ended Quick response Consistency in type of

response Easier to analyze

statistically

Open-Ended Detailed responses No preconceived

content of answer Must go through

several steps before ready for analysis



Basic Principles of Writing Survey Questions

Write Clear and Meaningful Questions Avoid Confusing Phrasing Minimize the Risk of Bias Avoid Making Either Disagreement or

Agreement Disagreeable Maximize the Utility of Response Categories

Minimize Fence-Sitting and Floating



Avoid Confusing Phrasing and Jargon

Use correct grammar Use shorter rather than longer words and

sentences Break complex ideas into several questions and

address each component separately A question (like a variable) should be about only 1 thing

BUT do not abbreviate questions in a way that results in confusion

Avoid vagueness Ask questions in reference to specific times or events

whenever possible



Which question is better?

“In what city or town do you live?”

OROR

Residential location: _____________________

“Do you support “shall issue” carry permit laws?”



double negatives: “Do you disagree that there should not be a death penalty?”

double-barreled questions and answers

More Confusing Phrasing to Avoid

“Do you think the prison system should stop releasing inmates for weekend furloughs and concentrate on rehabilitating criminals?”

“Do you know anyone who has used cocaine?” ___ I have used cocaine

___ I have not used cocaine



Minimize the Risk of Bias

Biased or “loaded” words and phrases tend to produce misleading answers

“Should U.S. troops be sent into action if a situation like Vietnam were to develop in another part of the world?”“Should U.S. troops be sent into action to stop a communist takeover?”

Do these questions have the same

meaning?



Avoid Making Either Disagreementor Agreement Disagreeable

Which question are people more likely to answer honestly?

“Have you ever taken anything from a store without paying for it?”

OROR

“Have you ever shoplifted something from a store?”

Social Desirability



Maximize the Utility of Response Categories

Questions with fixed response choices must provide one and only one possible response for everyone who is asked the question—that is, the response choices must be exhaustive and mutually exclusive. Ranges of ages, incomes, years of schooling, and so forth should not overlap and should provide a response option for all respondents.



Mutually Exclusive and Exhaustive Response Categories

How many times have you been arrested? 0

1-23-56-1011 or more



Minimize Fence-Sitting and Floating

Fence-sitters: people who see themselves as being neutral, may skew the results if you force them to choose between opposites

Floaters: respondents who choose a substantive answer when they really don’t know the answer



Effect of Floaters on Public Opinion Polls

Are government leaders smart?

No Opinion

Smart

Not Smart

Not given “No Opinion” choice

Not Smart

No Opinion

Smart

Given “No Opinion” choice



Likert “Questions”

Usually framed as declarative statements – not questions

React to statement by identifying intensity of feeling/belief/opinion

Usually thought of as “agreement” but can be any reaction from strongest to weakest With or without an explicitly measured neutral point in

the center

1 2 3 4 5Strongly Agree No Disagree Strongly Agree Opinion Disagree

1 2 3 4Strongly Agree Disagree Strongly Agree Disagree



Example of Likert-Type Responses

I think that “three-strikes” laws that increase penalties for individuals convicted of three or more felonies will help to decrease the crime rate.1 2 3 4

Strongly Agree Disagree Strongly Agree Disagree

Make sure that your statement and response categories “go together”To what degree do you support “three-strikes” laws that increase penalties for individuals convicted of three or more felonies ?

1 2 3 4Strongly Agree Disagree Strongly Agree Disagree

1 2 3 4Strongly Support Do not Support Strongly Support Do not Support



Filter Questions and Skip Patterns

Use when concept measured by a question or group of questions applies to only some respondents Reduces total time required to complete questionnaire Reduces respondent burden Components

Filter question question asked to establish whether concept is

applicable Skip pattern

Questions skipped by respondents to whom they are not applicable

Contingent questions Question or set of questions answered only by

respondents to whom they are applicable Clearly label questions to be skipped so that respondents

can easily follow the questionnaire



Example of Filter Question and Skip Pattern

4. Have you been found guilty of a disciplinary infraction while at this prison?

Yes, go to question 4a

No, go to question 5

4a. What was the most recent infraction?

5. Have you ever attempted to escape fromthis prison……?

Disobeying a direct order

Contraband

Assault

Theft

Malingering

Other

Filter questio

n

Contingent question

Next question, asked of everyone



Combining Questions into an Index

Use when several questions measure one concept

Sum (or take the average of) responses on each question to create a composite measure Score indicates degree to which concept is present

More complete measure of concept than a single question

If questions measure one concept, then answers should be consistent across people Use reliability measures to determine consistency

Not all lists of questions can be combined into an index



Considerations in Creating Indexes

Assumption that each question measures same concept may be mistaken

Combining responses to specific questions can obscure important differences in meaning

Questions may cluster together in subsets Sometimes particular questions are counted or

weighted more that others in calculation of the index



Composite Measure Example

“Attitude Toward Police” I think police officers are generally fair to all people regardless of their race or ethnicity.____ Strongly Agree ____ Agree ____ Disagree ____ Strongly Disagree

Police officers are given too much freedom to stop and frisk community residents.____ Strongly Agree ____ Agree ____ Disagree ____ Strongly Disagree

I think if someone resisted arrest, even a little, most police officers would become assaultive if they thought they could get away with it.____ Strongly Agree ____ Agree ____ Disagree ____ Strongly Disagree

Police officers put their lives on the line every day trying to make it safe for residents of this community.____ Strongly Agree ____ Agree ____ Disagree ____ Strongly Disagree

1 2 3 4

4 3 2 1



Demographic Questions

Basic description of members of a population

Typically includes Age Sex or gender Race or ethnicity Education Income Religion

May also include Employment status Occupation Region of residence Population or type of

place of residence (e.g., urban, rural, suburban)

Do not ask demographic questions that are not relevant to your studyWhenever possible, use standard categories



Essential step in preparing any survey Type and extent of pretesting depends on

Complexity of questions Resources (time, money, staff) Characteristics of respondents Type of survey design

Pretesting the Survey

Start by simply discussing the questionnaire content with other researchers



Organization of Questionnaire

Create a title that is descriptive of overall topic Make sure question order does not influence responses Create major topic divisions to help respondents

understand organization of questionnaire Use instructions throughout questionnaire to minimize

respondent confusion Questionnaire should look attractive, be easy to

complete, and have open space Use numbers to designate response choices, for ease of

coding and data entry Create introductory material that will enhance response

rate Cover letter Introductory statement read by interviewers Should establish credibility of researcher



The Five Basic Types of Surveys

Mail Group Administered Telephone In-Person Electronic

These are both Self-Administered surveys and will be discussed

together



Self-Administered

Group Mail Self Electronic

Email Internet Computer Assisted Self

Interview (CASI) Interactive Voice

Response (IVR)

Can also be administered individually

Respondents fill in answers on their own

Most economical way to survey large number of people, especially if population is geographically spread out

In electronic surveys, may not know representativeness of sample



Response Rate

Always aim for 100%, but rarely achieve it Minimum acceptable generally considered to be

around 70%

Different types of surveys have different response rate characteristics

Techniques exist to maximize response rate Many of the guidelines for survey design are

intended to increase response rate



Telephone Surveys

Interviewers question respondents over the phone and record answers Often use Computer Assisted Telephone

Interviewing (CATI) Very popular because almost all households

have phones … but what about cell phones?

Random digit dialing (RDD) Machine calls phone numbers randomly within

designated area codes and exchanges, regardless of whether the numbers are published



In-Person Interviews

Types Manual or Computer Assisted Personal Interview

(CAPI) Survey may be called “interview schedule”

Have better quality data than mail surveys Interviewer can clarify unclear items Interviewer can “probe” for open-ended responses

Usually requires extensive training



Electronic Surveys

Email Sent as messages to respondent’s email address

Internet-based Designed on a server controlled by researcher Can be longer than email surveys and have

inapplicable questions hidden from respondent’s view Flexible format Interactive Voice Response (IVR)

Respondent receives automated phone call from ‘computer’ and answers questions by pressing numbers on phone keypad or speaking answers

Best suited to specific populations known to have email and internet access (e.g., college students)

sociology 300 spring 2012 survey research. the practice of research in criminology and criminal...

Documents

criminal justice chapter

surveythe practice of

survey researchbasic

survey researchwhich

answersmore confusing

specific information

specific times

type of responseeasier