Use of Surveys in Extension for Program Development & EvaluationSarah Baughman, Ph.D. Mike Lambur, Ph.D.

eXtension Virginia Cooperative Extension

Benefits & Challenges Quick Cost effective Simplifies data analysis Capabilities/functionality Multiple devices

Intended audience must have internet access

Easily sent to spam folder Technical difficulties Literacy Easy to develop “bad”

surveys

Ask what you need to know, not what is interesting to

know

Developing Sound QuestionsSurvey questions should be:

ClearShortUnbiasedRelevant

Clear QuestionsTarget the vocabulary and grammar to the population surveyed What was your AGI last year?

Avoid ambiguity, confusion, and vagueness What is your income? Do you jog regularly? Does your company have pension and health

insurance benefits? How many years have you been in the program? 1-2 years 2-3 years 3-5 years

Short QuestionsTry to keep questions to 20 words or less

Long questions can be intimidatingTravel to other countries has become

increasingly popular recently. Have you ever traveled to another country? If yes, you might have traveled to other countries to enjoy the scenery. How important was the scenery in deciding to take the trip?

Unbiased QuestionsBiased questions contain words or phrases that

lead the respondent to a particular answerDo you think the new cafeteria provides better

food than the old cafeteria?Most doctors say that cigarette smoke causes

lung disease for those near a smoker. Do you agree?

Wouldn’t you like to receive our free brochure?

Relevant Questions

Consider the respondents’ interest in and ability to answer the questions

Most people do not want to appear uninformed, so rather than admit to having little or no knowledge about a question, they are likely to select or provide a response

Types of QuestionsFactual

What respondents’ knowAttitudinal

How respondents’ feelBehavior

What respondents’ doDemographic

Who respondents’ are

Question Response Formats Close-ended responses

Mutiple choice Must be mutually exclusive Must be exhaustive

Rating scales Respondents’ indicate the degree of response to the

question Ranking Respondents’ indicate, in order, their preference among

a list of responses

Question Response FormatsOpen-ended responses

Allow respondents’ to answer in their own wordsUnless carefully focused and worded,

respondents’ may provide minimal information

Other ConsiderationsLong questionnaires get less response than short questionnaires

Provide a well written cover letterGive your questionnaire a title that is short and meaningful to the respondent

Place the most important items in the first part of the questionnaire

Pay attention to the flow of questionsTry to keep similar questions/topics togetherUse page breaks Make it convenient to respond

So What?

Now what?

Qualtrics Informal Learning June 26 – Qualtrics Basics July 10 – Panels & Distributing July 17 – Reporting Ongoing discussion, tips, etc on the Google+ Online Surveys

community

2PM Eastern Time for all sessions

References Websites

http://www.statpac.com/surveys http://www.analytictech.com/mb313/principl.htm http://www.qualtrics.com/university/researchsuite/

Hardcopy Dillman, Don. (2007). Mail and internet surveys: The tailored

design method. John Wiley & Sons, Inc: Hoboken, NJ.