Research Support Services

Online Survey Development: Tips and Pitfalls

Research Support Services

Online Survey Development: Tips and Pitfalls

Jennifer McGugan

Research Support Specialist

mcgugj@mcmaster.ca

October 24, 2012

Surveys Many study designs use surveys to collect information

Online or web-based surveys are an increasingly more popular way to collect survey information

3

“Web surveys make feasible the delivery of multimedia survey content to respondents in a standardized way using self-administered methods. This opens up a whole new realm of survey possibilities that were heretofore impossible or extremely difficult to implement using more traditional methods.” (Couper, 2000)

4

Purpose Discuss tips and pitfalls of survey development with an online focus

Not instruction on technical aspects of online survey software

5

Before You Begin Clearly identify your research question

Is a questionnaire or survey the best way to answer your question?

Are there any existing tools or questionnaires that you can use to answer your research question?

• Valid• Reliable• Justify why a new questionnaire needs to be developed

6

Survey Development

“The design of a questionnaire must consider how to motivate someone who receives it to become a respondent and complete the survey.” (Dillman, 2009)

7

Design Create the most appealing appearance for respondents to complete your online questionnaire

Carefully consider the questionnaire design

8

Tips and Pitfalls Create informative welcome and closing screens written in a friendly, professional manner

Welcome Screen:- title, purpose, instructions, participant rights , study

contact information and consent

Closing Screen:- message to the respondent regarding completion of

the survey and acknowledgement of respondent’s contributions

9

We are inviting you to participate in this survey, led by Jane Smith, PhD, at McMaster University, as well as a follow-up survey six months from now. All adult members of the McMaster Fitness Centre have been invited to participate. Individuals who complete both the evaluation survey and the follow-up survey will be entered into a draw to win an ipad as a token of our appreciation. The survey questions will take less than 10 minutes to complete. The intent of this evaluation is to determine if, and the extent to which you use the exercise facility and how that contributes to your health.

All responses will be kept confidential and anonymous. Anonymity of data will be maintained so that any reports produced from the study will not provide identifying information. This information will be kept in the project coordinator's office with electronic files on a computer that is password protected, so that only research staff will have access to this data. There are no foreseen risks in participating in this study. If you choose not to participate in this study there will not be any negative consequences. You may also choose to withdraw from the survey at any time, or choose not to answer specific questions.   The results of the study can be made available to you, or you may request notification of any publications produced as a result of this study by contacting the research coordinator. If you have any questions or concerns please contact Jennifer McGugan at mcgugj@mcmaster.ca or 905-525-9140 ext 23220.   

This study has been reviewed by the Hamilton Health Sciences/McMaster Faculty of Health Sciences Research Ethics Board (HHS/FHS REB). The REB is responsible for ensuring that participants are informed of the risks associated with the research, and that participants are free to decide if participation is right for them. If you have any questions about your rights as a research participant, please call The Office of the Chair, HHS/FHS REB at 905.525.2100 x. 42013.   I have read the preceding information thoroughly.

By clicking the submit button, I agree to participate in this study.

EXAMPLE:

MCMASTER FITNESS CENTRE EVALUATION

10

Your answers have been received.

Thank you for taking the time to complete this survey. We value your contribution.

Sincerely,

Jane Smith

EXAMPLE:

THANK YOU!

11

Introduce your question groups with a description to inform participants about the next steps.

EXAMPLE:

MCMASTER FITNESS CENTRE EVALUTION

The next group of questions ask about your exercise habits. Please answer the questions while keeping in mind your regular weekly exercise habits.

12

Order questions logically and group by content

EXAMPLE : 1. What exercise equipment do you use? Select all that apply. Treadmill Exercise Bike Elliptical Machine Rowing Machine Free Weights Weight Machines Other Please Specify:2. How many times per week do you use the McMaster Fitness Centre?

# of times3. What is your occupation? Yes No 4. Have you ever had surgery? Yes No

13

Questions should apply to your respondents

EXAMPLE:

5. If you regularly use the treadmill, how long do you spend on it?

minutes

Specify the concept

EXAMPLE:

6. How many times did you exercise?

Number

14

Ask one question at a time

EXAMPLE:

7. Do you have and use a step tracker? Yes No

Use complete sentences with as few words as possible and keep them simple and familiar

EXAMPLE:

8. Do you feel like you have more doggedness after using the MUFC more than 3 times per week? Yes No

15

Create mutually exclusive response categories

EXAMPLE:

9. Please select the age category applies to you: 18-3030-4545-6060 and above

16

Have a purpose for open-ended questions

EXAMPLE:

10. How do you feel about exercise?

17

Use mandatory questions only if necessary

Put more sensitive questions toward the end of the survey

Include instructions to ensure respondents know how to answer your questions

Number questions consecutively

See Dillman (and others) for many other things that should and should not be done when writing questions and designing a questionnaire .

18

Online Survey Software Survey Monkey

Fluid Surveys

Checkbox

Survey Crafter

LIME SURVEY – Used by McMaster University

19

Question Formats

20

Single choice questions – List (radio)

21

Arrays – 5 point choice

22

Mask questions - Date

23

Multiple numerical input

24

Text questions

25

Multiple choice

26

Question editing

27

Survey procedures Email invite to participate in survey

tokens vs. self-identified ids date stamps IP address

Reminder email(s) depending on study protocol Frequency

Survey open and close dates Duration

28

Research Support ServicesSchool of Nursing Research Support Services can assist you with your online survey on a fee-for-service basis

Getting Started Complete a request form on our website Set up a contract with Susanne King kings@mcmaster.ca

29

Research Support ServicesQUESTIONNAIRE Your questionnaire in a complete form with all descriptions and introductions Welcome, closing and email messages, if applicable Are you going to be pilot testing your questionnaire?

Content validity Readability

If you are using an existing questionnaire, have you obtained the necessary permissions?

30

Research Support ServicesPARTICIPANTS Complete contact list of participants or list serv email addresses If you have a participant gift or prize how are you distributing it? What is your timeline? Reminders, close date, etc. Are there follow up surveys that need to be matched

Tokens vs. Self-identified IDs

31

Research Support ServicesDATA Do the data need to be anonymized? Will you need to do any recoding? In which format would you like to receive your data?

Excel SPSS

Couper, M. (2000) WEB SURVEYSA REVIEW OF ISSUES AND APPROACHESPublic Opinion Quarterly Volume 64:464–494.

Dillman, D. (2009) Internet, Mail and Mix-mode Surveys: The Tailored Design Method

Thank you

Questions?