04/11/2023 1

Questionnaire• A questionnaire is a self-report instrument designed to gather

‘valid’ and ‘reliable’ information• Questions may be structured (specified) or unstructured

(open or unspecified)• The use of questionnaires as data-gathering research

instruments is based on 3 key assumptions:1. Respondents can read and understand items2. Respondents possess the necessary information to answer the items3. Respondents are willing to answer the items honestly

Principles of questionnaire design 1: The basics• Select only those potential items (variables) that are

consistent with: (1) specific and theoretically-driven research questions, (2) operationally-defined constructs, AND (3) the target sample or population

• Limit questions/items to only those of major interest• Item development is crucial and should be based on

extensive:(a) interviewing (b) piloting (c) pre-testing

04/11/2023 2

Principles of questionnaire design 2: Constraints• Time:

- Not more than 30 minutes for adults (about 15 mins for mail questionnaire)- Not more than 10-15 minutes for children and adolescents

• Sensitivity or delicacy of content• Design• Avoid ‘non-response’ bias

- The attributes of a well-designed questionnaire do not arise naturally out of the process of questionnaire construction, but are the result of careful developmental work

Principles of questionnaire design 3• Decide major research question(s) variables and their possible

multiple indicators• Decide key demographic and classificatory variables [e.g., age,

sex, education, occupation, residential (or work) location, etc.]• Design response categories to avoid potential ‘non-response

bias’

04/11/2023 3

Principles of questionnaire design 4• Use simple nomenclature, direct statements/questions• Avoid, at all costs, double- or multiple-barreled questions• Wherever possible, avoid the use of negatively-worded

questions/items; i.e., state all items in the POSITIVE

Principles of questionnaire design 5• TRYOUT potential items with respondents drawn from the

target sample/population• Revise wording and ordering of items• Pay particular attention to LAYOUT• Conduct an ‘empirical’ TRIAL of the instrument and be

prepared to revise the instrument drastically• ADMINISTER the revised questionnaire

04/11/2023 4

Principles 2

SURVEY INSTRUMENT LENGTHOne outcome of a good pretest is to find out

how long it takes to complete a survey instrument … The federal Office of Management and Budget has set as a guideline that surveys should take less than half an hour unless there is a compelling reason why more information is needed. There are many academic surveys, however, that last an hour or longer. (Fowler, 1993, p. 103)

04/11/2023 5

Principles 4

Avoid also the use of negative statements if possible (surprisingly confusing), and double-barrelled questions (or two questions in one). Making questions easy to answer involves avoiding hypothetical situations, jargon, technical language and ambiguity. (Gorard, 2001, p. 95)

04/11/2023 6

Principles 5Ordering of itemsBefore writing actual questions it is useful

to consider the overall design of your questionnaire instrument. Perhaps the most crucial here is the order in which items will appear. This applies to the order of the questions in each section, and the order of each section within the whole. (Gorard, 2001, pp. 87-88)

04/11/2023 7

Organisation of the questionnaireUse of multiple questionsIt is common interviewing technique to use several questions to

meet a single objective.The sequence of questions within a topicThe funnel sequence of questionsThe term refers to a procedure of asking the most general or

unrestricted questions in an area first, and following it with successively more restricted questions. In this way the content is gradually narrowed to the precise objectives:1. How do you think this country is getting along in its relations with other countries?2. How do you think we are doing in our relations with Russia?3. Do you think we ought to be dealing with Russia differently from the way we are now?4. (if yes) What should we be doing differently?5. Some people say we should get tougher with Russia, and others think we are too tough as it is. How do feel about it?

04/11/2023 8

Inverted funnelThere are times when it is desirable to invert the sequence and

start with the specific questions, concluding by asking the respondent the most general question.

The inverted sequence is especially appropriate for topics in which the respondent is without strong feelings or on which he has not previously formulated a point of view.

Attitude scalesThere are, of course, a number of other patterns by which the

ordering of questions may be prescribed. Among the various methods is the attitude scale, in its many forms. Each of the various methods of scaling specifies or implies its own criteria for ordering questions. The techniques of Thurstone, Guttman, Likert, and Coombs are relevant examples.

Organisation of topicsThe sequence of topics is not haphazard or random; it is an

organised, systematic progression

04/11/2023 9

Labeled response categories

Sheatsley (1983, as cited in Maxim, 1999, p. 222) itemizes some of the more popular. Among these are:

• excellent – good – fair – poor • approve – disapprove; favour – oppose; good idea – bad

idea• agree – disagree• too many – not enough – about right; too much – too little

– about right amount• better – worse – about the same• very – fairly – not al all• regularly – often – seldom – never• always – most of the time – some of the time – rarely or

never• more likely – less likely – no difference

04/11/2023 10

Likert scales

The origin of attitude scaling can be identified with the work of Thurstone and Likert that appeared in the late 1920s and early 1930s (Thurstone, 1928; Likert, 1932).

- Thurstone’s binary: generally yes/no

- Rensis Likert: continues to be one of the most commonly used scale applications; the scale is simple to use; it seems to work well in a wide range of circumstances; and it appears to be fairly robust.

(Maxim, 1999, pp. 223-224)

1 = Strongly disagree

2 = Disagree to some extent

3 = Uncertain, neither agree nor disagree

4 = Agree to some extent

5 = Strongly agree

04/11/2023 11

Likert’s primary concern: unidimensionality (all the items measure the same thing)

Probably 100 respondents: sufficient for most purposes (but 250 or 300: not unusual)

Decision: a high score to mean a favourable or an unfavourable attitude consistent from then on

Adding up the items scores to obtain a total score:- Pool: 132 items- Possible range of total scores: from 132 to 660 (5 x 132) for each subject

Table: a scale for mothers, dealing with acceptance or rejection of children