marketing research - questionnaire design & sampling issues

8/3/2019 Marketing Research - Questionnaire Design & Sampling Issues

1/49

Questionnaire Design

&Sampling Issues

Prof. Rushen Chahal


2/49

Role of a Questionnaire

The Questionnaire Approach

Level of Data Analysis

Research Findings

Conclusions and Recommendations

Managerial Action

Researcher Follow-up

Survey Purpose

& Objectives

Respondent

Information

Survey

TimetableCustomer

Needs

The Questionnaire Approach

Level of Data Analysis

Research Findings

Conclusions and Recommendations

Managerial Action

Researcher Follow-up

Survey Purpose

& Objectives

Respondent

Information

Survey

TimetableCustomer

Needs

Issues continually

addressed throughout

the process.

Prof. Rushen Chahal


3/49

Criteria for a Good Questionnaire

=

Provide decision-making information;

Consider the respondent;

Meet editing and coding requirements.

Prof. Rushen Chahal


4/49

Key Questionnaire Mechanics

Going through each questionnaire to ensure that

skip patterns were followed and the required

questions were filled out.

Sequence in which questions are asked, based

on a respondents answer.

The process of grouping and assigning numeric

codes to the various responses to a question.

Prof. Rushen Chahal


5/49

Determine Survey Objectives, Resources, and Constraints

Determine the Data Collection Method

Determine the Question Response Format

Decide on the Question Wording

Establish Questionnaire Flow and Layout

Evaluate the Questionnaire

Obtain Approval of all Relevant Parties

Pretest and Revise the Questionnaire

Prepare the Final Copy

Execute the Survey

The Questionnaire Design Process

Prof. Rushen Chahal


6/49

Determine Survey Objectives, Resources, and Constraints

Determine the Data Collection Method

Determine the Question Response Format

Decide on the Question Wording

Establish Questionnaire Flow and Layout

As directed by management

Shaped by time & budget

Knowledge of respondent key

Remember dos and donts

Questions should flow logically


Prof. Rushen Chahal


7/49

Evaluate the Questionnaire

Obtain Approval of all Relevant Parties

Pretest and Revise the Questionnaire

Prepare the Final Copy

Execute the Survey

For length, missing &

unnecessary questions, etc

Ensure Mgmt. buy-in

Test & revise questions

Decide on format/layout

Mail, telephone, etc.


Survey

Says

Prof. Rushen Chahal


8/49

Questions to which the respondent replies in his or her own words.

Probed Vs. Un-probed.

Questions requiring respondents to choose from a list of answers.

Dichotomous: Choice is between two answers.

Multiple Choice: Choice is among three or more options.

Scaled Responses: Designed to capture the intensity of respondents feelings.

The Response FormatOpen and Closed Questions

Prof. Rushen Chahal


9/49

Questionnaire Dos

Be as brief as is appropriate for your audience;

Be grammatically simple;

Be focused on a single issue or topic;

Use the respondents core vocabulary;

Use plenty of white space between the questions;

Number the questions;

Use consistence scales;

State instructions clearly;

Questions should be interpreted equally by respondents.

Be as brief as is appropriate for your audience;

Be grammatically simple;

Be focused on a single issue or topic;

Use the respondents core vocabulary;


Number the questions;

Use consistence scales;

State instructions clearly;

Questions should be interpreted equally by respondents.

Prof. Rushen Chahal


10/49

Questionnaire Don'ts

Biasing the respondent;

Using loaded or leading phrasing;

Using words overstating the condition;

Assuming criteria that are not obvious;

Using specific example for a general case;

Being beyond the respondent's ability to answer; Requiring the respondent to guess at a generalization;

Asking for specifics when only generalities will be remembered.

Biasing the respondent;

Using loaded or leading phrasing;

Using words overstating the condition;

Assuming criteria that are not obvious;

Using specific example for a general case;

Being beyond the respondent's ability to answer; Requiring the respondent to guess at a generalization;

Asking for specifics when only generalities will be remembered.

Prof. Rushen Chahal


11/49

Screeners - QualifyingQuestions

Ask general questions first - establish respondent buy-in

Basic questions to lay the groundwork for upcoming questions

Example: Have you shopped at Macys in the past month?

Warm-ups - FirstFew Questions:

Gets the respondent thinking about the topic at hand

Establishes parameters about the respondents attitudes, behavior, etc.Example: How often do you go shopping?

Transitions - First Third ofQuestions:

Questions that set the tone for the more difficult questions to come

Example: Now Im going to ask you some more difficult questions

Complicated - Second Third ofQuestions:

Use of rating scales for attributes, attitudes, beliefs, opinion, etc Tackling controversial issues

Example: How likely are you to go to the movies? (scale 1 to X)

Classification - Last Third ofQuestions:

Personal & demographic type questions

Example: What is your religion?

Establishing Questionnaire Flow

Build them up during the

survey process with

increasing difficult/thought

provoking questions and

conclude with more probing

questions.

Prof. Rushen Chahal


12/49

Address these issues when

designing and conducting a survey

Address these issues when

designing and conducting a survey

Incidence rate; Time and budget issues;

Purpose of the information;

Quality of information desired;

Getting a representative sample; Willingness of respondents to participate;

Availability of respondents to participate.

Questionnaire Considerations

Prof. Rushen Chahal


13/49

Start with opening letter indicating:

Who you are, why you are doing the the survey, how

long it will take, how they were selected, the surveys purpose, whether

its confidential & anonymous, thank them for participating, etc.


Ensure the format, font, layout, and appearance is consistent;

State the instructions clearly;

Clarify questions as they are asked if necessary such as:

Clarify one, pick two, etc.

Allow enough space for open-ended questions;

Ensure questions are interrelated - not only stand alone questions

Include a closing remark - Thank You , etc.

Start with opening letter indicating:

Who you are, why you are doing the the survey, how

long it will take, how they were selected, the surveys purpose, whether

its confidential & anonymous, thank them for participating, etc.


Ensure the format, font, layout, and appearance is consistent;

State the instructions clearly;

Clarify questions as they are asked if necessary such as:

Clarify one, pick two, etc.

Allow enough space for open-ended questions;

Ensure questions are interrelated - not only stand alone questions

Include a closing remark - Thank You , etc.

Questionnaire Design Issues

Prof. Rushen Chahal


14/49

Use screening questions as appropriate;

Begin with interesting questions to nab respondent;

Ask general questions first harder/more invasive ones last;

Put instructions in capital letters;

Use proper transitions throughout the questionnaire; Ensure skip patterns are in place as needed.

Use screening questions as appropriate;

Begin with interesting questions to nab respondent;

Ask general questions first harder/more invasive ones last;

Put instructions in capital letters;

Use proper transitions throughout the questionnaire; Ensure skip patterns are in place as needed.

Questionnaire Design Issues

Chapter NineProf. Rushen Chahal


15/49

The questionnaire appearance consistency is easier to achieve;

The questionnaire can be checked for typos easily;

The survey can be created quickly; Skip patterns can be efficiently established;

The survey can be distributed quickly for expert review & input.

Over reliance on electronic survey construction can lead to the researchers

getting sloppy as he might thinkthe software will do the work and correct any

errors;

The researcher might feel less connected to the process;

Multiple versions of the survey might get circulated / distributed.

The Internet Impact

Prof. Rushen Chahal


16/49

The Internet Impact

On Questionnaire Development

Email

Surveys Internet

Surveys

Cost &

Profitability Software

The Internet Impact

Prof. Rushen Chahal


17/49

Cultural differences - gender, body language, behavioral; Traditions, religion, ways of conducting business, beliefs;

Word usage differences - phrases, expressions, idioms;

Acceptable & unacceptable types of questions;

Receptive level of audience for given question types;

The best way to execute the survey telephone,mail, etc.;

Which issues are sensitive & how to approach them;

Whether various dialects are present;

What issues are most important to your audience.

Global Research Issues

Information

Prof. Rushen Chahal


18/49

Basic Sampling Issues

Prof. Rushen Chahal

Prof. Rushen Chahal


19/49

The process of obtaining information from

a subset of a larger group.

The entire group of people about whom information is

needed; Also called the universe or population of interest.

The Concept of Sampling

Prof. Rushen Chahal


20/49

Census: Data collection from or about every

member of the population of interest. Also called

canvassing the population by asking everyone a

set of questions.

Sample: A subset of all the members of a

population of interest.

The Concept of Sampling

Prof. Rushen Chahal


21/49

Geographic Area:

City, county, state(s), MSA, country, etc.

Demographics: Age, income, ethnicity / race, religion, occupation, etc.

Usage:

Frequency, purpose, rate, etc.

Awareness:

How did they become aware, why are they not aware,

what are their perceptions, attitudes, beliefs, etc.

Defining the Population of InterestSome Basis to Consider

Prof. Rushen Chahal


22/49

Developing a Sampling Plan

1. Define the Population of Interest

2. Choose the Data Collection Method

3. Identify the Sampling Frame

4. Select a Sampling Method

5. Determine Sample Size

6. Develop Operational Procedures

7. Execute the Operational Sampling Plan

Prof. Rushen Chahal


23/49

Define the Population of Interest:

Determine the characteristics of those you are

interested in studying. Determine which group of

people or entities about which you want to learn more.

Choose the Data Collection Method:

Determine how you collect the sample - such as mail,

Internet, telephone, mall intercept, etc.

Identify the Sampling Frame:

A list of population elements from which units to be

sampled can be selected or a specified procedure for

generating such a list.
















Prof. Rushen Chahal


24/49

Selecting a Sampling Method:

Determine how you will get the sample list through

probability or non-probability methods.

Probability Sampling:

Samples in which every element of the population has

a known, nonzero, likelihood of selection.

Non-probability Sampling:

Samples in which specific elements from the

population have been selected on a nonrandom manner.
















Prof. Rushen Chahal


25/49

A sample selected by assigning a number to every element of the

population and then using some method for randomly selectingelements to be in the sample such as random digit dialing.

A sample in which the entire population is numbered and elements

are selected using a skip interval every nth name is selected.

Probability Sampling

For More Information

Prof. Rushen Chahal


26/49

A sample that is forced to be more representative through simple

random sampling of mutually exclusive and exhaustive subsets

either proportionally or disproportionally. Good for data that arenot normally distributed.

A sample in which the sampling units are selecting from a

number of small geographic areas to reduce data collection costs.

Probability Sampling

Prof. Rushen Chahal


27/49

A sample based on using people who are easily

accessible - such as mall intercepts or other high

traffic locations.

A sample in which the selection criteria are based onthe researchers personal judgment about

representativeness of the population under study. The

researcher selects who should be in the study.

Non-Probability Sampling

Prof. Rushen Chahal


28/49

A sample in which additional respondents are selected

based on referrals from initial respondents.

A sample in which quotas, based on demographic orclassification factors selected by the researcher, are

established for population subgroups.

Non-Probability Sampling

Prof. Rushen Chahal


29/49

Determine the Sample Size:

Determining the sample size will be based

on factors such as the level of accuracy you

want to achieve, the time and money you

have to do the survey, and on the sampling

collection method.

Develop Operational Procedures:

Your operational plan to conduct the probability or non-probability sampling.

Determine the phases of the sample

selection process. Multi-stage sampling

involves combining sampling methods.
















Prof. Rushen Chahal


30/49

Execute the Operational Sampling Plan:

The execution phase of the research.

Administering the questionnaire - sendingthe mailers, making the phone calls,

conducting the mall intercepts, etc.
















Prof. Rushen Chahal


31/49

Error that occurs because the sample

selected is not perfectly representative ofthe population.

All error other than sampling error - also

called measurement error.

Sampling Error:

Non-Sampling Error:

Sampling and Non-Sampling Errors

Prof. Rushen Chahal


32/49

X = + / - E ES NS+ / -

Where:

X = sample mean

= true population mean

= sampling error

= non-sampling, or measurement, error

E

S

E SN

Where:

XX = sample mean

= true population mean

= sampling error

= non-sampling, or measurement, error

E

S

E SN

Sampling and Non-Sampling Errors

Prof. Rushen Chahal


33/49

Targeted respondents can complete the survey at their convenience;

The data collection is relatively inexpensive;

Survey software can facilitate the data collection process;

The survey can be completed quickly;

The sample might not be representative of the population;

You can not always be sure who is completing the survey; Maintaining respondent confidentially can be a challenge;

Data security issues can be difficult to manage.

Internet Sampling

Prof. Rushen Chahal


34/49

Sample Size

Determination

Prof. Rushen Chahal


35/49

Chapter Eleven ObjectivesChapter Eleven Objectives

To learn the financial and statistical issues in the determination of the sample size.

To discover methods for determining the sample size.

To gain an appreciation of a normal distribution of data.

To understand population, sample, and sampling distributions.

To distinguish between point and interval estimates.

To recognize problems involving sampling means and proportions.

Prof. Rushen Chahal


36/49

Sample Size for Probability Samples

Census:

Population canvas - not really a sample

Asking the entire population

Judgment:

Best guess of experts

Draw on your experience to determine sample size

Conventional:

What have others done?

See what the sample size has been for similar studies

Prof. Rushen Chahal


37/49

Arbitrary / Rule ofThumb:

Applies some industry accepted rule of thumb

Generally better for smaller populations

Picking x percent of the population to be in the sample

Budget Available:

What can we afford?

How much do we want to spend?

How much time are we allotting for each respondent

Statistical:

Variance, SD, confidence interval play a key role

Sample Size for Probability Samples

Prof. Rushen Chahal


38/49

Central Limit Theorem: The idea that a distribution of a large number of sample means or

sample proportions will approximate a normal distribution - regardless

of the distribution of the population from which they were drawn.

Normal Distribution: The continuous distribution that is bell shaped and symmetrical

about the mean. The mean, median, and mode are equal. About 68%of the observations are within +/- one standard deviation, 96% are

within two standard deviations, and 99+% are within three standard

deviations of the mean.

The Normal Distribution

Prof. Rushen Chahal


39/49

Proportionate Properties:

A feature that the number of observations falling between

the mean and a given number of standard deviations from

the mean is the same for all normal distributions.

Standard Normal Distribution:

Normal distribution with a mean of zero and a standard

deviation of one.


Prof. Rushen Chahal


40/49

Standard Deviation:

The measure of dispersion calculated by subtracting the mean of the

series from each value in a series, squaring each result, summing the

results, dividing the sum by the number of observations minus 1, and

taking the square root of this value.

StandardDeviation

(N-1)

(X - X)12

= sumStandard

Deviation(N-1)

(X - X)12

= sum


Prof. Rushen Chahal


41/49

Population Distributions:

The frequency distribution of all the elements of a population.

Sampling Distributions:

The frequency distribution of all the elements of an individual sample.

Population and Sampling Distributions

Prof. Rushen Chahal


42/49

Sampling Distribution of the Mean:

The theoretical frequency distribution of the means of all

possible samples of a given size drawn from a particularpopulation; it is normally distributed.

Standard Error of the Mean:

Standard deviation of a distribution of sample means.

Sampling Distribution of the Mean

Prof. Rushen Chahal


43/49

Point Estimate:

The particular estimate of a population value.

Interval Estimate:

The interval or range of values within which the

true population value is estimated to fall.

Point and Interval Estimates

Prof. Rushen Chahal


44/49

Confidence Level:

The probability that a particular interval will include the true

population value - also called the confidence coefficient.

C

onfidence Interval:

The interval that, at the specified confidence level, includes the

true population value.

Point and Interval Estimates

Prof. Rushen Chahal


45/49

Sampling Distribution of the Proportion:

The relative frequency distribution of the sample proportions of

many random samples of a given size drawn from a particular

population; It is normally distributed.

Sampling Distribution of the Proportion

Prof. Rushen Chahal


46/49

N = Sample Size

Z = level of confidence desired in the results. A 95% confidence

interval would make Z=1.96. In other words, if we conduct

this survey 100 times, at least 95 of those time the true population

average would fall within out interval estimate.

= Population standard deviation.

e = error rate - a management decision (ex. plus or minus 3%)

N = Sample Size


interval would make Z=1.96. In other words, if we conduct

this survey 100 times, at least 95 of those time the true population

average would fall within out interval estimate.

= Population standard deviation.


Problems

Involving

Means:

Determining Sample Size

o

2

oN =

2

2e

Z*

Prof. Rushen Chahal


47/49

Problems

Involving

Proportions:

2

2

N = Sample Size


interval would make Z=1.96. In other words, we would by

95% confident that the average results in the whole population

(were we to survey the whole population) would be within 1.96standard deviations from the mean.

p = variance (how different you predict the population is), q = 100-p


N = Sample Size


interval would make Z=1.96. In other words, we would by

95% confident that the average results in the whole population

(were we to survey the whole population) would be within 1.96standard deviations from the mean.

p = variance (how different you predict the population is), q = 100-p



Prof. Rushen Chahal


48/49

To revise down your original sample size whilemaintaining the same level of accuracy. This technique

is good for small populations.

RSS

(revised sample size)=

N

(original sample size) *Population - Original Sample Size

Population - 1


Prof. Rushen Chahal


49/49

You need to over sample since its unlikely that everyone you contact will agree to

answer the questionnaire. If you determine that you need 800 respondents in thesurvey (RSS), and a typical response rate for your type of survey is 30%, then use

the calculations to below to get the number of people you would actually have to

contact to get 800 completed responses.

O = RSS/.30

orO = 800/.3

O = 2,667

Thus your new sample size is 2,667 - with 30%

response you can expect 800 people to respond.

O = RSS/.30

or

O = 800/.3O = 2,667

Thus your new sample size is 2,667 - with 30%

response you can expect 800 people to respond.


Prof. Rushen Chahal

marketing research - questionnaire design & sampling issues

Documents