Week 3Primary Data Collection:

Survey ResearchObservation MethodExperimental Research

By Dr. Muhammad Ramzanmramzaninfo@gmail.com, 03004487844

Edited by Ahsan Khan Ecoahsankhaneco@yahoo.com03008046243

Survey Research• To collect primary data, specifically for a

project in hand.

• Surveys provide a quick, inexpensive, efficient and accurate means of assessing information about a population

• Identifying characteristics of a target market, measuring customer attitude and describing consumer purchase patterns etc.

3

Survey Method• The survey method is based on the questioning of

respondents regarding their behaviour, intentions, attitudes, awareness, motivations, demographic and lifestyle characteristics. These questions may be verbally, in writing, or via computer and responses may be obtained in any of these forms.

• Typically the questioning is structured and a sample of respondents are asked to select from a pre-determined set of alternatives.

• Example: DisagreeAgree

Shopping at malls is fun 1 2 3 4 5

4

Types of Surveys• Structured versus unstructured

– structured-formal standardised questions used– unstructured -informal, no standardised questions– degree of structure influences choice of media

• Disguised versus undisguised– disguise - the concealing of purpose or sponsorship of

a study until completion– undisguised - respondent is aware of purpose and

sponsor of research

5

Advantages of Survey Methods• accommodate large sample sizes; increases

generalisability of results• standardisation – all respondents react to questions

worded identically; response options/scales are same• administrative ease – much simpler than a focus group/

in-depth interview; development of questionnaire is a more complex process than the administration

• ability of tapping into factors & relationships that are not directly observable ie attitudes, feelings, preferences

• tabulation and statistical analysis of data is easy• subgroup differences can be easily determined

6

Disadvantage of Survey Methods

• difficulty of developing accurate survey instruments (questionnaire design)

• limits to the in-depth detail of data

• lack of control over timeliness, & potential low response rates

• difficulties to determine if respondents are responding truthfully

• misinterpretation of data results & inappropriate use of data analysis procedures

A Classification of Survey Methods-Mode of Administration

Traditional Telephone

Computer-Assisted Telephone

Interviewing

Mail Interview

Mail Panel

In-Home Mall Intercept

Computer-Assisted Personal Interviewing

E-mail Internet

Survey Methods

Telephone Personal Mail Electronic

8

Mail Methods

• Can be conducted through mailing questionnaires to pre-selected respondents via ordinary mail or mail panel. It includes outgoing envelope, cover letter, questionnaire, return envelope, the purpose, assurance of anonymity, an appeal for completion and return within due date.

• There is no verbal interaction between researcher and the respondent

• Before stat of mail survey, respondents need to be broadly identified, a valid mailing list, closely related to population of interest is required

Mail Survey Advantages & Disadvantages–no interviewer bias, can respond at leisure–high geographic flexibility, more confidential information–cost confined to mailing list, forms and postage–follow up easy but time consuming

–low response rate, high cost per response, long time delays, no probing/observation; can read entire questionnaire before answering which may influence response; higher possibility misunderstand/clarification

–never sure that the target respondent actually answered the questions

–complex questions not responded to well; open-ended questions do not achieve lengthy written responses

Increasing Response Rates for Mail Surveys

• Effective cover letter

• Authorities help/reference letter

• Incentives helps

• Interesting questions

• Use of peers/colleagues/professional association

• Follow-ups

• Advanced notification

• Survey sponsorship

• Keying questionnaires

Personal Methods• Personal In-home Interviews; respondents are interviewed

face-to-face in their homes. Interviewer contact respondents ask questions and record responses

• Mall-Intercept: Respondents are intercepted while they are shopping in malls/superstores and brought to test facilities in the malls. Researcher administers a questionnaire. Methods is more efficient and becoming popular

• Computer Assisted Personal Interviewing: Respondent sits in front of a computer terminal and answer questions on the computer screen using mouse and keyboard. User-friendly electronic packages are available to design easily understandable questions. Interviewer remains available as a host to guide the respondent

12

Personal Methods Advantages & Disadvantages

–persuasion/cooperation; better response rate; use observation

–can use visual material; longer questionnaires

–Can assist to interviewee; more precise selection of sample

–more versatile questioning; extended probing possible

–if self-completed, respondent can complete when convenient

–travel time & expenses high, people reluctant to talk to strangers

–interviewers presence may bias, inhibited, difficult to supervise and control; difficult to recruit capable interviewers; geographic flexibility limited

–call back/follow up difficult; no anonymity of interviewees

–Self-completed – disadvantages similar to mail

Telephone Methods• Traditional Telephone Interview involves phoning

and asking a series of questions to respondents. Researcher uses paper questionnaire and records responses with a pencil

• Computer-Assisted Telephone Interviewing: Researcher wears a mini-hand set, reads questions from compute screen and records responses directly in the computer memory. In this method computer replaces pencil and paper questionnaire

Telephone Method Advantages & Disadvantages

– dialling efficient especially call backs; travel avoided

– respondent does not have to open door to strangers

– geographic coverage good; supervision and training excellent; speed of data collection

– less interviewer bias; greater anonymity of respondents

– no observation; visuals excluded; retaining attention more difficult therefore shorter questionnaires; only homes with telephones; cost can be a factor with STD calls

– there is limited interview times pertaining to the best times for making and gaining responses

– Interviewer is unable to control the interview; the respondent can hang up at any time

Electronic Methods• Email Interviews require list of email addresses.

Survey is written within the body of email message. Respondents type answers to close ended/open ended questions at designated places and reply through email. It requires data entry.

• Internet Interviews are posted on a website using HTML. Respondents need to open the website and complete the survey. Respondents are sent links usually through email. (Survey Monkey, Google Doc Forms)

16

Electronic Methods Advantages & Disadvantages

– speed of response/ data collection– easy of development and testing– little cost of administration– easy to administer

– the possibility of data corruption via virus transmission– possible unreliability of e-lists– are respondents representative of the population– possibility of bogus replies

Criteria Phone In-Home Interviews

Mall-Intercept

Interviews

CAPI Mail Surveys

Mail Panels

E-Mail InternetFlexibility of data collection

Moderate to high

High High Moderate to high

Low Low Low Moderate to high

Diversity of questions Low High High High Moderate Moderate Moderate Moderate to high

Use of physical stimuli Low Moderate to high

High High Moderate Moderate Low Moderate

Sample control Moderate to high

Potentially high

Moderate Moderate Low Moderate to high

Low Low to moderate

Control of data collection environment

Moderate Moderate to high

High High Low Low Low Low

Control of field force Moderate Low Moderate Moderate High High High High Quantity of data Low High Moderate Moderate Moderate High Moderate Moderate Response rate Moderate High High High Low Moderate Low Very

Low Perceived anonymity of the respondent

Moderate Low Low Low High High Moderate High

Social desirability Moderate High High Moderate to High

Low Low Moderate Low

Obtaining sensitive information

High Low Low Low to moderate

High Moderate to High

Moderate High

Potential for interviewer bias

Moderate High High Low None None None None

Speed High Moderate Moderate to high

Moderate to high

Low Low to moderate

High Very high

Cost Moderate High Moderate to high

Moderate to high

Low Low to moderate

Low Low

A Comparative Evaluation of Survey Methods

18

Factors determining choice of survey method• Researcher’s resources & objectives

– Time horizon– Budget – Desired quality of data

• Required level of generalization• Completeness & precision

• Respondent characteristics– Willingness to participate– Ability to participate– Diversity of respondents

• Type of questions asked– Complexity – Topic sensitivity– Amount of information required per respondent

There is no best form of survey; each has advantages

and disadvantages.

Selected Questions to Determine the Appropriate Technique

• Is the assistance of an interviewer necessary?

• Are respondents interested in the issues being investigated?

• Will cooperation be easily attained?

• How quickly is the information needed?

• Will the study require a long and complex questionnaire?

• How much is the budget?

Observation Method• It involves recording the behavioral pattern of people,

objects, and events in a systematic manner to obtain information about a phenomenon of interest.

• The observer does not question or communicate with the people being observed.

• Information obtained may be structured or unstructured, direct or indirect and may be conducted in a natural or contrived environment

Structured vs Unstructured Observation 

• For structured observation, the researcher specifies in detail what is to be observed and how the measurements are to be recorded, e.g., an auditor performing inventory analysis in a store.

• In unstructured observation, the observer monitors all aspects of the phenomenon that seem relevant to the problem at hand, e.g., observing children playing with new toys.

Disguised vs Undisguised Observation

• In disguised observation, the respondents are unaware that they are being observed. Disguise may be accomplished by using one-way mirrors, hidden cameras, or unnoticeable mechanical devices. Observers may be disguised as shoppers or sales clerks.

• In undisguised observation, the respondents are aware that they are under observation.

Natural vs Contrived Observation• Natural observation involves observing

behavior as it takes places in the environment. For example, one could observe the behavior of respondents eating fast food in Burger King.

• In contrived (artificial) observation, respondents' behavior is observed in an artificial environment, such as a test kitchen.

A Classification of Observation Methods

Observation Methods

Personal Observation

Mechanical Observation

Trace Analysis

Content Analysis

Audit

Personal Observation

• A researcher observes actual behavior as it occurs.

• The observer does not attempt to manipulate the phenomenon being observed but merely records what takes place.

• For example, a researcher might record traffic counts and observe traffic flows in a department store.

Mechanical Observation Mechanical devices rather than human observers record the

phenomenon.Do not require respondents' direct participation.– Audimeter (attached to a television to continuously record what

channel that set is tuned. People meters attempted to measure the channel to which TV is tuned but also who is watching)

– Turnstiles records number of people entering or leaving a building.– On-site cameras (still, motion picture, or video) – Optical scanners in supermarketsDo require respondent involvement. – eye-tracking monitors; psychogalvanometers (skin response)– voice pitch analyzers (emotional reactions through changes in

respondents voice– devices measuring response latency (amount of time it takes to

respond

Observation Methods: Audit• The researcher collects data by examining physical

records or performing inventory analysis. • Data are collected personally by the researcher. • The data are based upon counts, usually of physical

objects. • Retail and wholesale audits conducted by marketing

research suppliers

Observation Methods: Content Analysis

• The objective, systematic, and quantitative description of the manifest content of a communication.

• The unit of analysis may be words, characters (individuals or objects), themes (propositions), space and time measures (length or duration of the message), or topics (subject of the message).

• Analytical categories for classifying the units are developed and the communication is broken down according to prescribed rules.

Observation Methods: Trace AnalysisData collection is based on physical traces, or evidence, of past

behavior.

• The number of different fingerprints on a page was used to gauge the readership of various advertisements in a magazine.

• The position of the radio dials in cars brought in for service was used to estimate share of listening audience of various radio stations.

• The age and condition of cars in a parking lot were used to assess the affluence of customers.

• The magazines people donated to charity were used to determine people's favorite magazines.

• Internet visitors leave traces which can be analyzed to examine browsing and usage behavior by using cookies.

A Comparative Evaluation of Observation Methods

Criteria Personal Mechanical Audit Content Trace Observation Observation Analysis Analysis Analysis

Degree of structure Low Low to high High High MediumDegree of disguise Medium Low to high Low High High Ability to observe High Low to high High Medium Low in natural setting Observation bias High Low Low Medium MediumAnalysis Bias High Low to Low Low Medium

Medium General remarks Most Can be Expensive Limited to Method of

flexible intrusive commu- last resortnications

Relative Advantages of Observation

• They permit measurement of actual behavior rather than reports of intended or preferred behavior.

• • There is no reporting bias, and potential bias caused

by the interviewer and the interviewing process is eliminated or reduced.

• Certain types of data can be collected only by observation.

• If the observed phenomenon occurs frequently or is of short duration, observational methods may be cheaper and faster than survey methods.

Relative Disadvantages of Observation • The reasons for the observed behavior may not be

determined since little is known about the underlying motives, beliefs, attitudes, and preferences.

• Selective perception (bias in the researcher's perception) can bias the data.

• Observational data are often time-consuming and expensive, and it is difficult to observe certain forms of behavior.

• In some cases, the use of observational methods may be unethical, as in observing people without their knowledge or consent.

It is best to view observation as a complement to survey methods, rather than as being in competition with them.

Experiment Method• A research investigation in which conditions

are controlled so that causal relationship among variables is determined

• One or more independent variable are manipulated while holding all other possible independent variables to observe the effects on dependent variable

• IVs effect on a dependent variable is measured

• Used to test a hypothesis

Use of experimental research

• Experimentation is commonly used to infer causal relationship

• Causality is the relationship between an event (the cause) and a second event (the effect), where the second event is understood as a consequence of the first– A. the relationship of cause and effect – B. the principle that nothing can happen

without being caused – C. causal agency or quality

Basic issues of experimental design

• Manipulation of the independent variable

• Selection of dependent variable

• Assignment of subjects (or other test units)

• Control over extraneous variables

• The experimenter has some degree of control over the independent variable.

• The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be

Definitions and Concepts• Independent variables are variables or alternatives that

are manipulated and whose effects are measured and compared, e.g., price levels.

• Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g., consumers or stores.

• Dependent variables are the variables which measure the effect of the independent variables on the test units, e.g., sales, profits, and market shares.

• Extraneous variables are all variables other than the independent variables that affect the response of the test units, e.g., store size, store location, and competitive effort.

Independent Variable (IV)

• The experimenter controls independent variable.

• The variable’s value can be manipulated by the experimenters to whatever they wish it to be.

Dependent Variable (DV)

• Its value is expected to be dependent on the experimenter’s manipulation

• Criterion or standard by which the results are judged

• Selection– e.g... sales volume, awareness, recall,

• Measurement

• DVs are the outcome of interest to the researcher and the decision maker

Example

• Hypothesis: Changes in price influences sales

• Price here is IV and sales DV and other possible variables are extraneous variables

Example• Influence of brand name (IV) on consumers taste(DV)

perceptions

• Experimenter manipulated whether consumers preferred drinks of labeled or unlabelled bottles

• One week respondents were given drink a pack of bottles with label ABC. Next week respondents received another pack of drinks with brand labels (Pepsi, Fanta, Shezan etc)

• Researcher measured reactions to drinks after each tasting as the drink was the same.

• Therefore the differences observed in taste (DV) could only be attributed to the difference in labeling (IV).

• Participants preferred for branded drinks, so the conclusion (inference) is that brand name does influence consumers taste perception

How may an experimenter control forextraneous variation?

• Eliminate extraneous variables

• Hold conditions constant

• Randomization

• Matching subjects

Experimental Design

An experimental design is a set of procedures specifying:

the test units and how these units are to be divided into homogeneous (all the same) subsamples,

what independent variables or treatments are to be manipulated,

what dependent variables are to be measured; and

how the extraneous variables are to be controlled.

Validity in ExperimentationA researcher has to draw valid conclusions of effects of

independent variables (internal validity) on the study group (sample) and make valid generalization (external validity) on larger population of interest

• Internal validity refers to whether the manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity.

• External validity refers to whether the cause-and-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?

Extraneous variables• History refers to specific events that are external to the

experiment but occur at the same time as the experiment.

• Maturation (MA) refers to changes in the test units (sample) themselves that occur with the passage of time.

• Testing effects are caused by the process of experimentation. Typically, these are the effects on the experiment of taking a measure on the dependent variable before and after the presentation of the treatment.

• The main testing effect (MT) occurs when a prior observation affects a latter observation.

Extraneous Variables

• In the interactive testing effect (IT), a prior measurement affects the test unit's response to the independent variable.

• Instrumentation (I) refers to changes in the measuring instrument, in the observers or in the scores themselves.

• Statistical regression effects (SR) occur when test units with extreme scores move closer to the average score during the course of the experiment.

• Selection bias (SB) refers to the improper assignment of test units to treatment conditions.

• Mortality (MO) refers to the loss of test units while the experiment is in progress.

Controlling extraneous variables• Randomization refers to the random assignment of test

units to experimental groups by using random numbers. Treatment conditions are also randomly assigned to experimental groups.

• Matching involves comparing test units on a set of key background variables before assigning them to the treatment conditions.

• Statistical control involves measuring the extraneous variables and adjusting for their effects through statistical analysis.

• Design control involves the use of experiments designed to control specific extraneous variables.

Laboratory VS Field experiments

• A laboratory environment is an artificial one, which the researcher constructs with the desired conditions specific to the experiment. An artificial environment of experimentation in which the researcher constructs the desired conditions. To measure the effectiveness of a test commercial could be conducted in a laboratory environment by showing the test commercial embedded in a TV program to respondents in a test theater.

• Field environment is an experiential location set in actual market conditions. Such as running test commercials on actual TV stations.

Laboratory vs field experiments

Factor Laboratory Field

Environment Artificial RealisticControl High Low Reactive Error High Low Demand Artifacts High Low Internal Validity High LowExternal Validity Low HighTime Short LongNumber of Units Small LargeEase of Implementation High Low Cost Low High

Laboratory experiment Field experiment

Artificial-Low Realism

Few ExtraneousVariables

High control

Low Cost

Short Duration

Subjects Aware ofParticipation

Natural-High Realism

Many ExtraneousVariables

Low control

High Cost

Long Duration

Subjects Unaware ofParticipation

Limitations of experimentation• Experiments can be time consuming, particularly if the

researcher is interested in measuring the long-term effects.

• Experiments are often expensive. The requirements of experimental group, control group, and multiple measurements significantly add to the cost of research.

• Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment.

• Competitors may deliberately contaminate the results of a field experiment.

Test marketing• It is an application of a controlled experiment done in

limited but carefully selected test market. It involves a replication of the planned national marketing program for a product in the test markets. Often market mix (independent) variables are varied in test marketing and the sales (dependent variable) are monitored so that an appropriate national marketing strategy can be identified

• To test market acceptance of the product• Test alternative levels of marketing mix variables.

Criteria for the selection of test markets1) Be large enough to produce meaningful projections. They

should contain at least 2% of the potential actual population.

2) Be representative demographically and with respect to product consumption behavior.

3) Be representative with respect to media usage and with respect to competition.

4) Be relatively isolated in terms of media and physical distribution.

5) Have normal historical development in the product class.

6) Have marketing research and auditing services available.

7) Not be over-tested.

Summary• The scientific notion of causality implies that we can never

prove that X causes Y

• We can only infer that X is one of the causes of Y in that it makes the occurrence of Y probable's.

• Three conditions must be made (I) concomitant, which implies that X and Y must vary together in a hypothesized way (II) time order of occurrence of variables, which implies that X must precede Y and (III) elimination of other possible causal factors. Experiments provide the most convincing evidence of these conditions.

• An experiment is formed when one or more independent variables are manipulated by the researcher and their effect on one or more dependent variables is measured.

Conclusion

There is no best data collection technique; each has its advantages and disadvantages.

The suitability and appropriateness of data collection technique/method depends on the purpose, scope of the research and researchers resources.

THANK YOUBy Dr. Muhammad Ramzanmramzaninfo@gmail.com, 03004487844

Edited by Ahsan Khan Ecoahsankhaneco@yahoo.com03008046243