BUA 500: Introduction to Research Methods

Research design and methodology

Class will cover…

Aspects and issues in designing a research project

Issues and techniques in library research

Introductory issues in statistics and statistical methodologies

Survey Process

Information gathering and needs assessment

Sampling design

Instrumentation

Data Collection

Data Processing

Report Generation

Information gathering and needs assessment

Specify the information needsEstimate the information valueAssess internal resource availabilityChoose the data collection methodSelect the data processing methodMeasure the scope of the response taskDetermine a sample size and designSpecify the instrumentation and costsDescribe the type of reports requiredSummarize final costs and the timetable

Alternative Data Collection Methods

Mail Survey

Telephone Survey

Personal Interview

Experiments

Comparison of Data Collection Methods

Personal Telephone Mail

Data collection costs: High Medium Low

Time required: Medium Low High

Sample size needed: Small Medium Large

Data quantity: High Medium Low

Dispersed sample: No Maybe Yes

Reaches special locations: Yes Maybe No

Interactions with respondents: Yes Yes No

Degree of interview bias: High Medium None

Severity of non-response bias: Low Low High

Presentation of visual stimuli: Yes No Maybe

Field worker training required: Yes Yes No

Interviews versus Mail surveys

Does the task require interaction with the respondents?Is there a possibility of interaction respondent bias?Are there accurate mailing databases available?Must the data be collected at a specific location?How involved is the data recording process?Large amount of data from limited subjects or small amount of data from large pool of subjects?How widely dispersed are the subjects?Would there be embarrassment or threatening issues in face to face communication?

Telephone versus In-Person

Specific location required?Is there a “show and tell” component to the interviewing process?Are you using some type of quota based survey? E.g. is there some type of screening process required?Would appearances self-select some types of respondents undesirably?Is intimidation of any issue?Will the interview take a long time?Are there substitutive database to pull from?How rapid of a turnaround time is needed?

Factors determining sampling strategy—indications of large sample, small response task

Very precise numerical estimates are neededRange of volume of information is limitedLeaning toward a mail data collection approachOverall survey of information is of more interest than in-depth patterns by respondents

Controlling sampling error and bias

Any sample less than the population will have sampling errorSampling error is random error affecting both validity and reliabilityThe greater the sampling error, the lower the reliability and validityThe less the variation among individual members of the population, the small the sampling errorSurvey data cannot be more valid than it is reliable, because reliability limits validitySurvey data can be high reliable, but still be low in validity because of systematic biasLike sampling error, systematic bias can never be eliminated completelySystematic bias can best be controlled by identifying the things that are most likely to exert bias and eliminating them or protecting from them in the sampling design

Instrumentation bias issues

Does the question state the criterion for answering?Is the question applicable to all respondents?Does the item contain an example that is also a possible answer?Does the question require respondents to remember too much detail or recall distant events?Is the question as specific as it can reasonably be?Does the question overemphasize some condition?Are some of the words in the item ambiguous?Is the question as free from threat to respondents as possible?IMPORTANT: Does the question only include one issue?Will yea or nay sayers always chose one answer?Does the question lead respondents toward a particular answer?Is the question “loaded” with a reason for responding in a particular way?

Sources of response bias

Social desirability

Acquiescence

Yea or nay-saying tendency

Prestige

Threat

Hostility

Mental sets

Order

Avoiding sample selection bias

Are some sampling units more visible or accessible than others?Is there any systematic order to the sampling units?Are units clustered together?Will field workers have more affinity for some units over others?Is there a self-selection component to be concerned with?Will there be a large number of non-respondents?

Factors indicating a large sample

Decision is very serious and/or costlyVery high level of confidence neededHigh level of variance within the populationSmall sub samples are needed during analysis and interpretationCosts vary only slightly to sample size selectionTime and resources are readily available

Special sampling designs

Stratified or proportional sampling Divides population into two or more segments and

sample a different proportion of each E.g. opinions on a topic with respect to gender, race,

religious beliefs, party affiliation, etc

Cluster sampling Sample cluster of respondents, e.g. random city blocks

Quota sampling Sample a population and meet quota’s based on some

strata or delineation condition

Guide list for measuring Attitudes

Be sure to include all three components of the attitude: knowledge, feelings, and action tendencies, in that orderUse unaided recall to measure awareness, if possible, to avoid false reports of recognitionUse ratings scales to measure feelings, so that both direction and distance from neutral are revealedDo not ignore the intensity of feelings or assume intensity is the same as distance from neutral

Focus issues on question design

Wrong: Which brand do you like the best?Right: Which of these brands are you most likely to purchase?Wrong: When do you usually go to work?Right: What time do you ordinarily leave home for work?Wrong: Are you going to vote Democrat or Republican?Right: Which candidate will you vote for on election day?

Brevity of the question

Wrong: Can you tell me how many children you have, whether they are girls or boys, and how old they are?Right: What is the age and sex of each of your children?Wrong: If you own one or more automobiles, please list the year and the make of each one, starting with the newest one?Right: Please list the year and make or each car you own?Wrong: When was the last time that you went to the doctor for a physical examination on your own or because you had to?Right: How many months ago was your last physical examination?

Clarity of the question

Wrong: What do you have to say about the charities that your church contributes to?Right: How much influence do you, yourself have on which charities your church contributes to?Wrong: About how much of the storage space in your home do you and your spouse use?Right: What proportion of the storage space in your home is used for your things and what is used for your spouses?Wrong: Ordinarily, do you take aspirin when you feel some discomfort or when you feel actual pain?Right: Do you usually take aspirin as soon as you feel some discomfort, or only when you feel actual pain?

Question construction guidelines

Does the question focus directly on the issue or topic to be measured?

Is the question stated as briefly as it can be?

Is the question expressed as clearly and simply as it can be?

Use of vocabulary

Wrong: Are you cognizant of all the concepts to be elucidated?Right: Do you know about all the ideas that will be explained?Wrong: With what frequency have you experienced the of late?Right: How many times have you had this happen lately?Wrong: What emotions were evoked by perceiving the spectacle?Right: What kinds of feelings did you have when you saw it?

Use of compound sentences

Wrong: What would you do when you had only a few things to buy and there were a lot of people in the checkout line?Right: Suppose you have only a few things to buy. There are a lot of people in the checkout line. What would you do?Wrong: How do you work it out when you want on thing and your spouse wants another and you both feel very strongly about it?Right: How do you settle disagreements with your spouse when you both have strong feelings about it?Wrong: If you didn’t have a reservation ahead of time and you found out that the only seats that you could get were at the very top of the upper balcony, what would you do?Right: What would you do if the only seats available at show time were on the top of the upper balcony?

Expressing questions correctly

Use only core vocabulary; the words and phrases that one would use in casual speech.

Limit the vocabulary so that the least sophisticated respondent would be familiar with the words.

Use simple sentences where possible, and complex sentences only when they are actually required.

Use two or more short, simple sentences rather than one compound or compound-complex sentence.

Change long, potentially confusing clauses in sentences to words or short phrases where possible.

Unstated criteria issues

Wrong: How important is it for stores to carry a large number of different brands of this product?

Right: How important is it to you that the store you shop at carries a large number of different brands?

Inapplicable question

Wrong: how long does it take for you to find a parking place after you arrive at the plant?

Right: If you drive to work, how long does it take you to find a parking place after you arrive at the plant?

Examples in questions

Wrong: What small appliances, such as counter-top appliances, have you purchased in the past month?

Right: Aside from major appliances, what other smaller appliances have you bought in the past month?

Overdemanding recall

Wrong: How many times did you go out on a date with your spouse before you were married?

Right: How many months were you “dating” your spouse before you were married?

Overgeneralizations

Wrong: When you buy “fast food”, what percentage of the time do you order each of the following types of foods?

Right: Of the last 10 times you bought “fast food”, how many times did you each each type of food?

Over specificity

Wrong: When you visited the museum, how many times did you read the plaques that explain what the exhibit contains?

Right: When you visited the museum, how often did you read the plaques that explain what the exhibit contained? Would you say often, sometimes, rarely, or never?

Overemphasis

Wrong: Would you favor increasing taxes to cope with the current fiscal crisis?

Right: Would you favor increasing taxes to cope with the current fiscal problem?

Ambiguous wording

Wrong: About what time do you ordinarily eat dinner?

Right: About what time do you ordinarily dine in the evening?

Double-barreled questions

Wrong: Do you regularly take vitamins to avoid getting sick?

Right: Do you regularly take vitamins? Why are why not?

Leading questions

Wrong: Don’t you see some danger in the new policy?

Right: Do you see any danger in the new policy?

Loaded questions

Wrong: Do you advocate a lower speed limit to save human lives?

Right: Does traffic safety require a lower speed limit?

Types of question scales

Likert scale

Should be used to decipher differences on a number of itemsShould be able to compose statements that are opinionsMust be reasonably certain that the majority of respondents will not pick only the neutral categoryIf a summated score is to be computed, about half the items should be inclined to the pro side and half to the negative side to avoid yea and nay sayer bias.

Please pick a number from the scale to show how much you agree or disagree with each statement and jot it to the right of the item.

Scale 1=Strongly agree 2= Agree 3 = Neutral 4 = Disagree 5 = Strongly disagree

A man should never cry in public………………………………____

Higher education is more important for women than men……………………….______

Women should receive equal pay for equal work…………………………….._____

A man should not resent a women supervising his work…………………_____

A women’s place is in the home?…………………………..____

The husband should make the majority of the family decisions?………………______

Women should never put career before family?………………………….____

A man should help and protect a women in public……………………._____

Verbal frequency scale

Used for several items, not just one or twoCompose items that assume the opportunity to perform the actionShould be used only when approximate percentages are requiredMost appropriate when respondents are unable or unwilling to compute exact percentages

Please pick a number from the scale to show how often you do each of the things listed below and jot in the space to the right of the item.

Scale 1=Always 2=Often 3=Sometimes 4=Seldom 5=Never

Seek out information about candidates and issues……………………_____Actually vote during strictly local elections………………………_____Actually vote during a state or national election……………._____Vote along strict party lines………………………._____

Ordinal scale

Used when a direct question concerning a quantity or value would not be sufficient

The researcher should be aware of the statistical limitations of ordinal versus interval or ratio data

Response alternative must be positioned in a meaningful sequence

Important: the interval between categories must be of little or no interest

Ordinarily, when do you or someone in your family first turn on a television set in your home on a weekday? (Please check one)

___The first thing in the morning___ A little while after awakening___ Mid-morning___ Just before lunch___ Right after lunch___ Mid-afternoon___ Early evening, before dinner___ Right after dinner___ Late evening___ Usually don’t turn it on

Forced ranking scale

Number of things to be ranked should be less than 10The major focus should be on the relative standing of the itemsThe researcher must be willing to forego measurement of the distance between ranksA single judgment criterion must be clearly statedAs with other scales yielding ordinal data, analysis is confined to a limited set of statistical procedures that do not require equal intervals

Please rank the colas listed below in their order of preference. Jot the number 1 next to the one you prefer the most, number 2 by your second choice, and so forth.

___ Pepsi-Cola

___ Coca-Cola

___ Royal Crown Cola

___ Diet Rite Cola

Paired comparison scale

Number of things to compare should be < 10

The method is most effective when actual choices in the real situation are always between two things

The researcher must be willing to forego measurement of the ‘distance’ between items in each pair

A single judgment criterion must be clearly stated

For each pair of soft drinks listed below, please put a check mark by the one you most prefer, if you had to choose between the two

1 ___ Pepsi-Cola ___ Coca-Cola

2 ___ Royal Crown Cola ___ Pepsi-Cola

3 ___ Royal Crown Cola ___ Diet Rite Cola

4 ___ Royal Crown Cola ___ Coca-Cola

Comparative scale

The major emphasis should be the comparison of a single, standard entity with one or more othersThe actual rating scale should have an even number of alternatives, if ‘fence-riding’ is likelyThe researcher must be sure the respondent is very clear about which is the standard and which is to be ratedThis method is advisable only if all or nearly all respondents are very familiar with the standardThe methods is particularly applicable when interval data are desired, but a relative measure is required. BOTH RANKING and DISTANCE between entities are generated.

Compared to the previous management group, the new one is … (check one space)

Very Superior About the Same Very Inferior

_____ _____ ______ _____ _____

1 2 3 4 5

Horizontal numeric scale

The method is most applicable when evaluative responses are to be arrayed on a single dimensionThe scale is most economical where several items are all to be rated on the same dimensionScale extremes should be labeled, “Extremely _____” to define the dimension and the words used must be bipolar oppositesIn the vast majority of cases, the intermediate scale values should not be labeled with words, and only numbers should be used, spaced at equal intervals

How important to you is each of the public issues listed below

Extremely Extremely

Unimportant Important

1 2 3 4 5

The protection of endangered species of animals………………………………..___

The improvement of the quality of the air…………………………………………..___

The discovery of additional petroleum reserves…………………………………..___

The development of “renewable” sources of energy…………………………….___

The reduction or elimination of water pollution……………………………………___

The development of additional nuclear power…………………………………….___

The provision of social services to those in need…………………………………___

Semantic differential scale

This method is most effective for measuring image profile, people, things, organizations, or conceptsAdjectives must define a single dimension and each pair must be bi-polar opposites labeling the extremesPrecisely what the respondent is to rate must be clearly stated in the introductory instructionsNo more than about 20 items should be used, and about half should begin with a positive word

Please put a check mark in the space on each line below to show your opinion of the pizza served here

Hot ___ : ___ : ___ : ___ : ___ : ___ : ___ Cold

1 2 3 4 5 6 7

Bland ___ : ___ : ___ : ___ : ___ : ___ : ___ Spicy

1 2 3 4 5 6 7

Expensive ___ : ___ : ___ : ___ : ___ : ___ : ___ Inexpensive

1 2 3 4 5 6 7

Small ___ : ___ : ___ : ___ : ___ : ___ : ___ Large

1 2 3 4 5 6 7

Good ___ : ___ : ___ : ___ : ___ : ___ : ___ Bad

1 2 3 4 5 6 7

Adjective check list

Method is appropriate for measuring imagesInstructions and response task are quick and simple, and a large number of adjectives could be applicablePrecisely what the respondent is to rate or judge must be stated very clearly in the insturctionsThe scale yields a profile, but only in terms of discrete, nominal data indicating a dichotomous choice

Please put a check mark in the space in front of any word or phrase that describes your job

___ Easy ___ Safe

___ Technical ___ Exhausting

___ Boring ___ Difficult

___ Interesting ___ Rewarding

___ Low-Paying ___ Secure

___ Important ___ Degrading

Stapel scale

This method is appropriate for measuring images

A relatively large number of words or phrases can be inlcuded

Precisely what the respondent is to judge and rate must be stated very clearly in the instructions

The scale yields a profile indicating “how much” each phrase describes the topic, with continuous data

Please pick a number from the scale to show how well each word or phrase below describes your job and jot it in the space in from of each item

Not all all 1 2 3 4 5 6 7 Prefectly

___ Easy ___ Safe

___ Technical ___ Exhausting

___ Boring ___ Difficult

___ Interesting ___ Rewarding

___ Low-Paying ___ Secure

___ Important ___ Degrading

Fixed sum scale

The scale is used to measure proportions, rather than absolute valuesThe occurrences must be apportioned into not more than about ten categoriesThe instructions must state very clearly the value to which the responses must totalThe scale provides continuous data so that proportions can easily be compared among alternativesThe data can be converted to percentages by dividing each value by the actual total, so every case will total a hundred percent and be comparable

Of the last 10 times that you ate lunch or dinner at a casual or fast food restaurant, how many times did you have each of the things listed below? (please be sure to make the total 10)

___ Hamburgers___ Hot dogs or sausage___ Chicken___ Pizza___ Chinese food___ Fish or seafood___ Deli sandwiches___ Hot sandwiches___ Mexican food___ other (What?___________________)Total = 10

Scale Creation Criteria

Keep it simpleRespect the respondentDimension the responsePick the denominationsDecide on the rangeGroup only when requiredHandle neutrality carefullyState instructions carefullyAlways be flexiblePilot test the scales

Ethical issues in research

Honesty

Don’t over consume one’s time

Respect confidentiality

Integrity

Be aware to possible ramifications

Basic person-to-person consideration