PSYB01 Final Exam Notes: chapters 6-10

Chapter 6 - Observational Methods

Quantitative and Qualitative Approaches Observational methods can be broadly classified as primarily quantitative or qualitative.

Qualitative research focuses on people behaving in natural settings and describing their world in their own ways.

Quantitative research tends to focus on specific behaviors that can be easily quantified.

Naturalistic Observation Sometimes called field work or simply field observation.

Researchers makes observations in a particular natural setting (the field) over an extended period of time, using a variety

of techniques to collect info.

Reports includes observations and researchers interpretations of findings

Used to study many phenomena in all types of social and organizational settings (social sciences).

Has roots in anthropology and animal behavior.

A researcher uses natural observation when he/she wants to describe and understand how people in a social or cultural

setting live, work and experience the setting.

Description and Interpretation of Data

Natural observation demands that researchers immerse themselves in the situation.

Field researcher observes everything - the setting, the patterns of personal relationships, peoples reactions to events, and

so on.

Researchers needs to keep detailed field notes

Goal is to provide a complete/accurate picture rather than to test hypotheses formed prior to study.

Techniques:

observe people and events

interview key informants to provide inside info

talking to people about their lives

examining documents produced int he setting (newspapers, newsletters, or memos)

audio and videotape recordings

Natural observation = primarily qualitative, however could still be quantitative

Researcher must generate hypotheses that explain data and make it understandable = interpret

Analysis is done by building a coherent structure to describe the observations

Good natural observation report will support analysis by using multiple confirmations

Issues in Naturalistic Observation

Participation and Concealment

Is the researcher a participant or non-participant?

Will researcher conceal purposes from other people in the setting or not?

Qualitative Quantitative

- in depth info on few individuals or within a

very limited setting

- conclusions based on interpretations drawn by

investigator

- ex: focus groups (themes that arise) = non-

numerical

- larger samples

- conclusions based upon statistical analysis

- ex: surveys (numerical values assigned to

responses)

Potential problem with participating is losing objectivity necessary to conduct scientific observation. (especially if initially

a part of a group being observed).

Concealed observation may be preferable because presence of observer may influence and alter behavior of those being

observed = less reactive.

Still non-concealed observation may be preferable from an ethical viewpoint.

People often become used to observer and act naturally (ex: The Real World show)

The decision about concealment depends on both ethical concerns and nature of particular group/setting being studied.

There are degrees of participation and concealment = therefore researcher must carefully determine what their role in the

setting will be.

When anonymity is NOT threatened = exempt research = no informed consent necessary.

In non-concealed observation = IC verbal or written

Must be sensitive to ethical issues; whether observations are made in a public place with no clear expectations that

behaviors are private.

Defining the Scope of the Observation

Researchers must limit the scope of their observations to behaviors that are relevant to the central issues of their study.

Limits of Naturalistic Observation

Most useful when investigating complex social settings both to understand the setting and to develop theories based on

observations.

Less useful for studying well defined hypotheses under precisely specified conditions.

Field research is very difficult to do = ever changing pattern of events = time consuming.

Must ensure data is consistent with hypotheses.

Negative case analysis: an observation that does not fit the explanatory structure devised by the researcher = therefore

must revise hypothesis.

Systematic Observation Refers to the careful observation of one or more specific behaviors in a particular setting.

Much less global than natural observation.

This research is interested in only a few very specific behaviors, the observations are quantifiable and the research

frequently has developed prior hypotheses about the behaviors.

Coding Systems

Researcher must choose a setting, which behaviors are of interest and develop a coding system to measure behaviors.

Should be simple allowing observers to easily categorize behaviors.

Researchers can use coding systems that have been developed by others; advantage: system has been proven useful and

training materials are usually available.

Ex: FICS an SYMLOG

Methodological Issues

Equipment

Video recorders have the advantage of providing a permanent record of behavior that can be coded later.

Computer-based recording devices can be used to code the observed behaviors as well as to keep track of duration.

Reactivity

The possibility that the presence of observers will affect peoples behavior.

Reliability

Indicated by high agreement among raters (who code behavior).

Sampling

For many research questions, samples of behavior taken over a long period over time provide more accurate and useful

data than single, short observations.

Case Studies Provides a description of an individual = person or setting.

Case studies and natural observation overlap. However case studies do no necessarily involve natural observation.

a psychobiography is a type of case study in which a researcher applies psychological theory to explain the life of an

individual, usually an important historical figure.

Typically a case study is done when an individual possesses a particularly rare condition.

Case study presents: individuals history, symptoms, characteristic behaviors, reactions to situations or responses to

treatment.

Case studies are valuable in informing us of conditions that are rare or unusual and thus providing unique data about some

psychological phenomenon such as memory, language or social exchange.

Insights gained through case studies may also lead to development of hypothesis that can be tested using other methods.

Archival Research Involves using previously compiled info to answer research questions.

Analyze existing data

3 types: statistical record, survey archives and written records.

Statistical Records

Collected by many public and private organizations

The US Census Bureau maintain the most extensive set of statistical records available to researchers for analysis.

Public records can also be used as sources of archival data.

Survey Archives

Consist of data from surveys that are stored on computers and available tp researchers who wish to analyze them.

One very useful data set is the General Social Survey

Extremely important because most researchers do not have the financial resources to conduct surveys of randomly

selected national samples; the archives allow them to access such samples to test their ideas.

Written and Mass Communication Records

Written records are documents such as diaries and letters that have been preserved by historical societies, ethnographies,

public documents.

Mass communication records include books, magazine articles, movies, television programs and newspapers.

Archival data may also be used in cross-cultural research to examine aspects of social structure that differ from society to

society.

HRAF = Humans Relations Area Files = info from ethnographies.

Content Analysis of Documents

The systematic analysis of existing documents such as the ones described in this section.

Like systematic observation, content analysis requires researchers to devise coding system that raters can use to quantify

the info in the documents.

Researchers must define categories in order to code the info.

Use of archival data are a valuable supplement to traditional data collection methods.

Two problems with the use of archival data: 1) Desired records may be difficult to obtain

2) Can never be sure of accuracy of info collected by someone else.

Chapter 7: Asking People About Themselves - Survey Research

Why Conduct Surveys Common and important method of studying behavior

Methodology for asking people to tell us about themselves = society demands data about issues rather than only intuition

and anecdotes.

In basic research, many important variables are measured including attitudes, current emotional states and self-reports of

behaviors are most easily studied using questionnaires or interviews.

Survey method is an important way for researchers to study relationships among variables and ways that attitudes and

behaviors change over time.

Survey research complements experimental research findings.

Multiple methods are needed to understand any behavior.

A response set is a tendency to respond to all questions from a particular perspective rather then to provide answers that

are directly related to the questions.

Can affect usefulness of data.

Most common response set is called social desirability or “faking good”

most acute when questions concern sensitive topics such as violent/aggressive behavior.

If they trust the researcher, participants will provide honest answers.

Constructing Questions to Ask Defining the Research Objectives

What is it that the researcher wants to know? = research objectives

Survey questions must be tied to research questions.

Must decide what type of questions to ask: 1) Attitudes/Beliefs: the way people evaluate and think about issues

2) Facts/Demographics: things they know about themselves and their situation = necessary to describe sample = ex: age

and gender. Other factual info you might ask will depend on the topic of your survey.

3) Behaviors: past or intended future behaviors.

Question Wording

Cognitive psychologists have identified a number of potential problems with question wording (stem from a difficulty

with understanding the question): a) unfamiliar technical terms

b) vague or imprecise terms

c) ungrammatical sentence structure

d) phrasing that overloads working memory

e) embedding the question with misleading information

Simplicity

Avoid jargon and technical terms that people won’t understand.

Sometime you have to make question more complex to define a term or describe an issue.

Double-Barreled Questions

Avoid questions that ask two things at once.

Loaded Questions

A loaded question is written to lead people to respond in one way.

Questions that include emotionally charged words may influence the way that people respond and thus lead to biased

conclusions.

Negative Wording

Avoid phrasing questions with negatives.

Ex: Do you feel that the city should not approve the proposed women’s shelter?

“Yea-Saying” and “Nay-Saying”

When you ask several questions about a topic, there is a possibility that a respondent will employ a response set to agree

or disagree with all the questions = “yea-saying” or “nay-saying”.

Problem: Respondent may be agreeing with anything you say.

One way to detect this response set is to word the questions so that consistent agreement is unlikely.

Consistently agreeing or disagreeing with a set of related questions phrased in both standard and reversed formats is an

indicator that the individuals is “yea-saying” or “nay-saying”.

Graesser and his colleagues have developed a computer program called QUAID (Question Understanding Aid) that

analyzes question wording.

Responses to Questions Closed- Versus Open-Ended Questions

Close-ended questions, a limited number of response alternatives are given; with open-ended questions, respondents are

free to answer in any way they like.

Close ended = more structured= easier to code = response alternatives are same for everyone.

Open ended= need time to categorize and code= more costly = sometimes answers can’t even be categorized

However can still yield valuable insights into what people are thinking.

Open ended are most useful when researcher needs to know what people think and how they view the world. Close ended

are more useful when the dimensions of the variables are well defined.

The two approaches can lead to different conclusions -> therefore need to have a good understanding of topic when asked

closed-ended questions.

Number of Response Alternatives

In public opinion, “yes or no” or “agree or disagree” dichotomy is sufficient.

In more basic research, it is preferable to have a 5- or 7- point scale to allow people to sufficiently express themselves.

Rating Scales

Provide “how much” judgements on any number of dimensions, very commonly used in research.

Have many different formats, the format used depends on factors such as the topic being investigated.

Simplest, most direct scale = 5-7 point scale with endpoints on the scale defining the extremes.

Graphic Rating Scale

Requires a mark along a continuous 100-millimeter line that is anchored with descriptions at each end. A ruler is then

placed on the line to obtain the score on a scale that ranges from 0-100.

Semantic DIfferential Scale

A measure of the meaning of concepts that was developed by Osgood and associates.

Respondents rate ant concept-persons, objects, behaviors, ideas - on a series of bipolar adjectives using 7-point scales.

Ex: smoking cigarettes: good---- bad (evaluation), strong---weak (potency), active----passive (activity).

Research shows anything can be measured using this technique.

Concepts rated along 3 dimensions:

Evaluation (ex adjectives)

Activity

Potency

Nonverbal Scales for Children

young children can give ratings using smiley emoticons for example.

Labeling Response Alternatives

Sometimes researchers need to provide labels to more clearly define the meaning of each alternative.

Sometimes a perfectly balanced scale may not be possible or desirable.

High frequency scales and low frequency scales as indicated by response alternatives.

Schwartz points out that labels should be chosen carefully because people may interpret the meaning of the scale

differently, depending on the labels used.

Finalizing the Questionnaire Formatting the Questionnaire

Printed questionnaire = attractive and professional, neatly typed, no spelling errors, easy to identify questions and

responses.

If using a point scale, use it consistently throughout, don’t change it.

Consider sequence in which you will ask questions: more important/interesting questions should be at beginning to

capture attention, and similar themed questions should be grouped together.

Refining Questions

Before administering you should pilot questions by giving the questionnaire to a group of people and asking them to

“think aloud” while interpreting and responding to the questions.

Administering Surveys

Questionnaires

Written format

Advantages = less costly then interviews, anonymous.

Disadvantages= must be able to read/write, people find it boring.

Personal Administration to Groups or Individuals

Advantage: you have a captive audience that is likely to complete questionnaire once they start it and researcher is present

so people can ask questions if necessary.

Mail Surveys

Inexpensive way of contacting people who were selected for the sample.

Disadvantage: low response rates and researcher is not present for people to ask questions if they are confused.

Internet Surveys

Problem: how to sample people?

Internet is making it easier to obtain samples of people with particular characteristics.

One concern about INternet data collection is whether the results will be at all similar to what might be found using

traditional methods= research indicates that Internet results are in fact comparable.

Problem: ambiguity about characteristics of an individual providing info for the study.

Other Technologies

An interesting application is seen in studies aimed at sampling people’s behaviors and emotions over an extended period

of time. With cellphones and such you can contact person and ask them to report their current emotional

reactions/activities, etc. = “computerized experience sampling”.

Interviews

The fact that an interview involves an interaction between people has important implications: 1) People are likely to agree to answer questions for a real person than to answer a mailed questionnaire. = person is more

likely to answer all questions and complete survey.

Advantage: interviewer can clarify any problems/questions of interviewee, AND interviewer can ask interviewee for

clarification on their answers if needed.

Potential problem: interviewer bias =all the biases that can arise from the fact that the interviewer is a unique human

being interacting with another human.

Interviewer could subtly bias the respondent’s answers by inadvertently showing approval/disapproval of certain

answers.

If there are several interviewers, each could possess different characteristics (ex: level of attractiveness, age, race)

that might influence the way respondents answer.

Interviewers may have expectations that could lead them to “see what they are looking for”= could bias interviewers

interpretations of responses or probe certain people for answers. Solution: careful screening an training of

interviewers help to limit such biases.

Face-to-Face Interviews

Expensive and time consuming. Usually interviewer travels to persons home/office.

Most likely used when the sample size is fairly small and there are clear benefits to a face-to-face interaction.

Telephone Interviews

Almost all interview for large scale surveys are done via telephone.

Less expensive and allow data to be collected quicker because many interviewers can work on the same survey at once.

CATI system= computer-assisted telephone interview: questions are prompted on the computer screen and the data are

entered directly into the computer for analysis= lower cost by reducing labor and data analysis costs.

Focus Group Interviews

An interview with a group of about 6-10 individuals brought together for a period of usually 2-3 hours who have a

particular knowledge or interest in topic.

Usually some monetary or gift incentive.

Consists of open-ended questions.

Advantage: group interaction is possible.

Usually recored and may be transcribed = then analyzed to find themes and areas of group consensus/disagreement.

Prefer to conduct two/three discussion groups on a given topic to ensure info is not unique to one group = however it is

costly.

Survey Designs To Study Changes Over Time Surveys most frequently study people at one point in time, however researchers sometimes wish to make comparisons

over time = conduct same survey on multiple occasions to track changes in certain variables.

Another way to study changes over time is to conduct a panel study in which the same people are surveyed at two or

more points in time.Ex: “two-wave” panel study = people surveyed 2 times.

Panel studies are particularly important when the research question addresses the relationship between one variable at

“time one” and another variable at some later “time two”.

Sampling from a Population Most research projects involve sampling participants from a population of interest.

The population is composed of all individuals of interest to the researcher.

Although studying a whole population (if small) is possible, researchers usually opt to select a sample from the population

of interest.

Can use sample to predict characteristics of population as a whole.

Statistical theory allows us to infer what the population is like, based on data obtained from a sample.

Confidence Intervals

When researchers make inferences about populations, they do so with a certain degree of confidence.

Confidence interval: you can have 95% confidence that the true population value lies within this interval around the

obtained sample result. Best estimate of the population value is the sample value.

However because only using sample, there may be error = sampling error or margin of error (given by confidence

interval).

Sample Size

Larger sample size will reduce the size of the confidence interval. = increased accuracy

Most important factor in determing the size of the interval is sample size.

Larger samples are more likely to yield data that accurately reflects the true population value.

Sample size can be determined using a mathematical formula that takes into account the size of the confidence interval

and the size of the population you are studying.

Sample size is NOT a constant percentage of the population size.

Sample size needed does not change much even as the population increases (same degree of accuracy).

Sampling Techniques Two basic techniques for sampling individuals from a population:

1) Probability sampling: each member of the population has a specifiable probability of being chosen, this is very

important when you want to make precise statements about a specific population on the basis of the results of your

survey.

2) Non-probability sampling: we don’t know the probability of any particular member of the population being chosen.

Not as sophisticated, however common and useful in many circumstances.

Probability Sampling

Simple Random Sampling

Every member of the population has an equal probability of being selected for the sample.

Stratified Random Sampling

More complicated procedure: population is divided into subgroups (or strata) and random sampling techniques are then

used to select sample members from each stratum.

Dimensions chosen to divide population must be relevant to problem under study.

Advantage: built-in assurance that the sample will accurately reflect the numerical composition of the various subgroups.

Important when some subgroups represent a very small percentage of population 85

When important to represent a small group within a population, researchers will “oversample” that group to ensure that a

representative sample of the group is surveyed.

Cluster Sampling

Obtaining a list of all members of a population might be difficult, in that case we use cluster sampling: researcher

identifies individuals and then sample from these clusters. After clusters are chosen, all individuals in each cluster are

included in the sample.

Ex: classes = clusters, randomly sample from each class.

Most often, requires a series of samples from larger to smaller clusters - a “multistage” approach.

Main advantage: researcher does not have to sample from lists of individuals to obtain a truly random sample of

individuals.

Non-Probability Sampling

Quite arbitrary

Population may be defined but little effort is expended to ensure that sample accurately represents the population.

Cheap and convenient

Three types: 1) Haphazard Sampling

2) Purposive Sampling

3) Quota Sampling

Haphazard Sampling

“convenience” sampling

a “take them where you find them” method of obtaining participants

Likely to introduce biases into the sample= results do not accurately describe whole population.

Purposive Sampling

The purpose is to obtain a sample of people who meet some predetermined criterion.

Not a probability sample, limits sample to a certain group of people.

Quota Sampling

Researcher chooses a sample that reflects the numerical composition of various subgroups in the population. (Ex:19%

freshmen, 23% sophomores in whole population must be in sample as well).

Similar to stratified sampling however no random sampling occurs in quota sampling.

Problem remains: no restrictions are placed on how individuals in the various subgroups are chosen.

The sample does reflect the numerical composition of the whole population of interest but respondents within each

subgroup are selected in a haphazard manner.

Evaluating Samples A completely unbiased sample you need:

1) randomly sample from a population that contains all individuals in the population.

2) contact and obtain completed responses from all individuals selected to be in the sample.

However there can still be two sources:

the sampling frame used

poor response rates

Sampling Frame

The actual population of individuals (or clusters) from which a random sample will be drawn.

Ex: list of numbers to contact residents between 5PM and 9PM.

Rarely will this perfectly coincide with the population of interest- some biases will be introduced.

When evaluating the results of the survey, you need to consider how well the sampling frame matches the population of

interest. Biases are usually minor but could still be consequential.

Response Rate

The percentage of people in the sample who actually completed the survey.

Indicates how much bias there might be in the final sample.

The lower the response rate, the greater the likelihood that such biases may distort the findings and in turn limit the ability

to generalize the findings to the population of interest.

Many methods can be taken to maximize response rates.

Ex: researchers should attempt to convince people that the surveys purposes are important and their participation will be a

valuable contribution. Also incentives can be used.

Reasons for Using Convenience Samples Used a lot in research.

Cheap and not time consuming

Why aren’t researchers more worried about obtaining random samples from the general population for their research?

They are focused on studying the relationships between variables even though the sample may be biased (testing

hypotheses about behavior).

Research findings are still important even if they cannot be generalized.

Generalization in science is dependent upon replicating the results. the results of many studies (using multiple

samples/methods) can then be synthesized to gain greater insight into the findings.

Some non-probability samples are more representative than others.

Chapter 8 - Experimental Design

Confounding and Internal Validity A confounding variable is a variable that varies along with

the independent variable, confounding occurs when the

effects of the independent variable and an uncontrolled

variable are intertwined so you cannot determine which of the

variables is responsible for the observed effect.

When the results of an experiment can confidently be

attributed to the effect of the independent variable, the

experiment is said to have internal validity.

All other variables in an experiment should be kept constant

through direct experimental control or through

randomization. Then only can you conclude that the

independent variable is the cause of the different results.

Basic Experiments Simplest experimental design: two variables : IV and DV. The IV has 2 levels: experimental (receives treatment) and the

control group.

The basic, simple experimental design can take one of two forms: a posttest only or a pretest-posttest design.

Posttest only Design

Researcher must 1) obtain two equivalent groups of participants, 2) introduce the IV and 3) measure the effect of the IV

on the DV.

Procedures used must achieve equivalent groups to eliminate any potential selection differences: the people selected to

be in the conditions cannot differ in any systematic way.

Solution: randomly assign or have same participants participate in both conditions.

IV is measured using the same procedure for both groups, so that you can compare the two groups.

A statistical significance test would be used to assess the difference between the groups. An experiment must be well

designed and confounding variables must be eliminated before we can draw conclusions from statistical analyses.

Pretest-Posttest Design

The only difference between the posttest design and the pretest-posttest design is that in the latter a pretest is given

before the experimental manipulation is introduced. Makes it possible to ascertain that the groups were in fact equivalent

at the beginning of the experiment.

The larger the sample, the less likelihood there is that the groups will differ in any systematic way prior to the

manipulation of the IV. There is also an increasing likelihood that any difference between the groups on the DV is due to

the effect of the IV.

Advantages and Disadvantages of the Two Designs

With small sample sizes, a pretest is efficient to assess if they equivalent.

Sometimes a pretest is necessary to select the participants in the experiment.

Can use pretests to measure the extent of change in each individual.

A pretest is also necessary whenever there is a possibility that participants will drop out of the experiment (in studies that

take longer periods of time).

The dropout factor in experiments is called mortality.

Groups may become not equivalent due to different mortality rates. Mortality can become an alternative explanation for

the results.

Use of a pretest enables you to assess the effects of mortality; you can look at the pretest scores of the dropouts and

know whether mortality affected the final results.

However one disadvantage of a pretest is that it is time consuming and awkward to administer in the context of the

particular experimental procedures being used.

It can also sensitize participants, and they may figure out your hypothesis = may react differently.

Thus IV may not have an affect in the real world where pretests are not given.

Solution: disguise pretest.

It is possible to assess the impact of the impact of the pretest directly with a combination of both designs. In this design

half the participants receive only the posttest, and the other half receive both= Solomon four-group design.

IV

Pretest Condition Control Group Experimental Group

No pretest (posttest only)

Pretest and posttest

Assigning Participants to Experimental Conditions Independent groups design: participants are randomly assigned to the various conditions so that each participates in only

one group.

Repeated measures design: participants are in all conditions, each participant is assigned to both levels of the IV.

Independent Groups Design Different participants assigned to each of the conditions using random assignment = prevent systematic biases, and groups

will be equivalent in terms of participant characteristics such as income, intelligence, age, or political attitudes.

Researchers usually use a sequence of random numbers to determine assignment.

Repeated Measures Design Participants are repeatedly measured on the DV after being in each condition of the experiment.

Advantages and Disadvantages of Repeated Measures Design

Advantage: fewer research participants are needed.

Saves money as well.

Advantage: extremely sensitive to finding statistically significant differences between groups.

More likely to detect an effect of the IV on the DV, because its easier to separate the systematic individual

differences from the effect of the IV.

Ex: recall test.

Disadvantage: order effect - the order of presenting the treatments affects the DV.

There are several types of order effects: Order effects that are associated simply with the passage of time include practice

effects and fatigue effects. 1) Practice effect: an improvement in performance as a result of repeated practice with a task.

2) Fatigue effect: deterioration in performance as the research participant becomes tired, bored or distracted.

Other types of order effects occur when the effect of the first treatment carries over to influence the response to the second

treatment.

3) Contrast effect: when the response to the 2nd condition in the experiment is altered because the 2 conditions are

contrasted to one another.

Two approaches to deal with such problems:

Employ counterbalancing techniques.

Devise a procedure in which the interval between conditions is long enough to minimize the influence of the 1st

condition on the 2nd.

Counterbalancing

Complete Counterbalancing

In repeated measures design it is very important to counterbalance the order of the conditions. With complete

counterbalancing, all possible orders of presentation are included in the experiment.

By counterbalancing the order of conditions it is possible to determine the extent to which order is influencing the results.

Latin Squares

A technique to control for order effects without having all possible orders is to use a Latin square: a limited set of orders

constructed to ensure that 1) each condition appears at each ordinal position and 2) each condition precedes and follows

each condition one time.

The number of orders in a latin square is equal to the number of conditions.

Need 1 participant per row. The number of participants run in each order must be equal.

In an experiment in which individuals are tested over a series of trials (repeated measures variable), however in this case

counterbalancing is not needed because the order effect of changes in performance over trials is of interest to the

researcher.

Time Interval Between Treatments

Researchers need to carefully determine the time interval between presentation of treatments and possible activities

between them.

A rest period may counteract a fatigue effect.

Attending to an unrelated task between treatments may reduce the possibility of a contrast effect.

However the introduction of an extended time interval between treatments may lead to fewer people wanting to participate

and some may dropout.

Choosing Between Independent Groups and Repeated Measures Designs

Repeated measures design have two advantages over independent groups designs: 1) a reduction in the number of

participants required to complete the experiment and 2) greater control over participant differences and thus greater ability

to detect an effect if the IV.

A very different consideration in whether to use a repeated measures design concerns generalization to conditions in the

“real world”.

Any experimental procedure that produces a relatively permanent change in an individual cannot be used in a repeated

measures design. (Ex: psychotherapy treatment or surgery).

Matched Pairs Design Instead of simply randomly assigning participants to groups, the goal is to first match people on a participant

characteristic.

The matching variable will either be the dependent measure or a variable that is strongly related to the DV.

Goal is to achieve same equivalency of groups in a repeated measures design without the necessity of having the same

participants in both conditions.

Members of each pair are randomly assigned to the conditions in the experiment.

Groups are equivalent on the matching variable. Important for small sample sizes.

Therefore used when few participants are available or if its costly to run large numbers of participants.

Results in a greater ability to detect a statistically significant effect of the IV because it is possible to account for

individual differences in responses to the IV (just as with repeated measures design).

Matching procedures can be costly and time consuming (must measure participants on matching variable prior to

conducting study) and its only worthwhile when matching variable is strongly related to dependent measure.

Not commonly used.

Chapter 9- Conducting Studies

Selecting Research Participants The method used to select participants has implications for generalizing research results.

When it is important to accurately describe population, you must use probability sampling (crucial when conducting

scientific polls).

Ample evidence supports the view that we can generalize findings from haphazard samples to other populations and

situations.

Manipulating the Independent Variable To manipulate an IV you have to construct an operational definition of the variable and then turn a conceptual variable

into a set of operations: specific instructions, events, and stimuli to be presented.

In addition, the IV and the DV must be introduced within the context of the total experimental setting = setting the stage.

Setting the Stage

Provide participants with informed consent and explain to participants why the experiment is being conducted.

There are no clear cut rules for setting the stage except that the experiment must seem plausible to the participants nor are

there any clear cur rules for translating conceptual variables into specific operations.

How the variable is manipulated depends on the variable and the cost, practicality and ethics of the procedures being

considered.

Types of Manipulations

Straightforward Manipulations

Presenting written, verbal or visual material to participants; manipulate using instructions and stimulus presentations.

Stimuli may be presented verbally, written, via videotape or with a computer.

Studies on jury decisions often ask participants to read a description of a jury trial in which a crucial piece of information

is varied (ex: severity of injury).

Most memory research relies on straightforward manipulations.

Most manipulations of IV’s in all areas of research are straightforward. Researchers vary the difficulty of material to be

learned, motivation levels, the way questions are asked, characteristics of people to be judged and a variety of other

factors in a straightforward manner.

Staged Manipulations

Less straightforward. Sometimes it is necessary to stage events that occur during the experiment in order to manipulate the

IV successfully = staged or event manipulation.

Used frequently for two reasons:

Researcher may be trying to create come psychological state in the participants (ex: anger)

A staged manipulation may be necessary to simulate some situation that occurs in the real world.

Staged manipulations frequently employ a confederate (an accomplice) = usually appears to be another participant.

Staged manipulations demand a great deal of ingenuity and even some acting ability. Used to engage participants in an

ongoing social situation.

Such procedures allow for a great deal of subtle interpersonal communication that is hard to put into words; this may

make it difficult to interpret. (If many things happened during experiment, what one thing was responsible for the results?)

In general easier to interpret results when straightforward manipulations are used.

Strength of Manipulation

A general principle is to make manipulation as strong as possible = maximizes differences between two groups and

increases the chances that the IV will have a statistically significant effect on the DV.

Strong manipulation is most important when trying to demonstrate that a relationships exists between two variables.

The principle of using the strongest manipulation is tempered at two considerations:

strongest possible manipulation may involve a situation that rarely if ever occurs in the real world.

Ethics: a manipulation should be as strong as possible within the bounds of ethics.

Cost of Manipulation

Cost is another factor in the decision about how to manipulate the IV. Limited monetary resources.

Straightforward manipulations are less costly then staged.

Measuring the Dependent Variable Types of Measures

The DV in most experiments is one of three general types: self-report, behavioral, or physiological.

Self-Report Measures

Measure attitudes, liking for someone, judgements about someones personality characteristics, intended behaviors

emotional states, attributions about why someone performed well or poorly on a task, confidence in one’s judgements and

many other aspects of human thought and behavior.

Rating scales with descriptive anchors are most commonly used.

Behavioral Measures

Direct observations of behaviors.

Measurements of an almost endless number of behaviors is possible (just as with self-reports).

Often the researcher must decide whether to record the number of times a behavior occurs in a period of time = rate, how

quickly a response occurs after a stimulus = reaction time or how long a behavior lasts = duration. Or the

presence/absence of the behavior.

When both self report and behavioral measures could be used, a series of studies may be conducted to study the effects of

an IV on both types of measures.

Physiological Measures

Recordings of responses of the body. Physiological indicators of psychological variables.

Many such responses are available: galvanic skin response (GSR), electromyogram (EMG) and electroencephalogram

(EEG).

The GSR is a measure of general emotional arousal and anxiety, it measures the electrical conductance of the skin which

changes when sweating occurs.

The EMG measures muscle tension and is frequently used as a measure of tension or stress.

The EEG is a measure of electrical activity of brain cells. It can be used to record general brain arousal as a response to

different situations, activity in different parts of the brain as learning occurs, or brain activity during different stages of

sleep.

MRI’s (magnetic resonance imaging) provide an image of the brain structure of an individual

Functional MRI = allows researchers to scan areas of the brain while a research participant performs a physical or

cognitive task.

Provides evidence for what brain processes are involved in these tasks.

Other physiological indicators include: heart rate, temperature and blood/urine analysis.

Sensitivity of the Dependent Variable

The DV should be sensitive enough to detect differences between groups.

Issue of sensitivity is particularly important when measuring human performance.

When a task is too easy, everyone does well = ceiling effect - the IV appears to have no effect on the dependent measure

only because participants quickly reach the maximum performance level.

Opposite problem occurs when a task is too difficult that hardly anyone can perform well = floor effect.

Multiple Measures

Often desirable to measure more than one DV, because a variable can be measured in a variety of concrete ways

(operational definitions).

It is useful to know whether the same IV affects some measures but not others.

Researchers may also be interested in studying the effects of an IV on several different behaviors.

If there is a problem of order (regarding the measures) you could counterbalance. If there is no order problem, most

important measures go first.

Making multiple measurements in a single experiment is only valuable when it is feasible to do so. However may be

necessary to conduct a series of experiments to explore the effects of an IV on various behaviors.

Additional Controls Controlling for Participant Expectations

Demand Characteristics

Demand characteristics: any feature of an experiment that might inform participants of the purpose of the study.

When participants form expectations about hypothesis of study, they will do whatever to confirm it (if cooperative).

To control, you could use deception. Attempt to disguise the DV by using an unobtrusive measure placing it among a set

of unrelated filler items on a questionnaire.

Experiments conducted in field settings and observational research in which observer is concealed or unobtrusive

measures are used minimize demand characteristics.

Placebo Groups

A special kind of participant expectation aries in research on the effects of drugs.

We do not know whether an improvement in an experimental group was caused by the properties of the drug or by the

expectations about the effects of the drug = placebo effect.

To control for this possibility a placebo group can be added = they receive a pill that is not actually the drug but an inert

substance.

If experimental group shows greater improvement than placebo group we know its because of the drug itself.

Balanced placebo design: alcohol example.

Ethical implications: if drug has positive effects, must be given after study to those who were in the control groups.

Great deal of research and debate on the extent to which any beneficial effects of antidepressants such as Prozac are due to

placebo effects.

Controlling for Experimenter Expectations

Experimenter expectations can in turn bias the results = experimenter bias or expectancy effects.

Can occur when experimenter knows which condition participants are in. Two sources of bias:

Might unintentionally treat participants differently based on condition they are in.

Experimenter record the behaviors of the participants, there may be subtle differences in the way the experimenter

interprets and records behaviors.

Research on Expectancy Effects

Experimenter expectancies can be communicated to humans by both verbal and non verbal means.

Teacher expectancy = teacher’s expectations can influence students performance.

Expectations influence ratings of behavior.

Solutions to the Expectancy Problem

Experimenters should be well trained and should practice behaving consistently with all participants.

Run all conditions simultaneously so that experimenters behavior is the same for all participants.

Expectancy effects are also minimized when procedures are automated. (record responses using computers for example)>

Use experimenters who are unaware of the hypothesis. Person is blind to what is being studied or which condition the

participant is in.

In a single-blind experiment, the participant is unaware of whether a placebo or the actual drug is being administered.

In a double-blind experiment neither the participant or the experimenter knows whether the placebo or actual treatment is

being given.

Additional Considerations Research Proposals

Researcher write a research proposal = includes a literature review providing background of study.

Procedure, analysis procedures to be done, what and why the research is being done.

Proposals must be included in applications for research grants; ethics review committees require some type of proposal as

well.

Pilot Studies

Pilot study: researcher does a “trial run” with a small number of participants= reveals if instructions are understood, if

experimental setting is plausible, etc. Ask participants about experience of the experiment.

Another method: “think aloud”.

Manipulation Checks

A manipulation check is an attempt to directly measure whether the IV manipulation has the intended effect on

participants.

Provides evidence for the construct validity of manipulation.

Ask whether the IV manipulation was a successful operationalization of the conceptual variable being studied.

Should be done near the end of the experiment before debriefing = so no distraction during actual experiment.

Two advantages:

If manipulation is not working, saved expense of running experiment

Can identify whether nonsignificant results are due to a problem in manipulating the IV.

Debriefing

Discuss ethical and educational implications of study.

Learn what participants were thinking during experiment = useful i interpreting results and planning future studies.

Ask participants to refrain from discussing study with others.

Analyzing and Interpreting Results Statistical analyses of the data are carried out to allow the researcher to examine and interpret the pattern of results

obtained in the study; helps researcher decide if there is a relationship between the IV and the DV.

Communicating Research to Others Final step is to write a report that details why you conducted the research, how you obtained the participants, what

procedures you used, and what you found.

Research findings are then usually submitted as journal articles/papers to be read at scientific meetings = evaluated by two

or more knowledgable reviewers who decide whether paper is acceptable for publication/presentation or not (peer review).

Professional Meetings

Meetings sponsored by professional associations are important opportunities for researchers to present their findings to

others and learn about others research findings.

Verbal presentation of poster sessions.

Journal Articles

Almost 90% of articles submitted get rejected, very hard to publish.

Chapter 10- Complex Experimental Designs

Increasing the Number of Levels of an Independent Variable The simple design allows us to examine important parts of research such as internal validity.

Researcher might want to design an experiment with three or more levels of IV for several reasons.

First a design with only two levels of IV cannot provide very much info about the exact form of the relationship between

the independent and dependent variables.

If a curvilinear relationship is predicted, at least three levels must be used in order for the relationship to be detected.

Example: relationship between fear arousal and attitude change.

Finally, researcher frequently are interested in comparing more than two groups.

Increasing the Number of Independent Variables: Factorial Designs Researchers often manipulate more than one IV in a single experiment, typically 2 or 3 IVs are operating simultaneously

= closer approximation of real world conditions.

Factorial designs are designs with more than one IV (or factor).

All levels of each IV are combined with all levels of the other IVs.

Ex: 2 x 2 ; two IVs each having two levels. Has four groups

General format for describing factorial designs is: number of levels of 1st IV x Number of levels of 2nd IV x Number of levels of 3rd IV and so on.

Interpretation of Factorial Designs

Yields two kinds of information:

Info about the effect of each IV taken by itself: the main effect of an IV. (2 IVS= 2 main effects).

Second type of info is interaction, is there is an interaction between two IVs, the effect of one IV depends on the

particular level of the other variable. In other words, the effect that an IV has on the DV depends on the level of the

other IV.

Main Effects

A main effect is the effect each variable has by itself.

Main effect of each IV is the overall relationship between the IV and the DV.

Overall main effect means are obtained by averaging across all participants in each group.

Interactions

Interactions can be easily seen when the means for all conditions are presented in a graph. DV is always on the vertical

axis, one IV is placed on the horizontal axis, bars are then drawn to represent each of the levels of the other IV = useful

method of visualizing interactions in a factorial design.

Interaction between IVs indicates that the effect of one IV is different at different levels of the other IV. (Main effects of

the IVs must be qualified).

Factorial Designs with Manipulated and Non-manipulated Variables

One common type of factorial design includes both experimental (manipulated) and non experimental (measured or non-

manipulated) variables.

IV x PV designs (ex: IV by participant variable); allow researchers to investigate how different types of individuals

respond to the same manipulated variable.

“Participant variables” (aka subject or attribute variables) are personal attributes such as age, gender, ethnic group,

personality characteristics and clinical diagnostic category.

The simplest IV x PV designs include one manipulated independent variable that has at least two levels and one

participant variable with at least two levels.

Such experiments recognize that full understanding of behavior requires knowledge of both situational variables and the

personal attributes of individuals.

Interactions and Moderator Variables

In many research studies, interactions are discussed in terms of the operation of a moderator variable: influences the

relationship between two other variables.

Moderator variables may be particular situation or they may be characteristics of people.

Outcomes of a 2 x 2 Factorial Design

When analyzing the results there are several possibilities: 1) There may or may not be a significant main effect for IV A

2) There may or may not be a significant main effect of IV B

3) There may or may not be a significant interaction between the IVs.

Line graphs are usually used when the levels of the IV on the horizontal axis (IV A) are quantitative- low and high

amounts.

Bar graphs are more likely to be used when the levels of the IV represent different categories (ex: different types of

therapies).

Interactions and Simple Main Effects

A procedure called analysis of variance is used to assess the statistical significance of the main effects and the interaction

in a factorial design.

When there is a significant interaction, there is a need to statistically evaluate the individual means.

When there is a significant interaction, the next step is to look at the simple main effects. A simple main effect analysis

examines mean differences at each level of the IV.

Recall that the main effect of an IV averages across the levels of the other IVs; with simple main effect, the results are

analyzed as if we had separate experiments at each level of the other IV.

The simple main effect that you will be most interested in will depend on the predictions that you made when you

designed the study.

Assignment Procedures and Factorial Designs

The design can be completely independent groups, completely repeated measures or a mixed factorial design that is a

combination of the two.

Have different implications for the number of participants needed to complete the experiment.

Mixed Factorial Design Using Combined Assignment

Ex: IV A is an independent groups variable. 10 participants are assigned to level 1 of this IV and another 10 are assigned

to level 2. IV B is a repeated measures group, however the 10 participants assigned to A1 receive both levels of IV B.

Similarly the 10 assigned to A2 receive both levels of IV B.

Increasing the Number of Levels of an Independent Variable

The 2 x 2 is the simplest factorial design.

One way to increase complexity is to increase the number of levels of one or more of the IVs.

Increasing the Number of Independent Variables in a Factorial Design

A 2 x 2 x 2 contains three variables each with two levels.

Yields many effects for each of the IVs.

We can look at a three-way interaction that involves all three IVs, here we want to determine whether the nature of the

interaction between two of the variables differs depending on the particular level of the other variable.

When there are more than 3 or 4 IVs, many of the particular conditions that are produced by the combination of so many

variables do not make sense or could not occur under natural circumstances.