copyright, 2005, prentice hall, sarafino chapter 10 preparing to conduct a study

Copyright, 2005, Prentice Hall, Sarafino

CHAPTER 10

Preparing to Conduct a Study


The Developmental Phase

• The easiest part of any research is the actual gathering of data.– It is the easiest part because of all the

planning and effort you have put into in it to ensure that things go smoothly.

– The slides that follow will highlight the nine – yes 9 – phases that researchers must go through in order to ensure an efficient and true outcome of their research effort.


Phase 1: Identify the Research Topic and Hypothesis

• Things to consider:– Have you reviewed the ideas of other on

this topic? Do your ideas mesh with those others? Why or why not?

– Are there any theories relating to your topic?– Is your hypothesis testable and falsifiable?– Is your hypothesis theoretically supported

by previous literature?


Phase 2:Determine Participants and Their Availability

• Do you need human or animal participants and how many do you need?

• Do your participants need to have unique qualities (e.g., have a phobia, or prone to develop a disease)?– If so, can you find enough participants?

• Will you be able to generate equal groups?– If not, what are the implications?


Phase 3: Choose Your Variables

• Are the variables you selected consistent with previous research?– How do I find reliable and valid measurement

instruments to measure my variables?• Mental Measures Yearbook; Test Critiques

• Are you going to manipulate your IVs?– If so, what levels will you select to maximize

variance?

• Will your dependent variable(s) produce nominal, ordinal, interval, or ratio data?


• What is your hypothesis asking?– Will an experimental, quasi-experimental,

correlational, or descriptive strategy work best to support your hypothesis.

• Will you need a control group? If so, what kind?• Will the participants be repeatedly measured?

• What statistical analyses will need to be performed?– Are there assumptions associated with running these

analysis, are there limits to the interpretive power of these statistics?

Phase 4: Choosing a Design


• Try to predict unwanted sources of variance in your study and then implement control procedures.

– If you think participant or experimental bias will influence your result then implement a double blind procedure.

– If there may be order effects, make sure you counterbalance.

• How will you establish and maintain equivalent groups?• Is it possible to hold the conditions of the experiment

constant.– Will the different groups of participants really be treated

identically except for the IV manipulation?

Phase 5: Identify Sources of Error, and Controls


Phase 6: Identify Limitations on Generalization

• Will the types of participants used (e.g., students) limit the study’s external validity?

• Does the environment in which the study is run (e.g., laboratory) limit external validity?

• How far will your statistical analyses allow you to generalization?


Phase 7: Ethical Risks and Debriefing of Participants

• Ask yourself the following questions: Will or might your study:

– Physically or psychologically harm?– Use deception? Violate privacy?– Use coercion? Fail to use informed consent?

• If you answer yes to any of these – ask yourself if they can be avoided, if not, be sure to include measures in your debriefing to address and/or minimize any possible lasting effects.


Phase 8: Design An Informed Consent Form

• Informed consent is usually required in any study with human participants.– Informed consent involves providing

participants with the relevant details of the study, what they might experience, and that they can withdraw from the experiment at any time. Once participants are informed, then their written consent is obtained to participant in the research.

– See example in your textbook.


Phase 9: Submit a Proposal for IRB Approval

• IRB = Institutional Review Board

• The proposal often includes:– Detailed descriptions of the purpose,

methods to be use, the expected results, and the implications of the results.

• The previous 8 phases should provide you will enough information to submit a good proposal.


Pilot or Trail Runs

• Once you have received all the necessary approvals, it is time to run your study.

• Many studies include a consolidation phase to debug the study before the actual data collection begins.– This is often called a pilot study or a trail

run of the proposed research.


Pilot Study

• A pilot study is a trail run of your entire experiment with participants – the participants in your pilot usually are not included in the actual study.

• The pilot study will help you:– Isolate unforeseen problems and fix them– Figure out how long the testing takes– Identify problems with test materials and/or variables.

• Any of the above may lead to a redesign of the study, a slight change in the operational definitions of variables, or even an altering of the hypotheses.


A Crucial Problem: Sampling

• After debugging your study in the pilot phase, it is now time to collect data.

• One of the more important aspects you’ll need to consider is how you will recruit and select your participants. This is called sampling.– Sampling, if done correctly, will enhance external

validity. But if done incorrectly will significantly decrease your study’s external validity.

– The goal of sampling is to generate a representative sample of your target population.


Types of Sampling

• There are two main categories of sampling:– Probability Sampling

• The odds of being selected from a population to a research study can be calculated.

– Nonprobability Sampling• The odds of being selected from a population

to a research study cannot be calculated


Probability Sampling

• Probability samples almost always start with a “sampling frame” – a list of all individuals from which the sample will be selected. In a best scenario, all members of the target population will be on the list.

• There are several types of probability sampling:– Simple Random Sample

• Each individual in the population has an equal chance of being selected (e.g., names out of a hat).

– Stratified Random Sample• The sample frame is partitioned in some manner and simple

random sampling is done on each partition. Why?– Cluster Sample

• Units or clusters of similar individuals are identified. Some clusters are randomly selected, and each individual in each of the selected clusters is measured.


Nonprobability Sample

• In nonprobability sampling a sampling frame is not present.– As a result, nonprobability samples may not be representative

of the target population.• The most common type of nonprobability sample (and

the most common in psychology) is called a “convenience sample”

• Convenience samples are qualified volunteers who are easily available. You were likely part of a convenience sample if you participated in research while enroll in your first year Introduction to Psychology course.– If a convenience sample is used, what should researchers

report regarding the convenience sample?– How will a convenience sample affect external validity?


Descriptive Statistics

• Descriptive stats provide information about the central tendencies of a group of data.

• Importance terms include:– Mean: the arithmetic average in a data set– Median: the middlemost score in a data set– Mode: the most frequent score in a data set– Variance: the degree to which scores in a data set

deviate from the mean. There are various ways to measure variance or variability.• Range: measure of variability in a data set• Standard Deviation: most commonly used measure of

variability.


Inferential Statistics

• Inferential statistics are mathematical / statistical procedures for determining the probability that the relationships or differences we observe in our data actually occur in the population.

• Inferential statistics also tell us whether the differences we see in our data occurred by chance or not.

• There are 2 types of inferential statistics:– Parametric statistics– Nonparametric statistics


Parametric and Nonparametric Statistics

• Parametric Statistics test hypotheses that are based on data that allow us to estimate parameters (e.g., means and standard deviations).– In other words, parametric statistics are used with

interval or ratio data.– E.g. Pearson r, multiple regression, t-test, ANOVA

• Nonparametric Statistics test hypotheses that do not involve parameters (e.g., when the data are nominal or ordinal, or not normally distributed).– E.g., Spearman rank correlation, Chi-square.


Statistical Significance

• What do we mean when we say something is statistically significant?– We are simply saying that there is only a small probability that

what we found was due solely to chance.– That small “chance” probability goes by various names:

• Type I Error• Alpha level• And is often symbolized by an italicized p “p”

• The alpha level is predetermine before the study begins and in psychology the level is usual set to 5% or 0.05.– This means that if the results we gather cannot be obtained by

chance more that 5 times in 100 random trials we would say that our results are statistically significant. It is unlikely they occurred solely by change. So can we ever be wrong?

copyright, 2005, prentice hall, sarafino chapter 10 preparing to conduct a study

Documents

types of participants

animal participants

research topic

previous research

research effort

hypothesis testable

dependent variables

experimental bias