educational research

Educational Research

Presented by Erlan Agusrijaya

The PurposeOf Educational Research

To provide evidence to help people decide which opinions are

correct –at least more correct.

To help us develop better ways to think

about the field of education.

Exercise: Indicate, on a scale of 1-5, the extent to which you think research has demonstrated the truth of each statement.

Clearly refuted by research

Somewhat refuted by research

inconclusive


Clearly supported by research

1 2 3 4 5 1. The more time beginning readers spent on phonics, the better readers they become.

1 2 3 4 5 2. Bilingual education for nonnative speakers impairs their academic proficiency.

1 2 3 4 5 3. Increased contact with handicapped people results in a more positive attitude toward them.

1 2 3 4 5 4. Boys are better in math; girls are better in languages.

Answer

Answer

Answer

Answer

Exercise: Indicate, on a scale of 1-5, the extent to which you think research has demonstrated the truth of each statement. (Continued)



inconclusive



1 2 3 4 5 5. Requiring students who do not like one another to work together on a project results in an increase in their liking for each other.

1 2 3 4 5 6. Students who take moral education courses behave more ethically than students who do not take such courses.

1 2 3 4 5 7. The use of manipulatives in the elementary grades results in improved achievement in mathematics.

1 2 3 4 5 8. Behavior modification is an effective way of teaching skills to very slow learners.

Answer

Answer

Answer

Answer

Exercise: Indicate, on a scale of 1-5, the extent to which you think research has demonstrated the truth of each statement. (Continued)



inconclusive



1 2 3 4 5 9. Classroom discussion of real-life sexual issues and problems results in increased promiscuity among teenagers.

1 2 3 4 5 10. Among children who become deaf before languages has developed, those with hearing parents become better readers than those with deaf parents.

1 2 3 4 5 11. The more teachers know about a specific subject matter, the better they teach it.

Answer

Answer

Answer

Forms of Educational Research

Surveys

Experiments

Case Studies

Ethnographies

Statement 1 is rated at 3.

Despite a great deal of research on the topic, this statement can be neither clearly supported nor refuted. It is clear that phonics instruction is an important ingredient; what is not clear how much time should be devoted to it.


Evidence is unclear as to whether or not bilingual methods are superior to English-only instruction, but several studies indicate no impairment of academic skills.


Evidence indicate that a more positive attitude results only if the nature of the contact is structured beforehand.


There is a considerable amount of evidence that these gender differences exist, though the reasons are not clear.


The evidence here is quite clear that the outcome depends on whether the students involved see one another as necessary to achieving success.


There is relatively little research on ethical behavior.


The evidence is quite supportive of this method of teaching mathematics.


There is a great deal of evidence to support the statement


Not much evidence exists and the evidence that does not exist is inconclusive.


The findings of many studies refute the statement.


The evidence is inconclusive despite the seemingly obvious fact that the teacher must know more than the students.

Empirical Research vs. Nonempirical Research

ER is one that involves the collection of firsthand information.

NR does not involve the collection of information at firsthand.

• Thus, research that consists of locating and comparing references on a particular topic—the customary term paper—is not an example of ER.

Basic AppliedResults apply to a great many people and situations.

Results are applicable only to a specific group of people in a particular situation.

Result are related to general theory or to a general field of knowledge.

Results are not necessary related to a broader field of knowledge .

Results need not have immediate or even clear implications for practice.

Results must have immediate and clear implications for practice.

Basic Research vs. Applied Research

Feasible —enable you to collect data.

Clear (i.e. most people agree as to what the key terms in the question mean).

Significant (i.e. it is worth investigating because it will contribute important knowledge about the human condition)

Ethical (i.e. it will not involve physical or psychological harm or damage to human beings, or to the natural or social environment of which they are a part)

Indicates a relationship of some sort (i.e. two or more qualities are suggested as being connected or related to each other in some way).

Research Question

Examples of Research Questions (with an appropriate methodology)

Does client-centered therapy produce more satisfaction in clients than does traditional therapy? (experimental research)

Are the descriptions of people in social studies in textbooks biased? (content-analysis research)

What goes on in an elementary school classroom during an average week? (ethnographic research)

Do teachers behave differently toward students of different genders? (causal-comparative research)

How can we predict which students might have trouble learning certain kinds of subject matter? (correlational research)

How do parents feel about the school counseling program? (survey research)

How can a principal improve faculty morale? (interview research)

Exercise: Which research questions suggest relationships?

1. To what extent are manipulatives used to teach mathematics in the primary grades in the Colma Elementary School District?

2. Do the principals in our district favor mainstreaming gifted students?

3. Are people with eating disorders more likely to have a history of sexual abuse in early childhood?

4. Do students who are taught Spanish in a language laboratory become more fluent in the language than students who are taught Spanish in a traditional classroom setting?

Question 1 and 2 do not suggest a

relationship.

Question 1 asks for no more than a description regarding the current usage of manipulative materials in a particular school district. Similarly, question 2 asks only for a survey of administrative opinions. Investigations of such questions may be extremely useful in their own right, but they do not extend our knowledge as to why such conditions exist.

DISCUSSION:

Question 1 and 2 indicate a

relationship.

Question 3 seeks to investigate a possible relationship between eating disorders and sexual abuse. If a history of sexual abuse is related to eating disorders, this suggests (although it does not prove) that such abuse may be a cause of such disorders. It also suggests that counseling which addresses patient history may be helpful. Question 4 seeks to investigate a possible relationship between the type of language instruction and fluency in the language taught. If the language laboratory method is shown to be more effective than classroom instruction by individual teachers, this has clear implications for improving language learning.

DISCUSSION:

A relationship

is a statement

about variables.

A variable is any characteristic that is not always the same

—that is, any characteristic that

varies.

Examples of variables include gender, eye color,

achievement, motivation, and running speed.

RELATIONSHIP and VARIABLE

Exercise: What are the variables in this research question?

“Are younger students more anxious in mathematics courses than older students?”

Answer: the variables are age and level of anxiety in mathematics courses.

Measured/ Quantitative Variables

• ~exist in some degree rather than all or none• are measured along a continuum from “less” to “more”• assigned numbers to different individuals or objects• An example would be height.

Categorical/ Qualitative Variables

• ~not vary in degree, amount, or quantity, but are qualitatively different• e.g. eye color, gender, religious preference, occupation, position on a

baseball team, and most kinds of “treatments” or “methods”

Quantitative vs. Qualitative Variables

Independent Variables

• are those the investigator chooses to study (and often manipulate) in order to assess their possible effect(s) on one or more other variables

• are those the investigator chooses to study (and often manipulate) in order to assess their possible effect(s) on one or more other variables

Dependent Variable

• is the variable which the independent variable is presumed to affect• All outcome variables are dependent variables.

Independent vs. Dependent Variables

Exercise: What are the independent and the dependent variables in this question?

“Will students who are taught by a team of three teachers learn more than students taught by one individual teacher?”

DISCUSSION:

Do you see that the independent variable is the number of teachers, and the dependent variable is the amount of learning? Notice that there are two conditions (sometimes called levels) of the independent variable—three teachers and one teacher. Also notice that the dependent variable is not “learning” but “amount of learning”. Can you see why?

Extraneous Variables are independent variables that have not been controlled

Constants are potential variables that are not allowed to change

Extraneous Variables and Constants

Every researcher should consider:

the protection of participants from harms

the ensuring of confidentiality of research data

the knowing deception of research subjects

Ethics and Research

Hypotheses

A hypothesis is, simply put, a prediction of some sort regarding the possible outcomes of a study.

• Question: “Do individuals who see themselves as socially attractive want their romantic partners also to be socially attractive?”

• Hypothesis: “Individuals who see themselves as socially attractive will want their romantic partners also to be (as judged by others) socially attractive.”

A research question is often restated as a hypothesis.

A Directional Hypothesis is one that indicates the specific direction (e.g., higher, lower, more, less) that a researcher expects to emerge in a relationship.

Nondirectional Hypothesis does not make a specific prediction about what direction the outcome of a study will take.

Directional vs. Nondirectional Hypotheses

Reviewing the Literature:

General References: the sources a researcher

refers to first.

Secondary Sources: publications in which

authors describe the work of others.

Primary Sources: publications in which

investigators report the results of their studies.

1• Define the research problem as

precisely as possible.

2• Skim through some relevant

secondary sources.

3• Peruse one or two general reference

works.

4

•Formulate search terms (key words or phrases) that are pertinent to your research question.

5 •Search the general references for relevant primary sources.

6 •Read the relevant primary sources.

7 •Take notes and summarize the key points in the sources.

Steps Involved in a Literature Review

A Computer Search of the Literature

Define the problem as precisely as possible.

Decide on the extent of the search. Decide on the Database. (e.g.,ERIC) Select descriptors. Conduct the search. Broaden or narrow the search. Obtain a printout of the desired

references.

Writing Your Summaries

1. Try to locate at least five recent primary sources that are pertinent to your topic. At least three of these be should be research reports that present data of some kind (scores on a test, responses to a questionnaire, and so on). The other two may be the viewpoint or ideas of someone as expressed in an article (that is, merely an opinion piece that does not present data).

Writing Your Summaries

2. Limit your summary to approximately one-half page (200 words).

3. Be sure to describe what the author did and what the author’s conclusions were.

4. If the reference you are summarizing pertains to a research study, you should briefly describe the method of the researcher used. Be sure that you also note how the author arrived at his/her conclusions.

An Example of a SummaryWalberg, H. J., and Thomas, S. C. 1972. An operational definition and validation in Great Britain and the United States . American educational research journal, 9:197-216.The purpose of this article is to describe the development of an observation scale and a teacher questionnaire for assessing the degree of “openness” of a given elementary school classroom. Items were written within each of eight “themes” obtained from available literature and reviewed by a panel of authorities.The resulting instruments were used in approximately 20 classrooms for each of three types: British open, American open, and American traditional. The classrooms were identified by reputation and personal knowledge. Approximately equal numbers of lower and middle socioeconomic-level classrooms were included.Results showed that overall assessments obtained with the two different instruments (observation scale and questionnaire) agreed quite highly. Differences between the open and traditional classrooms were much greater than those between socioeconomic levels or between countries.

Subject: an individual who participates in a research study or is someone from whom data are collected. (McMillan, 1992: 68)

Population: a group of elements or cases, whether individuals, objects, or events, that conform to specific criteria and to which we intend to generalize the results of the research. (McMillan, 1992: 69)

Sample: the group of elements, or a single element, from which data are obtained.

Subjects and Sampling

Examples of populations

All of the high school principals in the United States.

All of the elementary school counselors in the state of California.

All of the students attending Central High School in Omaha, Nebraska, during the academic year 1987-1988.

All of the students in Mrs. Browns’ third-grade class at Wharton Elementary School.

Examples of samples

A researcher is interested in studying the effects of diet on the attention span of third-grade students in a large city. There are 1500 third graders attending the elementary schools in the city. The researcher selects 150 of these third graders, 30 each in five different schools, to study.

The principal of an elementary school district wants to investigate the effectiveness of a new U.S. history textbook being used by some of teachers in her district. Out of a total 22 teachers who are using the text, she selects 6, comparing the achievement of students in the classes of these 6 teachers with those of another 6 teachers who are not using the text.

SamplingProcedures

ProbabilitySampling

NonprobabilitySampling

Simple Random Sampling

Stratified Random Sampling

Random Cluster Sampling

Two Stage Random sampling

Convenience Sampling

Systematic Sampling

Purposive Sampling

Simple Random Sampling (SRS)

In SRS every member of the population has an equal and independent chance of being selected for the sample.

Example:" We interviewed a sample of 41 mothers of eight graders from one middle school. These mothers were randomly selected from a list of 129 mothers provided by the principal of the school.” (Baker and Stevenson, 1986, p.157).

Population

Sample

AC

B

D

F

G

IH

K

J

M

L

E

O

N

Q

P

S

R

U

T

ZV

WYX

N

D

P

HL

Y

Simple Random

Stratified Random Sampling

Stratified sampling is a process whereby certain subgroups, or strata, are selected for the sample in the same proportion as they exist in the population.

Example: ”From a pool of all children who returned a parental permission form (more than 80% return rate) 24 first graders (10 girls, 14 boys; mean age, 6 years, 6 months), and 24 third graders (13 girls, 11 boys; mean age, 8 years, 8 months) were randomly selected.” (Clements and Nastasi, 1988, p.93)

Population

Sample

Stratified RandomA B C D E

25%

F G H I J

K L M N O50%

P Q R S T25%

B D 25%

F M O J 50%

P S 25%

Random Cluster Sampling

When it is not possible to select a sample of individuals from a population--for example, a list of all members of the population of interest is not available—cluster sampling is used. It involves the random selection of naturally occurring groups or areas and then the selection of individual elements from the chosen groups or areas.

Population

Sample

QR

AB

Cluster Random

STU

NOP

CD

EFG

LM

JK

HI

EFG

CDQR

Two-Stage Random Sampling

It is often useful to combine cluster sampling with individual sampling. Rather than randomly selecting 200 students from a population of 3000 ninth graders located in 100 classes, the researcher might decide to select 25 classes randomly from the population of 100 classes and then randomly select 8 students from each class.

Population

Sample

QR

ABTwo-Stage Random

STU

NOP

CD

EFG

LM

JK

HI

STU

LMCD

C,L,T

Sample of clusters

Sample of individuals

Convenience Sampling

A convenience sample is a group of individuals who (conveniently) are available for study.

Example:" A high school counselor interviews all of the students who come to her for counseling about their carrier plans.”

Population

Sample

AC

B

D

FG

HK

J

M

E

O

N

PSR

U

T

Z

V

W

LQX

I

Y

Convenience

QX

IY

L

Easily Accessible

Purposive Sampling

In purposive sampling the researcher selects particular elements from the population that will be representative or informative about the topic.

Purposive sampling is different from convenience sampling in that the researcher does not simply study whoever is available, but uses his or her judgment to select the sample for a specific purpose.

Population

Sample

AC

B

D

F

G

IH

K

J

M

L

E

O

N

Q

P

S

R

U

T

ZV

WYX

N

B

V

L

F

Purposive

Example of Purposive Sampling

“Introductory psychology students (N=210) volunteered to take the Dogmatism Scale (Form E) for experimental credit. From the upper and lower quartiles on the Dogmatism Scale, 44 high and 44 low dogmatic subjects were selected for the experiment.” (Rickards and Slife, 1987, pp.636-637)

Systematic Sampling

In systematic sampling every nth element is selected from a list of all elements in the population.

A C D E

F H I J

K M N O

P R S T

Population

SampleB G L

Q

Systematic B

G

L

Q

Measurement

Measures are specific techniques or instruments used for measurements and generally refer to quantitative devices.

These are often tests and questionnaires that provide objective and quantifiable data.

Measurement is an essential component of quantitative research because it provides a standard format for recording observations, performance, or other responses of subjects and because it allows a quantitative summary of the results from many subjects.

The Purpose of Measurement

~To provide information about the variables that are being studied.

In an experiment, the dependent variable is measured.

In correlational research each variable is measured.

In practice, the variable is defined by how it is measured (operational definition), not by how it is labeled or defined by the researcher.

Instrument vs. Instrumentation

An instrument is a device or procedure for systematically collecting information. Common types of instruments include tests, questionnaires, rating scales, checklists, and observation forms.

Instrumentation refers not only to the instrument itself but also to the conditions under which it is used, when it is to be used, and by whom it is to be used.

validity •Validity refers to the extent to which an instrument gives us the information we want.

validity•Validity is a judgment of the appropriateness of a measure for the specific inferences or decisions that result from the scores generated by the measure.

Content-related evidence •refers to the nature of the content included within the instrument, and the specifications the researcher used to formulate the content

Criterion-related evidence •refers to the relationship between scores obtained using the instrument and scores obtained using one or more other instruments or measures (often called criteria)

Construct-related evidence

•refers to the nature of psychological construct or characteristic being measured by the instrument

Types of Evidence for Judging Validity

Reliability refers to the consistency of scores or answers—how consistent they are for each individual from one administration of an instrument to another, and from one set of items to another.

Reliability

A validity coefficient

•expresses the relationship which exists between scores of the same individuals on two different instruments

A reliability coefficient

•expresses a relationship between scores of the same individuals on the same instrument at two different times, or between two forms of the same instrument

Validity and Reliability Coefficients

Methods of Estimating Reliability

Require two Administrations Require One Administration

The Test-Retest Method

The Equivalent Forms Method

Internal Consistency Methods

The Kuder-Richardson Approaches

Split-Half Testing

KR20 KR21

RESEARCH DESIGN

Nonexperimental Research Experimental Research

Weak Experimental Designs:• The One-Shot Case Study Design• The One-Group Pretest-Posttest Design• The Static-Group Comparison Design

Descriptive Studies

Relationship Studiese.g. Simple Correlational

Studies, and Prediction Studies

Causal-Comparative Studies

True Experimental Design• The Randomized Posttest-Only Control Group Design• The Randomized Pretest-Posttest Control Group Design• The Randomized Solomon Four-Group Design

Quasi-Experimental Design• The Matching Only Posttest-Only Control Group Design• The Matching Only Pretest-Posttest Control Group Design

Common Statistical Tests

The t Test

The F Test(ANOVA)

Test for r

Chi-square Test

To compare two means

To test two or more means

To test the significance of a correlation coefficient

To test for relationships involving frequency data in the form of tallies or percentages

Descriptive Studies

A descriptive study simply describes a phenomenon.

Example: “Their initials attributions were primarily task attributions (46% to 58% said the words were easy). Their own effort was the next most common cause of their success (40% of the responses). When asked for a second response, the subjects evenly divided their answers among the four types of attributions.” (Cauley and Murray, 1982, p.476)

Back to research designs

Criteria for Evaluating Descriptive Studies

1. Conclusions about the relationships and causal relationships should not be made.

2. Subjects and instrumentation should be well described.

3. Graphic presentations should not distort the results.

(McMillan, 1992: 146)

Relationship Studies

Relationship studies investigate the degree to which variations or differences in one variable are related to variations or differences in another variable.

Examples:

1. Correlational Studies indicate relationships by obtaining two scores from each subject.

2. A predictive study shows how one variable can predict what the value will be on a second variable at a later time.


Example: Relationship Study of Differences Among Groups

“Advanced level students were more internally responsible for their intellectual-academic failures than general level students. Surprisingly, neither general nor advanced level students were internally responsible for their intellectual-academic failures than the basic level students. (p.320)

(McMillan, 1992: 149)

Example: Predictive Research

“Our final three hypotheses dealt with classroom environment factors…In elementary schools we find that where teachers perceive class size as manageable, the reported level of career dissatisfaction is lower than in elementary schools in which teachers perceive class size as less manageable…. In secondary schools, only the perceived absence of student learning problems…and the perceived absence of student behavior problems…emerged as predictors of teacher career dissatisfactions.” (p.72)

(McMillan, 1992: 153)

Criteria for Evaluating Correlational Studies

1. Causation should not be inferred from correlation.2. The reported correlation should not be higher or

lower than the actual correlation.3. Practical significance should not be confused with

“statistical” significance. 4. The size of the correlation should be sufficient for

the use of the results.5. Prediction studies should report accuracy of

prediction for new subjects.6. Procedures for collecting data should be clearly

indicated.

(McMillan, 1992: 153-156)

Using Surveys in Descriptive and Relationship Studies

In a survey, the researcher selects a group of respondents, collects information (by asking them a number of questions), and then analyzes the information to answer the research questions.

In a Cross-Sectional Survey, information is collected from one or more samples or populations at one time.

In a Longitudinal Survey the same group of subjects is studied over a specified length of time.

Causal-Comparative Study

Ex Post facto Research In Ex Post facto Research the investigators

decide whether one or more preexisting conditions have caused subsequent differences between subjects who experienced different types of conditions (the phrase ex post facto means “after the fact”).


Ex post facto vs. experimental and correlational designs Ex Post facto designs have some similarities with

both experimental and correlational designs. Like an experiment, there is typically a “treatment” and/or “comparison” group, and the results are analyzed with the same statistical procedures. Of course in Ex Post facto Research there is no manipulation of the independent variable because it has already occurred, but the comparison of group differences on the dependent variable is the same. Like correlation studies, no manipulation of the independent variable, so that technically the study is nonexperimental. However, in a correlation two or more measures are taken from each subject, whereas in ex post facto research each subject is measured on the dependent variable.

Correlational Research Correlational research, like causal-comparative

research, is an example of what is sometimes called associational research.

In associational research, the relationships among two or more variables are studied without any attempt to influence them.

In their simplest form, correlational studies investigate the possibility of relationships between only two variables, although investigations of more than two variables are common.

A correlational study describes the degree to which two or more quantitative variables are related, and it does so by use of a correlation coefficient.

Causal-Comparative Study

Similarities and Differences between Causal-Comparative and Correlational Research

Similarities. Both causal-comparative and correlational studies are examples of associational research, that is, researchers who conduct them seek to explore relationships among variables. Both attempt to explain phenomena of interest. Both seek to identify variables that are worthy of later exploration through experimental research, and both often provide guidance for subsequent experimental studies. However, neither permits the manipulation of variables by the researcher.

Similarities and Differences between Causal-Comparative and Correlational Research

Differences. Causal-comparative studies typically compare two or more groups of subjects, while correlational studies require two (or more) scores on each variable for each subject. Correlational studies investigate two (or more) quantitative variables, whereas causal-comparative studies involve at least one categorical variable (group membership). Correlational studies analyze data using scatterplots and/or correlation coefficient, while causal-comparative studies compare averages or use crossbreak tables.

Similarities and Differences between Causal-Comparative and Experimental Research

Similarities. Both causal-comparative and experimental studies typically require at least one categorical variable (group membership). Both compare group performances (average scores) to determine relationships. Both typically compare separate group of subjects.

Differences. In experimental research, the independent variable is manipulated; in causal-comparative research, no manipulation takes place. Causal-comparative studies provide much weaker evidence for causation than do experimental studies. In experimental research, the researcher can sometimes assign subjects to treatment groups; in causal-comparative research, the groups are already formed—the researcher must locate them. In experimental studies, the researcher has much greater flexibility in formulating the structure of the design.

Criteria for Evaluating Causal-Comparative Research

The primary purpose of the research should be to investigate causal relationships when an experiment is not possible.

The presumed causal condition should have already occurred.

Potential extraneous variables should be recognized and considered.

Differences between groups being compared should be controlled.

Causal conclusions should be made with caution.

(McMillan, 1992: 161-162)

Survey Research

A common form of research involving researchers asking a number of questions about a particular topic or issue (often prepared in the form of a written questionnaire or ability test) to a large number of individuals (either by mail, by telephone, etc.).

Survey Research

Cross-sectional

collects information from a sample that

has been drawn from a predetermined population at just

point in time

Longitudinal

Collects information at different points in time in order to study

changes over time

Longitudinal Survey Research

Trends in the same population over time

Changes in a subpopulation group identified by a common characteristic over time

Changes in the same people over time

Cross-sectional Survey Research

Attitudes and Practices

Group Comparisons

National assessment

Program evaluation

Community needs

Weak Experimental Designs

These designs are referred to as “weak” because they do not have built-in controls for threats to internal validity.

Any researcher who uses one of these designs has difficulty assessing the effectiveness of the independent variable.


1. The One-Shot Case Study: a single group is exposed to a treatment or event, and a dependent is subsequently observed (measured) in order to assess the effect of the treatment.

X Otreatment Observation

(dependent variable)


2. The One-Group Pretest-Posttest Design: a single group is measured or observed, not only after being exposed to a treatment of some sort, but also before.

O X Otreatment Posttest

Pretest


3. The Static-Group Comparison Design: Two already existing, or intact, are used. Comparisons are made between groups receiving different treatments.

X1 O

Note:

------ : already formed, not randomly assigned

X1 and X2: different treatments

Os : placed vertically to each other, occurs at the same time

X2 O

True Experimental Designs

Subjects are randomly assigned to treatment groups for controlling the subject characteristics threat to internal validity.

True Experimental Designs

1. The Randomized Posttest-Only Control Group Design: involves two groups, one receives the experimental treatment while the other does not.

Treatment Group R X1 O

Control Group R X2 O

True Experimental Designs2. The Randomized Pretest-Posttest Control Group

Design: both groups are measured twice, the first measurement serves as the pretest, the second as the posttest.

Treatment Group R O X1 O

Control Group R O X2 O

True Experimental Designs3. The Randomized Solomon Four-Group Design:

involves random assignment of subjects to four groups, with two of the groups being pretested and two not. One of the pretested groups and one of the unpretested groups is exposed to the experimental treatment. All four groups are then posttested.

Treatment Group R O X1 O

Control Group R O X2 O

Treatment Group R X1 O

Control Group R X2 O

Quasi-Experimental Designs

Do not include the use of random assignment.

Researchers who employ these design rely instead on other techniques to control (or at least reduce) threats to internal validity.


A. The Matching Only Design: The researcher still matches the subjects in the

experimental and control groups on certain variables, but he/she has no assurance that they are equivalent on others since subjects are not randomly assigned to groups.

The two groups are intact (they are already existed before the intervention) and so are probably not comparable.


1. The Matching Only Posttest-Only Control Group Design

Treatment Group M X1 O

Control Group M X2

O


2. The Matching Only Pretest-Posttest Control Group Design

Treatment Group O M X1 O

Control Group O M X2

O


B. Counterbalanced Designs: Represent another technique for equating experimental and control groups. Each group is exposed to all treatments, however many there are, but in a different order.

Any number of treatments may be involved. Researchers determine the effectiveness of the various treatments simply by comparing

the average scores for all groups on the posttest for each treatment.

Example: A Three-Treatment Counterbalanced Design

Group One X1 O X2 O X3 O

Group Two X2 O X3 O X1 O

Group Three X3 O X1 O X2 O


C. Time-Series Designs:

involves repeated measurements or observations over a period of time both before and after treatment.

O1 O2 O3 O4 X O5 O6 O7

O8


D. Factorial Design: extend the number of relationships that may be examined in an

experimental study allows a researcher to study the interaction of an independent variable

with one or more other variables, sometimes called moderator variables

Treatment Group R O X1 Y1 O

Control Group R O X2 Y1 O

Treatment Group R O X1 Y2 O

Control Group R O X2 Y2 O

Threats to Internal Validity

Mortality

Subject Characteristics

Location

Instrumentation

Testing

History

Maturation

Implementation

Regression

Attitude of Subject

Suggested Readings Butler, Christopher. 1985. Statistics in Linguistics.

New York: Basil Blackwell. Fraenkel, Jack R. and Norman E. Wallen and Jack

and. 1990.How to Design and Evaluate Research in Education. New York: McGraw-Hill, Inc.

McMillan, James H. 1992. Educational Research: Fundamentals for the Consumer. New York: HarperCollinsPublishers.

Suter, W.Newton. 1991.Primer of Educational Research. Boston: Allyn and Bacon.

Singleton, Royce and Bruce Straits. 1999. Approaches to Social Research (3rd Edition). Oxford: Oxford University Press.

Wallen, Norman E. and Jack R.Fraenkel. 1991. Educational Research: A Guide to the Process. New York: McGraw-Hill, Inc.

educational research

Documents

basic research

educational research

correlation studies

actual correlation

reported correlation

independent variable

better readers

specific group of people