-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
1/43
EXPERIMENTALRESEARCH
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
2/43
A very common way of research design
A good design to establish cause and effectrelationships among variables
It is the only type of research that
directly attempts to influence a particular variable
tests hypotheses about cause-and-effectrelationships
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
3/43
Major Characteristics of
Experimental Research
The researcher looks at the effect(s) of at leastone independent variable (experimental ortreatment variable) on one or more dependentvariables (criterion or outcome variable)
The researcher manipulatesthe independentvariable, which is not the case in other researchdesigns
Aim of the researcher is to see whether thetreatment/manipulation has made a difference.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
4/43
Some independent variables: Methods of instruction
Types of assignment
Learning materials
Types of questions asked by teachers
Some dependent variables: Achievement
Interest
Motivation Attitudes
Now, write possible research questions and hypothesesconsidering these variables.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
5/43
Comparison of Groups
Experimental research involves two groups ofsubjects
Experimental group
Control / comparison group
Experimental groupreceives the treatment;control groupreceives no treatment /comparison
groupreceives a different treatment
Comparison group rather than control group ineducational research
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
6/43
Manipulation of the
Independent Variable
The researcher actively manipulates theindependent variable: s/he decides the natureof the treatment, to whom and to what extent it
will be applied, and when, where and how itwill be done
Variables thatcanbe manipulated: teachingmethod, learning activities, materials, etc
Variables thatcannotbe manipulated: gender,ethnicity, age, etc
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
7/43
Ways of establishing
independent variables
A) one form of the variable versus another E.g. comparing the inquiry method vs the lecture
method in teaching listening
B) presence versus absence of a particular form E.g. comparing the use of videos vs no videos in
teaching speaking
C) varying degrees of the same form Comparing the effects of different amount of
homework on students motivation
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
8/43
Control of Variables
In experimental research, the researcher has thechance to control the variables for internal validitypurposes.
It is important for the researcher to be sure that NOTOTHER variables have affected the result.
E.g. studying students attitudes towards English: ifyou dont make sure that the groups are equivalent in
ability, then ability might be a reason for the cause.
There are some ways to control variables andminimize threats to internal validity.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
9/43
a) Randomization
Random assignment: assigning the subjects togroups
Points to consider
Takes place before the experiment starts It is a process of assigning individuals to groups
It allows the researcher to assign two equivalentgroups at the beginning of the study
Equivalence is not possible if there are notenough number of subjects in a group (norules for the number, but generally not fewerthan 40 in a group)
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
10/43
b) Holding certain variables
constant
To eliminate its possible effects, the researchercan remove a variable from the study.
E.g. if gender is considered to influence thestudy, you can restrict the subjects to femalesor males only.
In that case, generalization is reduced!
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
11/43
c) Building the variable into the
design
Opposite of the previous idea
E.g. The researcher includes both females andmales (as distinct groups) and analyze theeffects of both gender and method onoutcomes
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
12/43
d) Matching
Pairs of subjects can be matched on certainvariables of interest.
E.g. if age is considered to be an effect, theresearcher can assign one member of eachage group to each of the comparison groups
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
13/43
e) Using subjects as their own
control
Using the same subjects for all the treatments
E.g.
teaching the same students by an inquiry methodand later by a lecture method
Assessing behavior during a period of time before
and after a treatment to see whether changes inbehavior occur
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
14/43
f) Using analysis of covariance
Equating groups statistically on the basis of apretest or other variables. The posttest scores
are adjusted accordingly
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
15/43
Internal Validity
Possible threats to internal validity
A) Selection Bias (subject characteristics: Might be differences in groups (age, gender,
intelligence)
B) Loss of Subjects (mortality) Losing some of the subjects (e.g. some subjects may
not return the questionnaires)
C) Location Place in which data are collected or an
implementation is conducted may affect the study
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
16/43
D) Instrumentation
instrument decay (e.g. essay scoring causes fatigueand can be score differently)
data collector characteristics (age, gender, experience
of data collector) data collector bias (e.g. if the collector knows the
hypotheses, s/he might unconsciously distract thedata)
E) Testing
Subjects can remember the test if the same is used inpre and post test at a short interval
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
17/43
F) History personal experiences during the course of the
study
G) Maturation E.g. young learners will change due to maturation
H) Attitude of Subjects The way subjects view the study (e.g.
experimental group might perform better becausethey know they are having a special treatment)
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
18/43
I) Regression
(e.g. if you include all good students in one group,they will improve no matter what you do)
J) Implementation
The researcher can give the experimental group
an advantage
Group Designs in Experimental
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
19/43
Group Designs in ExperimentalResearch:
WEAK Experimental Designs The One-Shot Case Study: A single group is exposed to a treatment and a
dependent variable is observed to assess the effect ofthe treatment
X O
treatment observation Why weak?
Absence of control: the researcher does not knowwhether the result is due to the treatment
Not possible to compare so the researcherdoesnt know whether the treatment had anyeffect at all
To remedy this, a comparison can be made with
another group who had the same course content
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
20/43
The One-Group Pretest-Posttest Design: A single group is observed before and after treatment
O X O
Pretest treatment posttest
Better than one-shot case study (we know whetherchange occurred), but it is still weak. Why?
Threats to internal validity: history, maturation,instrument decay, data collector characteristics, datacollector bias, testing, regression, attitude of subjectsand implementation (The researcher does not know ifthe difference is due to the treatment or any of these!)
To remedy this, a comparison group which does nothave the treatment can be added.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
21/43
The Static-Group Comparison Design Two already existing groups (nonequivalent) are
usedX O..
O
Although it provides better results, it might stillhave problems. Why?
Some control on testing, regression threats; alittle control on history and maturation;problems of control on mortality and location
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
22/43
The Static-Group Pretest-Posttest Design Similar to static group comparison. The only
difference is to give a pretest to both groupsO X O
O O While analyzing, the pretest score is subtracted
from the posttest score, so change in eachstudent is analyzed.
Some possible threats? Subject characteristics (group scoring higher on
the pretest is likely to improve more)
Testing threat (pre and post test)
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
23/43
True Experimental Designs
The Randomized Posttest-Only Control GroupDesign Involves two groups which are formed by random
assignment: one receives the treatment; the other does not
Treatment group R (random) X O
Control group R (random) C O Control of some threats is excellent: subject
characteristics, maturation, regression
No testing threat because subjects are not testedtwice
Possible threats: Mortality (if more subjects drop out in one, then results
can be affected; the number of the subjects whodropped out should be reported)
Implementation, data collector bias, location andhistory threats may also exist
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
24/43
The Randomized Pretest-Posttest ControlGroup The difference is in using both pre and post test
for both groups
Treatment group R O XO
Control group R O X O
This design helps the researcher see if therandom assignment actually succeeded(pretest).
This is particularly important
if the number in each group is small
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
25/43
The Randomized Solomon Four-Group Design Used to eliminate the possible effect of a pretest
Treatment group R O X O
Control group R O C O
------------------------------------------------Treatment group R X O
Control group R C O
The first group is like pretest-posttest control group;the second is posttest only control group
Best control of threats Weakness: requires a large sample (subjects are
assigned to four groups); requires a lot of energy andeffort
QUASI EXPERIMENTAL
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
26/43
QUASI-EXPERIMENTALDESIGN
It is like experimental design but it doesnt userandom assignment.
It uses other techniques to control threats
There are many techniques used in this designbut the most commonly used one in socialresearch is nonequivalent groups design
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
27/43
Nonequivalent groups design
It is like pretest-posttest randomized
experiment but does not use randomassignment.
Instead similar groups are used as the
treatment and the control groups E.g. you can pick up two comparable
classrooms or schools.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
28/43
CONCLUSION
In any type of experimental design, we create twogroups that are equivalent to each other.
One group gets the treatment; the other does not.
In all other respects the groups are treated the
same. If we observe differences in the outcome between
these groups, then we believe that the differenceis due to the treatment.
However, in all these designs, we are setting upan artificial situation, so we have to consider allthe situation.
If the situation is right, then an experiment can bea very strong design to use.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
29/43
EXERCISES
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
30/43
Exercise 1. A researcher uses morning and afternoon historyclasses to test a new instructional strategy. In the morningclass the students are given a pretest to assess theirknowledge of WWI and then they are given the opportunity tolearn about this topic using a computerized program. In theafternoon class the students are also given a pretest; theylearn about WWI through discussion. At the end of six weeks,a final exam is given to both classes.
1. a) How is the research designed?
Quasi-experimental (non-equivalent pre-test/post-test)Gr. A (NR) O X1 O
Gr. B (NR) O X2 O
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
31/43
A researcher uses morning and afternoon history classes totest a new instructional strategy. In the morning class thestudents are given a pretest to assess their knowledge ofWWI and then they are given the opportunity to learn aboutthis topic using a computerized program. In the afternoonclass the students are also given a pretest; they learn aboutWWI through discussion. At the end of six weeks, a finalexam is given to both classes.
1. b) What are possible threats to internal validity?
Selection bias?
Location?
Instrumentation (data collector bias)?
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
32/43
A researcher uses morning and afternoon historyclasses to test a new instructional strategy. In themorning class the students are given a pretest toassess their knowledge of WWI and then they are
given the opportunity to learn about this topic using aprogram. In the afternoon class thestudecomputerized nts are also given a pretest; theylearn about WWI through discussion. At the end of sixweeks, a final exam is given to both classes.
1. c) How can you analyze the data?
t-test (comparison of the means in pre- and post-testsfor both groups)
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
33/43
Exercise 2. A social studies teacher isinterested in determining the effects of a newtextbook series, which has a multicultural
thematic approach. The teacher will teach aunit on colonization in America. After the unit,a multicultural attitude scale will beadministered to students.
2. a) How is the research designed?
one-shot case study
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
34/43
A social studies teacher is interested indetermining the effects of a new textbookseries, which has a multicultural thematic
approach. The teacher will teach a unit oncolonization in America. After the unit, amulticultural attitude scale will be administeredto students.
2. b) What are possible threats to internalvalidity?
Almost no control
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
35/43
A social studies teacher is interested indetermining the effects of a new textbookseries, which has a multicultural thematic
approach. The teacher will teach a unit oncolonization in America. After the unit, amulticultural attitude scale will be administeredto students.
2. c) How can you analyze the data?
Descriptive statistics
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
36/43
Exercise 3 A school administration wants to field-test aMethodology curriculum in the ELT course that is offered inthe summer. Two classes are to be offered during thesummer session. Students in both classes will be given apretest to measure their knowledge of Methodology. Oneclass will study Methodology using traditional textbookmaterials with a mostly lecture-style of delivery. Anotherclass, an experimental class, will study Methodology usingfilms and contemporary materials along with presentations byguest speakers. The final exam for the course, a parallelform of the pretest, will be given to both groups.
3. a) How is the research designed?
Quasi-experimental (pre-test/post-test nonequivalentcomparison groups)
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
37/43
A school administration wants to field-test a Methodologycurriculum in the ELT course that is offered in the summer.Two classes are to be offered during the summer session.Students in both classes will be given a pretest to measuretheir knowledge of Methodology. One class will studyMethodology using traditional textbook materials with amostly lecture-style of delivery. Another class, anexperimental class, will study Methodology using films andcontemporary materials along with presentations by guestspeakers. The final exam for the course, a parallel form ofthe pretest, will be given to both groups.
3. b) What are possible threats to internal validity?
Regression?
Selection bias?
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
38/43
A school administration wants to field-test aMethodology curriculum in the ELT course that isoffered in the summer. Two classes are to be offeredduring the summer session. Students in both classeswill be given a pretest to measure their knowledge of
Methodology. One class will study Methodology usingtraditional textbook materials with a mostly lecture-style of delivery. Another class, an experimentalclass, will study Methodology using films andcontemporary materials along with presentations by
guest speakers. The final exam for the course, aparallel form of the pretest, will be given to bothgroups.
3. c) How can you analyze the data?
t-test between and within groups
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
39/43
Exercise 4 With the current emphasis on parentalinvolvement in childrens academic endeavors, aresearcher is interested in assessing theconfidence level of parents who will read orally to
children in schools. He also wants to know ifthere is a difference in the confidence level ofparents of different education levels. From a listof volunteers who are interested in reading orallyto students, parents will be randomly selected and
assigned to one of two conditionstraining or notraining. Following training of parents in theexperimental condition, all parent volunteers willcomplete a confidence scale. Describe this studyusing a true experimental posttest-only control
group design.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
40/43
Random assignment for two groups: give anumber to each and draw out a number for GroupA and then Group B until you finish all thenumbers.
Indep variable: training parents
Dep variable: parents confidence level
Group A will get training whereas Group B will not.Parents in both groups will read orally to childrenand then they will be given a confidence scale. T-test for the mean scores obtained from this scalefor each group will be found out to see the effectof training on confidence level.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
41/43
Exercise 5 The dean of a college of education is interested indetermining if the education majors who take an optionalcourse in working with students with disabilities are using thestrategies learned in the course. He decides to study thisissue. Fifty education majors will be randomly assigned totake the course when they enter as freshman. The controlgroup of fifty education majors will be assigned a multiculturaleducation course. The junior field experience will occur priorto the courses, thus allowing monitoring for strategies prior totaking the course. Students will again be monitored for useof strategies during student teaching. In both the pre-courseand post-course field experiences, students will each teachten lessons in their classrooms. These lessons will bevideotaped. A researcher will view the videotapes andobserve the lessons to determine which strategies are used.Describe this study using a true experimental, pretest-posttest control group design.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
42/43
For the following exercises, work in pairs anddescribe the research design answeringfollowing questions:
1. How will the subjects be assigned togroups?
2. Identify relevant variables
3. Describe the treatment design 4. Identify the way the data will be analyzed.
-
8/2/2019 1330105791.13392.Webmail191.Mynet.com 8778184033 Quantitative-experimental
43/43
Please read a few article which used
experimental research method!!