1 educ 586educ 586 educational research design. 2 how do we know?how do we know? tenacitytenacity...

1 •EDUC 586EDUC 586

Educational ResearchEducational ResearchDesignDesign

2 •How Do We Know?How Do We Know?

• TenacityTenacity

• AuthorityAuthority

• IntuitionIntuition

• ScienceScience

3 •Research MethodsResearch Methods

• QualitativeQualitative

• QuantitativeQuantitative

4 •Qualitative ResearchQualitative Research

• NaturalisticNaturalistic• PhenomenologicalPhenomenological• Flexible, Evolving Design-InductiveFlexible, Evolving Design-Inductive

• InterviewsInterviews• Document AnalysisDocument Analysis• ObservationObservation

5 •Quantitative ResearchQuantitative Research

• Logical PositivismLogical Positivism

• VariablesVariables

• Quantitative DataQuantitative Data

• Fixed Research DesignFixed Research Design

• Hypothesis Testing-DeductionHypothesis Testing-Deduction

• Statistical AnalysesStatistical Analyses

6 •Scientific MethodScientific Method

• Identify the problemIdentify the problem• Review the literatureReview the literature• Identify variablesIdentify variables• State research questions and hypothesesState research questions and hypotheses• Data collectionData collection• Data analysisData analysis• Interpret the resultsInterpret the results• Draw conclusionsDraw conclusions

7 •Research DesignResearch Design

•““Is it reasonably plausible Is it reasonably plausible that the outcome is created that the outcome is created

by the treatment?”by the treatment?”

Experimental or Quasi-ExperimentalExperimental or Quasi-Experimental

8 •Teaching MethodTeaching Method

1 2 3

X X XAchievement

9•Research Questions (Quantitative)Research Questions (Quantitative)

• 1.1. State a relationship between two or more State a relationship between two or more variablesvariables

• 2.2. Presented in question formPresented in question form• 3.3. Variables can be measured or operationally Variables can be measured or operationally

defineddefined• EXAMPLES:EXAMPLES:• Is there a relationship between self concept and Is there a relationship between self concept and

achievement?achievement?• Do students taught by Method A have higher Do students taught by Method A have higher

achievement than those taught by Method B?achievement than those taught by Method B?

10 •Research QuestionResearch Question

• GeneralGeneral• What is the effect of ability grouping on What is the effect of ability grouping on

achievement?achievement?

• SpecificSpecific• Will students in grouped classes have higher Will students in grouped classes have higher

achievement (attitudes toward subjects) than achievement (attitudes toward subjects) than students in ungrouped classes?students in ungrouped classes?

11 •Hypotheses (Quantitative)Hypotheses (Quantitative)

• 1.1. State a relationship between two or more State a relationship between two or more variablesvariables

• 2.2. Presented in statement formPresented in statement form

• 3.3. Is statistically testableIs statistically testable• EXAMPLES:EXAMPLES:

• There will be no relationship between self concept and There will be no relationship between self concept and achievement.achievement.

• Students taught by Method A will have higher Students taught by Method A will have higher achievement than those taught by Method B.achievement than those taught by Method B.

12•Hypotheses - RelationshipsHypotheses - Relationships

• There will be no significant relationship between There will be no significant relationship between Academic Competence and Social IntegrationAcademic Competence and Social Integration

• There will be no significant relationship betweenThere will be no significant relationship betweenSocial Competence and Social IntegrationSocial Competence and Social Integration

• CorrelationCorrelation

• ScattergramScattergram

• rr22

• Statistical significanceStatistical significance

13 •Hypotheses - DifferencesHypotheses - Differences

• There will be no significant difference between males There will be no significant difference between males and females for:and females for:

• I feel good about my school work.I feel good about my school work.

People like me. People like me.

• There will be no significant difference between special There will be no significant difference between special and regular education students for:and regular education students for:

I feel good about my school work. I feel good about my school work.

Kids like me to be on their teams.Kids like me to be on their teams.

•

14 •Hypotheses - DifferencesHypotheses - Differences

• t-test (2 groups)t-test (2 groups)

• Analysis of Variance (3 or more groups)Analysis of Variance (3 or more groups)

• Analysis of CovarianceAnalysis of Covariance

• Chi-SquareChi-Square

15 •VariablesVariables

• VariableVariable• Any characteristic that varies from individual Any characteristic that varies from individual

to individualto individual

• Measured VariablesMeasured Variables• Vary in degree, amount or quantity Vary in degree, amount or quantity

(e.g., attitude, achievement)(e.g., attitude, achievement)

• Categorical VariablesCategorical Variables• Describe characteristics that are qualitatively Describe characteristics that are qualitatively

different (e.g., gender, eye color, method of different (e.g., gender, eye color, method of teaching)teaching)

16 •Independent, Moderator & Independent, Moderator & Dependent VariablesDependent Variables

Teaching MethodGender

Achievement

MF

1 2 3

High

Med.

Low

Ability


• Independent VariableIndependent Variable• Measured (Ability), manipulated (Method), or Measured (Ability), manipulated (Method), or

selected (Gender) by the experimenter to selected (Gender) by the experimenter to determine its relationship to an observed determine its relationship to an observed phenomenon (Achievement)phenomenon (Achievement)

• Dependent VariableDependent Variable• Observed and measured to determine the effect Observed and measured to determine the effect

of the independent variable, i.e., that factor that of the independent variable, i.e., that factor that appears, disappears or varies as the appears, disappears or varies as the experimenter introduces, removes or varies the experimenter introduces, removes or varies the independent variableindependent variable


• Moderator VariableModerator Variable• Measured (Ability), or selected (Gender) by the Measured (Ability), or selected (Gender) by the

experimenter to determine whether it modifies the experimenter to determine whether it modifies the relationship of the independent variable to the relationship of the independent variable to the dependent variable. It is a special type of dependent variable. It is a special type of independent variable.independent variable.

• Control VariableControl Variable• Controlled by the experimenter to cancel out or Controlled by the experimenter to cancel out or

neutralize any effect they might otherwise have on neutralize any effect they might otherwise have on the observed phenomenon (Grade 9)the observed phenomenon (Grade 9)

• Extraneous VariableExtraneous Variable• Independent variable not controlled by experimenterIndependent variable not controlled by experimenter

19 •Review of LiteratureReview of Literature

• PurposesPurposes• Limit/identify research problem/hypotheses/gapsLimit/identify research problem/hypotheses/gaps

• What has already been doneWhat has already been done

• Possible design/methodology issuesPossible design/methodology issues

• Provide empirical/judgmental basis for development Provide empirical/judgmental basis for development of testable hypotheses (quantitative) or working of testable hypotheses (quantitative) or working hypotheses (qualitative)hypotheses (qualitative)

• Provide basis for interpreting resultsProvide basis for interpreting results

20

Sample Problem StatementsSample Problem Statements


• Form/OrganizationForm/Organization

• Introduction/OverviewIntroduction/Overview

• SummarySummary

• LabelsLabels

22•Synthesizing Research: Types of ErrorsSynthesizing Research: Types of Errors

• Unexplained selectivityUnexplained selectivity• Lack of discriminationLack of discrimination• Erroneous detailingErroneous detailing• Double countingDouble counting• Nonrecognition of faulty author conclusionsNonrecognition of faulty author conclusions• Unwarranted attributionsUnwarranted attributions• Suppression of contrary findingsSuppression of contrary findings• Consequential errorsConsequential errors• Failure to find all evidence for generalizationFailure to find all evidence for generalization


• Wilson Education AbstractsWilson Education Abstracts

• ERICERIC

• CIJE (EJ###-###)CIJE (EJ###-###)• Documents (ED###-###)Documents (ED###-###)

• UF (used for)UF (used for)

• NT (narrower term)NT (narrower term)

• BT (broader term)BT (broader term)

• RT (related term)RT (related term)

24Review of LiteratureReview of Literature

• ERIC ThesaurusERIC Thesaurus

• ProQuest DirectProQuest Direct

• Psychological AbstractsPsychological Abstracts

• Social Sciences Citation IndexSocial Sciences Citation Index

• Dissertation AbstractsDissertation Abstracts

• The CATThe CAT

25Review of LiteratureReview of Literature

AutismAutism

Reading ComprehensionReading Comprehension

Intersection Of TwoIntersection Of Two(or more) Sets(or more) Sets

26 •Types of Literature AnalysesTypes of Literature Analyses

• Traditional ReviewTraditional Review

• Meta Analysis (Glass)Meta Analysis (Glass)• Effect SizeEffect Size

• Best-Evidence Synthesis (Slavin)Best-Evidence Synthesis (Slavin)• High internal and external validityHigh internal and external validity

• Specified inclusion criteriaSpecified inclusion criteria

• Use effect size and qualitative analysisUse effect size and qualitative analysis

27 •Review of Literature LabelsReview of Literature Labels

• Section Title Section Title (centered)(centered)

• Paragraph Title Paragraph Title (left margin)(left margin)

• Paragraph Label. Paragraph Label. (indented)(indented)

• References (APA Format)References (APA Format)

28 •Types of ResearchTypes of Research

• HistoricalHistorical• DescriptiveDescriptive

• SurveySurvey• QualitativeQualitative

• CorrelationalCorrelational• Causal-ComparativeCausal-Comparative• Quasi-ExperimentalQuasi-Experimental• ExperimentalExperimental

29 •Historical ResearchHistorical Research

• ““What Was”What Was”

• Study the past to better understand Study the past to better understand and predict the futureand predict the future

• Examples:Examples:• Shaker EducationShaker Education• American Guidance AssociationAmerican Guidance Association• 9th Grade Proficiency Test9th Grade Proficiency Test• CT Legislature Support for EducationCT Legislature Support for Education

30 •Descriptive ResearchDescriptive Research

• ““What Is”What Is”

• Describe the existing situationDescribe the existing situation

• Examples:Examples:• Needs AssessmentNeeds Assessment• Special Education AttitudesSpecial Education Attitudes• Graduate Follow-UpGraduate Follow-Up

31 •Survey ResearchSurvey Research

• Things You Must KnowThings You Must Know

• Survey DevelopmentSurvey Development

• Characteristics of the SampleCharacteristics of the Sample

• Distribution and Follow-Up ProceduresDistribution and Follow-Up Procedures

• Return Rate Issues/ActionsReturn Rate Issues/Actions


• Demographic QuestionsDemographic Questions

• Item ContentItem Content

• Response FormatResponse Format

• Written CommentsWritten Comments• Thematic AnalysisThematic Analysis

• Valence (+, - )Valence (+, - )

• Self-CodingSelf-Coding

• Layout and PrintingLayout and Printing

Survey Development - FormSurvey Development - Form


• Personal Request LetterPersonal Request Letter• Pre-contactPre-contact

• Authority FigureAuthority Figure

• IncentivesIncentives

• LengthLength

• Time of YearTime of Year

Survey Development - ConsiderationsSurvey Development - Considerations


• Education LevelEducation Level

• Group Identification with IssueGroup Identification with Issue

• Knowledge of SubjectKnowledge of Subject

• Interest in SubjectInterest in Subject

• Perceived ImportancePerceived Importance

Characteristics of the SampleCharacteristics of the Sample


• Time of YearTime of Year

• CostCost• MaterialsMaterials• PostagePostage

• Distribution MethodDistribution Method

• Return MethodReturn Method

• Follow-Up ProceduresFollow-Up Procedures

Distribution and Follow-UpDistribution and Follow-Up


• Response-Rate BiasResponse-Rate Bias

• Sample vs. Population DemographicsSample vs. Population Demographics• LeslieLeslie• Murphy, Gable, OwenMurphy, Gable, Owen

• Analysis of ReturnsAnalysis of Returns• Trend AnalysisTrend Analysis• Sample Non-RespondentsSample Non-Respondents

Return Rate Issues/ActionsReturn Rate Issues/Actions

37 •Correlational ResearchCorrelational Research

• Measure the degree of relationship between two Measure the degree of relationship between two or more variablesor more variables

• Examples:Examples:• Construct Validity (Correlations, Factor Analysis)Construct Validity (Correlations, Factor Analysis)

• Economic and Organizational FactorsEconomic and Organizational Factors

• Feedback Seeking and Managerial EffectivenessFeedback Seeking and Managerial Effectiveness

• Criterion - Related Validity (Attitudinal Predictors Criterion - Related Validity (Attitudinal Predictors of Dropping Out of School)of Dropping Out of School)

““What relationship exists?”What relationship exists?”

38 •Causal-Comparative Causal-Comparative (Ex-Post-Facto)(Ex-Post-Facto)

• Independent Independent variable(s) have already variable(s) have already occurredoccurred

• Start with observation of the Start with observation of the dependentdependent variablevariable

• Study Study independentindependent variable(s) in variable(s) in retrospect for possible relation to the retrospect for possible relation to the dependentdependent variable variable

““Attempt to deduce Attempt to deduce howhow?”?”

39 •Causal ComparativeCausal Comparative

GradeGender

MF

4 6 7

Adv.

Disadv.

School

5 8

““Self-Perception of Culturally Disadvantaged Children”Self-Perception of Culturally Disadvantaged Children”Soares and SoaresSoares and Soares

40 •Quasi-ExperimentQuasi-Experiment

• Field Study - Can’t do a true experimentField Study - Can’t do a true experiment

• No random assignmentNo random assignment

• Can not control all relevant variablesCan not control all relevant variables

• ExampleExample• Material Incentives ProjectMaterial Incentives Project

““Attempt to deduce Attempt to deduce howhow and and whywhy””

41 •ExperimentExperiment

• Examine cause and effectExamine cause and effect

• Random assignmentRandom assignment

• ““Manipulation” of Manipulation” of independent independent variablevariable

• Observe effect on Observe effect on dependentdependent variable variable• ExamplesExamples

• Alternate school projectAlternate school project• Project ConcernProject Concern• Police attitudesPolice attitudes

““Determine Determine howhow and and whywhy””


• SelectionSelection• Low Attitude (Grade 8)Low Attitude (Grade 8)

• Teacher RecommendationsTeacher Recommendations

• Student/Parent VolunteerStudent/Parent Volunteer

• High Absent/Low GradesHigh Absent/Low Grades

• Random SelectionRandom Selection

• Participants/Non-ParticipantsParticipants/Non-Participants

Alternate School ProjectAlternate School Project


• Check Random AssignmentCheck Random Assignment• AbsentAbsent

• SuspensionsSuspensions

• Grades (Math, English)Grades (Math, English)• ProgramProgram

• AcademicAcademic

• OccupationalOccupational

Alternate School ProjectAlternate School Project


• BehavioralBehavioral• AbsentAbsent• SuspensionsSuspensions• Drop Out RateDrop Out Rate

• CognitiveCognitive• AchievementAchievement

• AffectiveAffective• McCook Attitude ScaleMcCook Attitude Scale

• OccupationalOccupational• KnowledgeKnowledge• AspirationAspiration

Alternate School Project - Dependent VariablesAlternate School Project - Dependent Variables

45 •SamplingSampling

• PopulationPopulation• The target group of interestThe target group of interest

• All individuals who possess a certain characteristicAll individuals who possess a certain characteristic

• High school teachersHigh school teachers

• PrincipalsPrincipals

• 4th grade students4th grade students• SampleSample

• Any subgroup on which information is obtainedAny subgroup on which information is obtained


• Does the sample represent the population?Does the sample represent the population?

• Target PopulationTarget Population

• Accessible PopulationAccessible Population

• SampleSample


• Why Sample?Why Sample?

• Save TimeSave Time

• Save MoneySave Money

• Impossible to work with the Impossible to work with the populationpopulation


• Types of SamplesTypes of Samples• RandomRandom

• SimpleSimple• StratifiedStratified• ClusterCluster

• Non-RandomNon-Random• SystematicSystematic• ConvenienceConvenience• PurposivePurposive


• Sample SizeSample Size

• Use of tablesUse of tables

• BiasBias

• PrecisionPrecision

50 •ValidityValidity

• Are the inferences made from test Are the inferences made from test scores appropriate, meaningful and scores appropriate, meaningful and useful?useful?

• Does the accumulated evidence Does the accumulated evidence (judgmental and empirical) support the (judgmental and empirical) support the inferences?inferences?

• Inferences are validated, not the testInferences are validated, not the test


• Types of ValidityTypes of Validity

• ContentContent

• ConstructConstruct

• Criterion-RelatedCriterion-Related


• Content ValidityContent Validity

• To what extent do the items on the test To what extent do the items on the test adequately sample from the intended adequately sample from the intended universe of content?universe of content?

• Judgmental EvidenceJudgmental Evidence• Literature ReviewLiterature Review

• Expert ReviewExpert Review


• Construct ValidityConstruct Validity• To what extent do certain explanatory concepts To what extent do certain explanatory concepts

explain:explain:• covariation in the responses to the items, orcovariation in the responses to the items, or• relationships with other know indicators?relationships with other know indicators?

• Empirical evidenceEmpirical evidence• CorrelationsCorrelations• Factor analysisFactor analysis• Structural equation modelingStructural equation modeling• Item response theoryItem response theory• Known groupsKnown groups


• Criterion-Related ValidityCriterion-Related Validity

• What is the relationship between scores on the What is the relationship between scores on the instrument and some external criterion that instrument and some external criterion that provides a more direct measure of the targeted provides a more direct measure of the targeted characteristic?characteristic?

• TypesTypes• ConcurrentConcurrent

• PredictivePredictive

55 •ReliabilityReliability

•““Does the instrument Does the instrument provide us with accurate provide us with accurate

assessments?”assessments?”


• TheoryTheory

•XXTotalTotal = X = XTrueTrue + X + XErrorError

•VVTotalTotal = V = VTrueTrue + V + VErrorError

• Reliability = Reliability = VVTrueTrue

VVTotalTotal


• TheoryTheory

•rr1122

TrueTrue = = ReliabilityReliability

• “ “Proportion of variance in Proportion of variance in scores that can be considered scores that can be considered

‘true’ variance”‘true’ variance”


• Sources of ErrorSources of Error

• Inadequate sampling of itemsInadequate sampling of items• Cronbach’s alpha internal Cronbach’s alpha internal

consistency reliabilityconsistency reliability

• Different occasionsDifferent occasions• Stability reliabilityStability reliability


• Types of ReliabilityTypes of Reliability

• Internal consistencyInternal consistency

• StabilityStability


•Generalizability TheoryGeneralizability Theory

•““Simultaneously examines Simultaneously examines homogeneity of the items and homogeneity of the items and

stability of responses over time”stability of responses over time”


•Procedures which produce Procedures which produce descriptive data -- peoples’ descriptive data -- peoples’ written or spoken word, or written or spoken word, or observations of behaviorobservations of behavior


• Ethnography - Naturalistic InquiryEthnography - Naturalistic Inquiry

• Study behavior in a natural settingStudy behavior in a natural setting

• Involvement with subjectsInvolvement with subjects

• See the world through their eyesSee the world through their eyes


• Selecting a Research Selecting a Research StrategyStrategy

• QualitativeQualitative

• QuantitativeQuantitative


•Philosophical BasePhilosophical Base Logical PositivismLogical Positivism PhenomenologyPhenomenology

•PurposePurpose VerificationVerification DiscoveryDiscovery

•Framework/DesignFramework/Design Preordinate/FixedPreordinate/Fixed Emergent/FlexibleEmergent/Flexible

•ConditionsConditions ControlledControlled Invited InterferenceInvited Interference

•TreatmentTreatment StableStable VariableVariable

Quantitative Quantitative QualitativeQualitative

65 •Research TargetResearch Target

• ProcessProcess ProductProduct

• ImplementationImplementationOutcomeOutcome

• ““Implementation as an Implementation as an independent variable”independent variable”


• Front StageFront Stage• Packaged RealityPackaged Reality

• Back StageBack Stage• RealityReality

• GatekeepersGatekeepers

““Schools are not researchers’ sandbox”Schools are not researchers’ sandbox”

Access and EntryAccess and Entry


• InterviewInterview• What is What is saidsaid

• Document AnalysisDocument Analysis• What is What is writtenwritten

• Participant ObservationParticipant Observation• What is What is donedone

Data CollectionData Collection


• Combination of Methodologies Combination of Methodologies • Multiple Reference PointsMultiple Reference Points

• Increase AccuracyIncrease Accuracy• Increase ConfidenceIncrease Confidence

• Convergence of EvidenceConvergence of Evidence• Among MethodsAmong Methods• Within MethodsWithin Methods

TriangulationTriangulation


• OngoingOngoing At conclusionAt conclusion

• InductiveInductive DeductiveDeductive• Generating TheoryGenerating Theory Testing TheoryTesting Theory

QualitativeQualitative QuantitativeQuantitative

--

Data AnalysisData Analysis


• Reporting ResultsReporting Results• Natural HistoryNatural History

• Thematic AnalysisThematic Analysis

• Social NetworksSocial Networks

• Life HistoryLife History

• Allocation and Distribution of ResourcesAllocation and Distribution of Resources

• Organizational AnalysisOrganizational Analysis

• Critical IncidenceCritical Incidence


• Development of EducationDevelopment of EducationResearch MethodsResearch Methods

• Genetics Genetics • Correlation (Galton)Correlation (Galton)

• AgricultureAgriculture• Controlled experiments (Fisher - ANOVA)Controlled experiments (Fisher - ANOVA)


• Complexity of Educational Complexity of Educational ProcessesProcesses

• Experimental research: failure of the Experimental research: failure of the

promisepromise

• Trend toward qualitative methodsTrend toward qualitative methods


• Experimental and Quasi-Experimental Designs for Experimental and Quasi-Experimental Designs for Research Research (1963)(1963)

• Donald CampbellDonald Campbell• Julian StanleyJulian Stanley

• Shift focus from statistics to problems of acquiring Shift focus from statistics to problems of acquiring data in educationdata in education

• Develop benchmarks for research designsDevelop benchmarks for research designs

• Compare research designs with respect to benchmarksCompare research designs with respect to benchmarks


• Research Design FocusResearch Design Focus

• 1960’s - 1970’s: Experimental1960’s - 1970’s: Experimental

• The External Validity of ExperimentsThe External Validity of ExperimentsBracht & Glass, 1968Bracht & Glass, 1968

• 1980 +: Quasi-Experimental1980 +: Quasi-Experimental

• Quasi-Experimentation: Design and Analysis Issues for Quasi-Experimentation: Design and Analysis Issues for Field SettingsField SettingsCook & Campbell, 1979Cook & Campbell, 1979


•Internal ValidityInternal Validity

•To what extent have extraneous variables been To what extent have extraneous variables been controlled so that the observed effect (dependent) controlled so that the observed effect (dependent) can be attributed to the treatment variable can be attributed to the treatment variable (independent)?(independent)?

•Is it reasonably plausible that the outcome Is it reasonably plausible that the outcome (dependent) is created by the treatment (dependent) is created by the treatment (independent)?(independent)?


• Threats to External ValidityThreats to External Validity

• Results generalizable to other:Results generalizable to other:

• People?People?• Settings?Settings?• Independent variables?Independent variables?• Dependent variables?Dependent variables?


• Internal Validity ThreatsInternal Validity Threats

• TestingTesting

• InstrumentationInstrumentation

• Statistical regressionStatistical regression

• Differential selectionDifferential selection

• Experimental treatment diffusionExperimental treatment diffusion


• Internal Validity ThreatsInternal Validity Threats

• Compensatory rivalry by the control Compensatory rivalry by the control groupgroup

• Compensatory equalization of Compensatory equalization of treatmentstreatments

• Resentful demoralization of the control Resentful demoralization of the control groupgroup

79

One Group Pre-PostOne Group Pre-Post

X

•Research DesignResearch Design

80

Pre-Post Comparison GroupPre-Post Comparison Group

X

•Research DesignResearch Design

81

Pre-Post Comparison GroupPre-Post Comparison Groupwith Random Assignmentwith Random Assignment

XRR

Research DesignResearch Design

82

Post-Only Comparison GroupPost-Only Comparison Groupwith Random Assignmentwith Random Assignment

XRR

Research DesignResearch Design

83 •External Validity of ResearchExternal Validity of Research

• ““Generalizability”Generalizability”

• Population ValidityPopulation Validity• Generalize from sample to population?Generalize from sample to population?

• Personalogical variables interact with Personalogical variables interact with treatment?treatment?

84 •External Validity of ResearchExternal Validity of Research

• ““Generalizability”Generalizability”

• Ecological ValidityEcological Validity• Explicit description of treatmentExplicit description of treatment• Multiple treatment interferenceMultiple treatment interference• Hawthorne effectHawthorne effect• Novelty and disruption effectsNovelty and disruption effects• Experimenter effectExperimenter effect• Pretest sensitizationPretest sensitization• Measurement of dependent variableMeasurement of dependent variable

85 •Statistical AnalysisStatistical Analysis

Experimental

Xchange

ComparisonChange Scores

Post Pre = Change Post Pre = Change

Xchange


Repeated Measures ANOVARepeated Measures ANOVARMANOVARMANOVA

ExperimentalExperimental

ComparisonComparison

PrePre PostPost


RMANOVARMANOVASources of VarianceSources of Variance

BetweenBetweenGroupGroup FFgroupgroup

Error (b)Error (b)

WithinWithinTimeTime FFtimetime

Time x GroupTime x Group FFtime x grouptime x group

Error (w)Error (w)

TotalTotal


Research Question

Will there be a difference between groups Will there be a difference between groups with respect to posttest scores after with respect to posttest scores after controlling for initial differences in controlling for initial differences in pretest scores?pretest scores?

Analysis of CovarianceAnalysis of CovarianceANCOVAANCOVA

89

Pre CovariatePre Covariate

PostPostDependentDependent

ExpExp

ControlControl

89

90

Pre CovariatePre CovariatePÕST = b(PRE) + aPÕST = b(PRE) + a


ExpExp

ControlControl

90

91



ExpExp

ControlControl

XXContCont XXExpExp

YYCC

YYEE

MarginalMarginalDistributionDistribution

91

92



ExpExp

ControlControl

XXContCont XXExpExp

YYCC

YYEE

MarginalMarginalDistributionDistribution

XX

92

93 •Effect SizeEffect Size

““The difference between two means expressed The difference between two means expressed in standard deviation units”in standard deviation units”

ES =ES = Mean 1 - Mean 2Mean 1 - Mean 2Standard DeviationStandard Deviation

ESES Qualitative DifferenceQualitative Difference

.20.20 SmallSmall

.50.50 MediumMedium

.80.80 LargeLarge

94 •Statistical Significance: Statistical Significance: Sample Size and Effect SizeSample Size and Effect Size

““I have a meaningful say in designing my job/work.”*I have a meaningful say in designing my job/work.”*

StandardStandard Sign. Sign. EffectEffect Group Group NN Mean Mean DeviationDeviation LevelLevel SizeSize

.000.000 .22.22MalesMales 10,74710,747 3.003.00 1.201.20

FemalesFemales 8,0888,088 3.263.26 1.191.19

MalesMales 111111 3.033.03 1.171.17

FemalesFemales 8181 3.283.28 1.221.22.157.157 .21.21

* Responded to on a 5-point Likert agreement scale.* Responded to on a 5-point Likert agreement scale.

1 educ 586educ 586 educational research design. 2 how do we know?how do we know? tenacitytenacity...

Documents

significant relationship

method b

method of teachingindependent

manipulated method

higher achievement attitudes

specificwill students

significant difference

social competence