Research and Studies Research and Studies in Educational in Educational

TechnologyTechnology

Dr. Mohamed El Tahir OsmanDr. Mohamed El Tahir OsmanAssistant Dean for Research & Assistant Dean for Research &

Post Graduate StudiesPost Graduate Studiesmosman@squ.edu.om

How do people comprehend How do people comprehend the world around themthe world around them??

By three types of reasoning:By three types of reasoning: Deductive reasoning. Example:Deductive reasoning. Example: All planets orbit the sunAll planets orbit the sun The earth is a planet.The earth is a planet. Therefore the earth orbits the sun.Therefore the earth orbits the sun. Inductive reasoning: Inductive reasoning: A study of a number of individual cases would A study of a number of individual cases would

lead to a hypothesis and eventually to a lead to a hypothesis and eventually to a generalizationgeneralization..

A combined inductive-deductive approachA combined inductive-deductive approach

How do people comprehend the How do people comprehend the world around them (cont.)world around them (cont.)??

A combined inductive-deductive A combined inductive-deductive approach:approach:

This approach combines Aristotelian This approach combines Aristotelian deduction with Baconian induction. Here deduction with Baconian induction. Here the researcher is involved in a back-and-the researcher is involved in a back-and-forth process of induction (from forth process of induction (from observation to hypothesis) and deduction observation to hypothesis) and deduction (from hypothesis to implication).(from hypothesis to implication).

What is researchWhat is research??

A systematic attempt to provide answers to A systematic attempt to provide answers to questions. The answers may be either questions. The answers may be either abstract and general (e.g. abstract and general (e.g. basic researchbasic research), ), or highly concrete and specific (e.g., as in or highly concrete and specific (e.g., as in demonstration or demonstration or applied researchapplied research).).

What is research (cont.)What is research (cont.)??

Basic researchBasic research: is concerned with the : is concerned with the relationship between two or more variables. relationship between two or more variables. It is carried out by:It is carried out by:

Identifying a problem, Identifying a problem, Examining selected relevant variables,Examining selected relevant variables, Constructing hypotheses, Constructing hypotheses, Creating a relevant research design, Creating a relevant research design, Collecting and analyzing data, andCollecting and analyzing data, and Drawing conclusionsDrawing conclusions

What is research (cont.)What is research (cont.)??

Applied ResearchApplied Research: Using the findings of : Using the findings of basic research to develop a product (e.g., a basic research to develop a product (e.g., a curriculum, a training program, a multimedia curriculum, a training program, a multimedia instruction, etc.), and then systematically instruction, etc.), and then systematically test or tryout that product. test or tryout that product.

What is research (cont.)What is research (cont.)??

A systematic, controlled empirical and A systematic, controlled empirical and critical investigation of hypothetical critical investigation of hypothetical proposition about the presumed relations proposition about the presumed relations among natural phenomena.among natural phenomena.

Systematic and controlledSystematic and controlled: basing its : basing its operations on the inductive-deductive operations on the inductive-deductive model. model.

Empirical:Empirical: subjective and personal belief has subjective and personal belief has to have a reality check against objective, to have a reality check against objective, empirical facts and tests. empirical facts and tests.

What is research (cont.)What is research (cont.)??

Research is a combination of both Research is a combination of both experience and reasoning and must be experience and reasoning and must be regarded as the most successful approach regarded as the most successful approach to the discovery of truth.to the discovery of truth.

The role of researchThe role of research

Basic research doesn’t often provide Basic research doesn’t often provide immediately usable information for altering the immediately usable information for altering the environment.environment.

Its purpose is to develop a model or theory that Its purpose is to develop a model or theory that identifies all the relevant variables in a particular identifies all the relevant variables in a particular environment and hypothesizes about their environment and hypothesizes about their relationship.relationship.

To uncover facts and then formulate a To uncover facts and then formulate a generalization based on the interpretation of generalization based on the interpretation of those facts.those facts.

Validity in researchValidity in research

Internal validityInternal validity: The outcomes of the study is : The outcomes of the study is function of the program or approach being function of the program or approach being tested rather than the result of other causes not tested rather than the result of other causes not systematically dealt with in the study.systematically dealt with in the study.

Internal validity affects our Internal validity affects our certainty.certainty. External validityExternal validity affects our ability to credit the affects our ability to credit the

research results with research results with generalitygenerality based on the based on the procedures used. In other, the results would procedures used. In other, the results would apply in the real world to other similar programs apply in the real world to other similar programs and approaches.and approaches.

Validity in research (cont.)Validity in research (cont.)

Validity is related to Validity is related to ManipulationManipulation and and ControlControl of the experiment of the experiment::

Internal validity:Internal validity: Selection Selection biasbias History History biasbias External validity:External validity: Selection Selection limitationlimitation History History limitationlimitation

Internal and external validityInternal and external validity

Which of the following statements represents Which of the following statements represents threats to threats to internalinternal validity or threats to validity or threats to external external validity:validity:

1.1. No attempt to establish group equivalence.No attempt to establish group equivalence.

2.2. Reading gains due to maturation.Reading gains due to maturation.

3.3. Groups not representative of al first graders.Groups not representative of al first graders.

4.4. Teachers of the two classes have different Teachers of the two classes have different styles.styles.

Internal and external validity (cont.)Internal and external validity (cont.)

Which of the following statements best Which of the following statements best describes describes internalinternal validity or validity or external external validity:validity:

1.1. Ensuring that an experiment is reasonably Ensuring that an experiment is reasonably representative of reality.representative of reality.

2.2. Ensuring that the results really occurred.Ensuring that the results really occurred.

3.3. Ensuring that the experimenter followed the Ensuring that the experimenter followed the rules.rules.

4.4. Ensuring that the results were really a function Ensuring that the results were really a function of the experimental treatment.of the experimental treatment.

Internal and external validity (cont.)Internal and external validity (cont.)

Which of the following statements best Which of the following statements best describes the relationship between internal describes the relationship between internal and external validity :and external validity :

1.1. If there is no internal validity to an experiment If there is no internal validity to an experiment there can be no external validity.there can be no external validity.

2.2. If there is no external validity to an experiment If there is no external validity to an experiment there can be no internal validity.there can be no internal validity.

3.3. Internal validity and external validity are Internal validity and external validity are essentially unrelated.essentially unrelated.

Internal and external validity (cont.)Internal and external validity (cont.)

You have just read about an experiment. You have just read about an experiment. You can not apply the results because You can not apply the results because conclusions do not seem warranted conclusions do not seem warranted based on the design of the experiment. based on the design of the experiment. That means: the experiment lacks That means: the experiment lacks ………. ………. (internal/ external)(internal/ external) validity. validity.

Characteristics of the research processCharacteristics of the research process

Research is systematicResearch is systematic Research is logicalResearch is logical Research is EmpiricalResearch is Empirical Research is reductiveResearch is reductive Research is replicable and transmittableResearch is replicable and transmittable

Research is systematicResearch is systematic

It follows a structured process and rules It follows a structured process and rules that include: procedural specifications for that include: procedural specifications for identifying and defining variables, identifying and defining variables, selecting a relevant design to determine selecting a relevant design to determine their effect on other variables, relating the their effect on other variables, relating the collected data to the originally stated collected data to the originally stated problem and hypotheses.problem and hypotheses.

Research is logicalResearch is logical

It follows a system that employs logical It follows a system that employs logical examination of the specified procedures. examination of the specified procedures. The logic of valid research makes it a The logic of valid research makes it a valuable tool for drawing conclusions, or valuable tool for drawing conclusions, or decision making that is certainly far decision making that is certainly far superior than intuition, or using (“of –the-superior than intuition, or using (“of –the-top-of the-head”) observations for data.top-of the-head”) observations for data.

Research is empiricalResearch is empirical

The empirical quality of research depends The empirical quality of research depends on the quality of data collection and on the quality of data collection and treatment.treatment.

Research is reductiveResearch is reductive

It employs analytic procedures to the data It employs analytic procedures to the data collected to reduce confusion of particular collected to reduce confusion of particular events to a more general and events to a more general and understandable conceptual categories. In understandable conceptual categories. In other words, other words, reductionismreductionism enables enables research to research to explainexplain rather than simply rather than simply describedescribe..

Research is replicable and transmittableResearch is replicable and transmittable

Explain the following statement:Explain the following statement:

“ “TheThe transmittable property of research is transmittable property of research is critical to its role both in extending critical to its role both in extending knowledge and in decision-making”knowledge and in decision-making”..

Describe in one sentence each of the Describe in one sentence each of the following characteristics of the research following characteristics of the research processprocess::

SystematicSystematic LogicalLogical EmpiricalEmpirical ReductiveReductive TransmittableTransmittable

Some ethical considerationsSome ethical considerations

The right to privacy or nonparticipation.The right to privacy or nonparticipation. The right to remain anonymous.The right to remain anonymous. The right to confidentiality.The right to confidentiality. The right to expect experimenter The right to expect experimenter

responsibility.responsibility.

Steps in the research processSteps in the research process Identifying a problemIdentifying a problem Reviewing the literatureReviewing the literature Constructing a hypothesisConstructing a hypothesis Identifying and labeling variablesIdentifying and labeling variables Constructing operational definitionsConstructing operational definitions Manipulating and controlling variablesManipulating and controlling variables Constructing a research designConstructing a research design Identifying and constructing devices for observation and Identifying and constructing devices for observation and

measurementmeasurement Identifying and constructing devices for observation and Identifying and constructing devices for observation and

measurementmeasurement Constructing questionnaires and interview schedulesConstructing questionnaires and interview schedules Carrying out statistical analysesCarrying out statistical analyses Using the computer for data analysisUsing the computer for data analysis Writing a research reportWriting a research report

Identifying a problemIdentifying a problem A research problem is a practical statement that A research problem is a practical statement that

represents a relationship between two or more variables, represents a relationship between two or more variables, and is characterized by the following:and is characterized by the following:

It should ask about a relationship between two or more It should ask about a relationship between two or more variables.variables.

It should be clearly and unambiguously stated.It should be clearly and unambiguously stated. It should be stated in a question form (or, in a form of an It should be stated in a question form (or, in a form of an

implicit question such as, “the purpose of this study was implicit question such as, “the purpose of this study was to determine whether …).to determine whether …).

It should be testable by empirical methods (i.e., possible It should be testable by empirical methods (i.e., possible to collect data to answer the questions asked).to collect data to answer the questions asked).

It should not represent a moral or ethical position.It should not represent a moral or ethical position.

Types of Research problemTypes of Research problem:: Descriptive:Descriptive: Example: How many students in school X have an I.Q.’s Example: How many students in school X have an I.Q.’s

in excess of 120?in excess of 120? This type of problem requires simple recording of This type of problem requires simple recording of

observed frequencies.observed frequencies. Manipulative:Manipulative: This type requires the inclusion of at least two variables This type requires the inclusion of at least two variables

and their relations.and their relations. Examples: Are boys more likely than girls to have I.Q.’s Examples: Are boys more likely than girls to have I.Q.’s

in excess of 120? This would involve the relationship in excess of 120? This would involve the relationship between the variables of gender and I.Q. between the variables of gender and I.Q.

Examples of problems stated in question formExamples of problems stated in question form::

What is the relation between IQ and What is the relation between IQ and achievement?achievement?

Do student learn more from a directive Do student learn more from a directive teacher or a nondirective teacher?teacher or a nondirective teacher?

Is there a relation between parents’ Is there a relation between parents’ educational level and high school dropout educational level and high school dropout rates? rates?

Examples of problems stated in an implicit Examples of problems stated in an implicit question formquestion form::

The purpose is to discover the relationship The purpose is to discover the relationship between information search ability and IT between information search ability and IT skills.skills.

The study investigated whether the ability The study investigated whether the ability to discriminate among parts of speech to discriminate among parts of speech increased with chronological age and increased with chronological age and educational level.educational level.

How to narrow the range of problemsHow to narrow the range of problems??

Using a three-dimensional model for Using a three-dimensional model for problem consideration:problem consideration:

Available inputs – Available inputs – instructional activitiesinstructional activities and and organization-organization- anticipated outcomesanticipated outcomes

Using an inquiry model that identifies different sets or Using an inquiry model that identifies different sets or classes of variables that may be linked in their effect on classes of variables that may be linked in their effect on outcomes inoutcomes in a school setting. a school setting.

Using a conceptual model or a proposed Using a conceptual model or a proposed set of linkages set of linkages between specific variablesbetween specific variables, often along a path from input , often along a path from input to process to outcome, with the expressed purpose of to process to outcome, with the expressed purpose of predicting specific outcomes.predicting specific outcomes.

Specific considerations in choosing a Specific considerations in choosing a problemproblem

Workability.Workability. Is the contemplated study Is the contemplated study within the limits of your resource and time within the limits of your resource and time constraints? Will you have access to the constraints? Will you have access to the necessary sample in the numbers necessary sample in the numbers required?required?

Critical mass.Critical mass. Is the problem of sufficient Is the problem of sufficient magnitude and scope to fulfill the magnitude and scope to fulfill the requirement that motivated the study in the requirement that motivated the study in the first place? Are there enough variables?first place? Are there enough variables?

Specific considerations in choosing a Specific considerations in choosing a problem (cont)problem (cont)

Interest.Interest. Are you interested in the problem Are you interested in the problem area, specific problem, and potential area, specific problem, and potential solution? Does it relate to your solution? Does it relate to your background?background?

Theoretical value.Theoretical value. Does the problem fill a Does the problem fill a gap in the literature? Will others recognize gap in the literature? Will others recognize its importance?its importance?

Specific considerations in choosing a Specific considerations in choosing a problem (cont)problem (cont)

Practical valuePractical value.. Will the solution to the Will the solution to the problem improve educational practice? problem improve educational practice? Are practitioners likely to be interested in Are practitioners likely to be interested in the results?the results?

Reviewing the literatureReviewing the literature Literature is a collective body of prior work that helps to Literature is a collective body of prior work that helps to

uncover and provide the following:uncover and provide the following: Uncover ideas about variables that have proven Uncover ideas about variables that have proven

important or unimportant in a given field of study.important or unimportant in a given field of study. Discovering important variables.Discovering important variables. Distinguishing what has been done from what needs to Distinguishing what has been done from what needs to

be done.be done. Synthesizing and gaining perspectives.Synthesizing and gaining perspectives. Determining and supporting meanings and relationships.Determining and supporting meanings and relationships. Establishing the context of a problem.Establishing the context of a problem. Establishing the significance of a problem.Establishing the significance of a problem.

Literature review sourcesLiterature review sources

ERIC ERIC (Educational Resources Information (Educational Resources Information Center)Center)

ReviewsReviews Journal and booksJournal and books WebsitesWebsites

Evaluating Literature ReviewEvaluating Literature Review

AdequacyAdequacy Clarity: can you understand the points being Clarity: can you understand the points being

made?made? Imperialness: actual findings vs. opinions.Imperialness: actual findings vs. opinions. Recency: up-to-date.Recency: up-to-date. Relevance: related to the variables and Relevance: related to the variables and

hypotheses.hypotheses. OrganizationOrganization Convincingness: helps the authors make their Convincingness: helps the authors make their

case.case.

Constructing a hypothesisConstructing a hypothesis A hypothesis is an intelligent guess, or an expectation A hypothesis is an intelligent guess, or an expectation

about an event, based on generalizations of the about an event, based on generalizations of the assumed relationship between variables. assumed relationship between variables.

It is a suggested answer to the research problem It is a suggested answer to the research problem characterized by:characterized by:

It should conjecture upon a relationship between two or It should conjecture upon a relationship between two or more variables.more variables.

It should be stated in a clear declarative sentence.It should be stated in a clear declarative sentence. It should be possible to restate it in an operational form It should be possible to restate it in an operational form

that can then be evaluated based on data.that can then be evaluated based on data.

Examples of hypothesesExamples of hypotheses::

I.Q. and achievement are positively I.Q. and achievement are positively correlated.correlated.

Directive teachers are more effective than Directive teachers are more effective than nondirective teachers.nondirective teachers.

On line learning is more effective for high On line learning is more effective for high achievers than for low achievers. achievers than for low achievers.

What is the difference between observations, What is the difference between observations, specific hypothesis, and general hypothesisspecific hypothesis, and general hypothesis??

ObservationObservation refers to what is (i.e., what refers to what is (i.e., what can be seen).can be seen).

specific hypothesisspecific hypothesis: : setting forth the anticipated setting forth the anticipated relationship between two variablesrelationship between two variables..

general hypothesis: general hypothesis: A generalization from the A generalization from the first hypothesis and must be tested by making first hypothesis and must be tested by making observation. observation. Specific hypothesis requires fewer Specific hypothesis requires fewer observation for testing than do general observation for testing than do general hypotheseshypotheses. .

Constructing alternative Constructing alternative hypotheseshypotheses

Generate three hypotheses from the Generate three hypotheses from the following problem:following problem:

What is the combined effect of student What is the combined effect of student personality and instructional procedure on personality and instructional procedure on the amount of learning achieved?the amount of learning achieved?

Constructing alternative Constructing alternative hypotheseshypotheses

1.1. More-structured More-structured instructional procedures will provoke instructional procedures will provoke greater achievement among greater achievement among concrete studentsconcrete students, while , while less-structuredless-structured approaches will provoke greater approaches will provoke greater achievement among achievement among abstract studentsabstract students..

2.2. Less-structuredLess-structured instructional procedures will provoke instructional procedures will provoke greater achievement among greater achievement among concrete studentsconcrete students, while , while more-structured more-structured approaches will provoke greater approaches will provoke greater achievement among achievement among abstract studentsabstract students..

3.3. More-structured and less-structuredMore-structured and less-structured instructional instructional procedures will provoke equal achievement among procedures will provoke equal achievement among both both abstract students and concrete students.abstract students and concrete students.

Classroom Research hypothesesClassroom Research hypotheses

Treatment Treatment AA will increase learning outcome will increase learning outcome XX more than will more than will treatment treatment B.B.

1.1. StudentsStudents who receive who receive individually guided educationindividually guided education will will demonstrate greater gains in reading achievementdemonstrate greater gains in reading achievement than than studentsstudents who receive who receive conventional instruction. (two variables)conventional instruction. (two variables)

2.2. Poor readersPoor readers who receivewho receive individually guided education individually guided education will will demonstrate greater gains in reading achievement thandemonstrate greater gains in reading achievement than poor poor readersreaders who receivewho receive conventional instruction, conventional instruction, while no such while no such differences in reading achievement will occur betweendifferences in reading achievement will occur between good good readersreaders who receivewho receive one or the other instructional treatment. one or the other instructional treatment. (three variables).(three variables).

3.3. Or stated differently: Among Or stated differently: Among Poor readersPoor readers those receivingthose receiving individually guided education individually guided education will outgain those receiving will outgain those receiving conventional instructionconventional instruction on reading achievement, while among on reading achievement, while among good readersgood readers no differences will occurno differences will occur. (three variables).. (three variables).

Identifying and labeling variablesIdentifying and labeling variables

Among students of the same Among students of the same age and age and intelligenceintelligence, , skill performanceskill performance is directly is directly related to the related to the number of practice trials,number of practice trials, particularly among particularly among boysboys, but also, though , but also, though less directly among less directly among girlsgirls..

Identifying and labeling variablesIdentifying and labeling variables

Types of variables:Types of variables: Independent: Independent: number of practice trialsnumber of practice trials Dependent: Dependent: skill performanceskill performance Moderator: Moderator: gendergender Control: Control: age, intelligenceage, intelligence Intervening: Intervening: learninglearning

Definition of variablesDefinition of variables

Independent variableIndependent variable: is that factor that is : is that factor that is measured, manipulated, or selected by the measured, manipulated, or selected by the researcher to determine its relationship to researcher to determine its relationship to an observed phenomenon. (i.e., the an observed phenomenon. (i.e., the stimulus, or input)stimulus, or input)

Dependent variableDependent variable: is that factor that is : is that factor that is observed and measured to determine the observed and measured to determine the effect of the independent variable. (i.e., the effect of the independent variable. (i.e., the response, or output). response, or output).

Definition of variables (cont.)Definition of variables (cont.)

ModeratorModerator (e.g., (e.g., gendergender ): is that factor ): is that factor that is measured, manipulated, or selected that is measured, manipulated, or selected by the researcher to discover whether it by the researcher to discover whether it modifies the relationship of the primary modifies the relationship of the primary independent variable to an observed independent variable to an observed phenomenon. (i.e., secondary phenomenon. (i.e., secondary independent variable).independent variable).

Definition of variables (cont.)Definition of variables (cont.)

Control variablesControl variables (e.g., age, intelligence): (e.g., age, intelligence): are those factors that controlled by the are those factors that controlled by the experimenter to cancel out or neutralize experimenter to cancel out or neutralize any effect they might otherwise have on any effect they might otherwise have on the dependent variables. (i.e., all the the dependent variables. (i.e., all the variables in a situation (situational) or in a variables in a situation (situational) or in a person (dispositional variables) person (dispositional variables)

Definition of variables (cont.)Definition of variables (cont.)

Intervening variableIntervening variable (e.g., learning): is that (e.g., learning): is that factor that theoretically affect the observed factor that theoretically affect the observed phenomenon but phenomenon but can not be seen, can not be seen, measured, or manipulatedmeasured, or manipulated; its effect must ; its effect must be be inferredinferred from the effects of the from the effects of the independent and moderator variables on independent and moderator variables on observed phenomenon. observed phenomenon.

The combined variablesThe combined variables

Independent Variables

Moderator Variables

Control Variables

Intervening Variables

Dependent Variables

Causes Relationships Effects

Some considerations for variable choiceSome considerations for variable choice::

1.1. Theoretical considerationsTheoretical considerations: When : When choosing a moderator variable, you will choosing a moderator variable, you will need to ask:need to ask:

Is the variable related to the theory with Is the variable related to the theory with which I am working?which I am working?

How helpful would it be to know if an How helpful would it be to know if an interaction exists?interaction exists?

How likely is there to be an interaction?How likely is there to be an interaction?

Some considerations for variable choice (cont.)Some considerations for variable choice (cont.)::

2- Design considerations2- Design considerations: These are : These are questions related to the experimental questions related to the experimental design and its adequacy for controlling design and its adequacy for controlling for sources of bias. For example:for sources of bias. For example:

Have my decisions about moderator and Have my decisions about moderator and control variables met the requirements of control variables met the requirements of experimental design in terms of dealing experimental design in terms of dealing with sources of invalidity?with sources of invalidity?

Some considerations for variable choice (cont.)Some considerations for variable choice (cont.)::

3- Practical considerations3- Practical considerations: (limitations, : (limitations, time, resources, logistics, etc.)time, resources, logistics, etc.)

How difficult is it to make a variable a How difficult is it to make a variable a moderator as opposed to a control moderator as opposed to a control variable?variable?

What kinds of resources are available What kinds of resources are available and what kinds are required?and what kinds are required?

How much control do I have over the How much control do I have over the experimental situation?experimental situation?

Connect the terms in column Connect the terms in column AA with those in with those in BB??

AA

A.A. Independent Independent variablevariable

B.B. Dependent variableDependent variable

C.C. Moderator variableModerator variable

D.D. Control variableControl variable

E.E. Intervening variableIntervening variable

BB

1.1. AvoidedAvoided

2.2. InferredInferred

3.3. CauseCause

4.4. ModifierModifier

5.5. EffectEffect

Label the variables in the following hypothesisLabel the variables in the following hypothesis??

With age held constant, left handed With age held constant, left handed children with perceptual-motor training will children with perceptual-motor training will perform better on eye-hand coordination perform better on eye-hand coordination tasks than left-handed children without tasks than left-handed children without training, whereas such differences will not training, whereas such differences will not appear among right-handed children appear among right-handed children

Correct answerCorrect answer::

A.A. Independent variableIndependent variable

B.B. Moderator variableModerator variable

C.C. Control variableControl variable

D.D. Intervening variableIntervening variable

E.E. Dependent variableDependent variable

A.A. Perceptual-motor training Perceptual-motor training Vs, no trainingVs, no training

B.B. Handedness: left Vs. right.Handedness: left Vs. right.

C.C. AgeAge

D.D. Coordination (as a property Coordination (as a property of the nervous system)of the nervous system)

E.E. Performance on eye-hand Performance on eye-hand coordination tasks.coordination tasks.

Use ERIC to locate at least two Use ERIC to locate at least two relevantrelevant

studies to one of the followingstudies to one of the following : : Scaffolding students metacognitive Scaffolding students metacognitive

problem solving skillsproblem solving skills Study habits of undergraduate students.Study habits of undergraduate students.

Constructing variable definitionsConstructing variable definitions

Conceptual definitionConceptual definition is to identify is to identify something in terms of conceptual or something in terms of conceptual or hypothetical criteria rather than observable hypothetical criteria rather than observable ones. For example, ones. For example, defining effective defining effective teaching as teaching that promotes teaching as teaching that promotes learning.learning.

Constructing variable definitionsConstructing variable definitions (cont.)(cont.)

Operational definitionOperational definition is based on the is based on the observable characteristics of that which is being observable characteristics of that which is being defined. defined. What is important is the nature of the What is important is the nature of the observations upon which operational definitions observations upon which operational definitions are based, how they are made, and how they are based, how they are made, and how they are measured.are measured.

For example: For example: defining teaching as delivering a 45 defining teaching as delivering a 45 minute lesson following a nine events of minute lesson following a nine events of instructioninstruction

The Research SpectrumThe Research Spectrum

Problem

Relevant Theories

& Concepts

RelevantFindings

Hypotheses (with variables

Labeled)Predictions

ExperimentalDesign

MeasurementDevices

FindingsFinal

Report

Reading

& thinkin

g

Literaturesearch

deduction

induc

tion

operationallydefiningvariables

designing

oper

atio

nal z

ing

datacollection

&analysis

Manipulating and controlling Manipulating and controlling variablesvariables

What is a What is a controlcontrol group? group? Is a group of subjects whose selection and Is a group of subjects whose selection and

experiences are identical in every way experiences are identical in every way possible to the experimental group except possible to the experimental group except that they don’t receive the treatment (i.e., that they don’t receive the treatment (i.e., the independent variable is absent)the independent variable is absent)

Manipulating and controlling Manipulating and controlling variables (cont.)variables (cont.)

What are the reasons for using a control group?What are the reasons for using a control group? The essence of experimental research is The essence of experimental research is control.control. To eliminate other conditions and factors that To eliminate other conditions and factors that

also influence the obtained effect. That means also influence the obtained effect. That means all conditions other than the independent and all conditions other than the independent and moderator variables must be kept constant for moderator variables must be kept constant for both the experimental and control groups.both the experimental and control groups.

Experimental and control groupsExperimental and control groups

Experimental GroupExperimental Group

StimuliStimuli ResponseResponse

SSxx …………R …………Rxx

S1………….R1S1………….R1

S2………….R2S2………….R2

S3………….R3S3………….R3

S4………….R4S4………….R4

Control GroupControl Group

StimuliStimuli ResponseResponse

SS00 …………R …………R00

S1………….R1S1………….R1

S2………….R2S2………….R2

S3………….R3S3………….R3

S4………….R4S4………….R4

Methodological stages: Methodological stages: Designing and measuringDesigning and measuring

Identifying Techniques for the Identifying Techniques for the manipulation and control of variables.manipulation and control of variables.

Constructing research designConstructing research design Describing procedures for observation and Describing procedures for observation and

measuring.measuring. Constructing and using questionnaires and Constructing and using questionnaires and

interview schedulesinterview schedules

Factors or variables affecting internal validityFactors or variables affecting internal validity::

These are extraneous variables that can be a These are extraneous variables that can be a source of internal bias if not controlled:source of internal bias if not controlled:

HistoryHistory SelectionSelection MaturationMaturation TestingTesting InstrumentationInstrumentation Statistical regressionStatistical regression Experimental mortalityExperimental mortality

Constructing a research Constructing a research designdesign

Pre-experimental designsPre-experimental designs

One shot case studyOne shot case studyThe one-shot case study can be diagrammed as follows:The one-shot case study can be diagrammed as follows:

XX O O One-group pretest-posttest designOne-group pretest-posttest design

OO11 X X OO22

Intact-group comparison Intact-group comparison XX OO11

OO22

True experimental designsTrue experimental designs Posttest-only control group designPosttest-only control group design

R R XX O O11

R OR O22

R R XX11 O O1 1 (high enthusiasm)(high enthusiasm)

R R XX22 O O2 2 (medium enthusiasm)(medium enthusiasm)

R R XX33 O O3 3 (low enthusiasm)(low enthusiasm)

This is potentially the most useful true design. This is potentially the most useful true design. Why?Why?

True experimental designs (cont.)True experimental designs (cont.)

Pretest-posttest control group designPretest-posttest control group designR OR O11 X O X O22

R OR O33 O O44

This design controls for all the simple sources This design controls for all the simple sources

of invalidity of invalidity except testingexcept testing (e.g., history, (e.g., history, maturation, selection, and mortality)maturation, selection, and mortality)

Factorial designsFactorial designs

For pretest-posttest control group designFor pretest-posttest control group designR OR O11 XX Y Y11 O O22

R OR O33 Y Y11 O O44

R OR O55 XX Y Y22 O O66

R OR O77 Y Y22 O O88

Factorial designs (cont.)Factorial designs (cont.)

By modifying the posttest-only control By modifying the posttest-only control group design.group design.

R X YR X Y11 O O11

R YR Y11 O O22

R X YR X Y22 O O33

R YR Y22 O O44

Factorial designs (cont.)Factorial designs (cont.)

All that can be said of the true All that can be said of the true experimental design can be said of experimental design can be said of factorial design, with the addition that the factorial design, with the addition that the factorial design makes it possible to deal factorial design makes it possible to deal systematically with more than one systematically with more than one independent variable. independent variable. That is, within the That is, within the factorial designs, more than one variable factorial designs, more than one variable can be manipulated and thus studied.can be manipulated and thus studied.

Factorial designs (cont.)Factorial designs (cont.) Examples for the factorial version of the posttest-only Examples for the factorial version of the posttest-only

control group design. (4 by 2)control group design. (4 by 2)Treatments (X) Treatments (X)

R XR X11 Y Y11 O O11 X X11 computer-assisted instructioncomputer-assisted instruction

R XR X22 Y Y11 O O22 X X22 programmed textprogrammed text

R XR X33 Y Y11 O O33 X X33 televised lecturetelevised lecture

R XR X00 Y Y11 O O44 X X00 lecture-discussionlecture-discussion

R XR X11 Y Y22 O O55

R XR X22 Y Y22 O O66 Moderator (Y) Moderator (Y)R XR X33 Y Y22 O O7 7 YY11 high motivationhigh motivationRR XX00 Y Y22 O O8 8 YY22 low motivation low motivation

Identifying and constructing devices Identifying and constructing devices for observation and measurementfor observation and measurement

Test reliability:Test reliability: Means that the test is consistent. Means that the test is consistent. Test-retest reliability:Test-retest reliability: The resulting coefficient is an The resulting coefficient is an

indication of the extent to which the test is measuring indication of the extent to which the test is measuring stable and enduring characteristics of the test taker stable and enduring characteristics of the test taker rather than the variable or the temporary ones. rather than the variable or the temporary ones.

Alternate-form reliability:Alternate-form reliability: administering the alternate administering the alternate forms of a test to the same group and computing the forms of a test to the same group and computing the relation between each person’s score on the two forms. relation between each person’s score on the two forms. This can be used to assess the reliability of either of the This can be used to assess the reliability of either of the two forms by comparison with the other to determine the two forms by comparison with the other to determine the extent to which the two forms are parallel.extent to which the two forms are parallel.

Identifying and constructing devices for observation and measurement

Split-half reliability: To determine the internal consistency of a test relatively quickly: split a test into two halves, usually the odd-numbered items and the even-numbered items, and then correlate the scores obtained by each person on one half with those obtained by each person on the other.

Identifying and constructing devices for observation and measurement

Test validity: the validity of a test is the extent to which a test measures what it purports to measure.Predictive validityConcurrent validityConstruct validityContent validity

Constructing measures

A paper-and-pencil performance test. Constructing a scale. Constructing an observation recording

device.

Constructing questionnaires and Constructing questionnaires and interview schedulesinterview schedules

Choosing statistical analyses

Choosing a statistical test depends on: The purpose of the analysis (e.g., to describe

or explore data, to test a hypothesis, to seek correlations, to identify the effects of one or more independent variables on a dependent variable, etc,).

The kind of data with which one is working (parametric and non-parametric)

Choosing statistical analyses (cont.)

The scales of data being used (nominal, ordinal, interval, or ratio).

The number of groups in the sample. The assumption in the tests Whether the samples are independent of

each other or related to each other.

Assumptions of tests

For T-test and analysis of variance (ANOVA): Population is normally distributed. Sample is selected randomly from the population. Each case is independent from each other. The groups to be compared are nominal, and the

comparison is made using interval and ratio data. The sets of data to be compared are normally

distributed

COURSE GOALCOURSE GOAL The primary goal of this course is to The primary goal of this course is to

empower the student with skills necessary empower the student with skills necessary to carry out scientific research methods in to carry out scientific research methods in posing research problems, stating posing research problems, stating hypotheses in a testable form, and hypotheses in a testable form, and verifying these hypotheses by means of verifying these hypotheses by means of experimentation and observation.experimentation and observation.

COURSE OBJECTIVESCOURSE OBJECTIVESUpon completion of this course, you will be able toUpon completion of this course, you will be able to::

Describe the characteristics of the research process.Describe the characteristics of the research process. State a research problem clearly. State a research problem clearly. Describe the relation between internal and external validity.Describe the relation between internal and external validity. Describe purposes and strategies for reviewing the literature.Describe purposes and strategies for reviewing the literature. Construct testable hypotheses.Construct testable hypotheses. Distinguish between dependent and independent variables.Distinguish between dependent and independent variables. Choose a statistical test appropriate for different combinations Choose a statistical test appropriate for different combinations

of variables.of variables. Evaluate the scientific merit of research and studies in the Evaluate the scientific merit of research and studies in the

field. field. Write a research proposal.Write a research proposal. Conduct a theoretically- sound study .Conduct a theoretically- sound study .

Conducting evaluation studiesConducting evaluation studies

Conducting qualitative researchConducting qualitative research

Author, A. A., Author, B. B., & Author, C. C. Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article.(Year). Title of article.

Title of Journal, xx, xxx-xxx.Title of Journal, xx, xxx-xxx.

NOTE: The journal title and the volume number NOTE: The journal title and the volume number are in italics. Issue numbers are not required are in italics. Issue numbers are not required if the journal is continuously paged. If paged if the journal is continuously paged. If paged individually, the issue number is required individually, the issue number is required and is in regular type in parentheses and is in regular type in parentheses adjacent to the volume number.adjacent to the volume number.

One AuthorOne Author Williams, J. H. (2008). Employee engagement: Williams, J. H. (2008). Employee engagement:

Improving participation in safety. Improving participation in safety. Professional Professional Safety, 53(12), 40-45.Safety, 53(12), 40-45.

Two to Seven Authors [List all authors]Two to Seven Authors [List all authors] Keller, T. E., Cusick, G. R., & Courtney, M. E. Keller, T. E., Cusick, G. R., & Courtney, M. E.

(2007). Approaching the transition to adulthood: (2007). Approaching the transition to adulthood: Distinctive profiles of adolescents aging out of Distinctive profiles of adolescents aging out of the child welfare system. the child welfare system. Social Services Social Services Review, 81, 453-Review, 81, 453-484.484.