ValidityValidityDegree to which evidence Degree to which evidence
supports inferences madesupports inferences madeAppropriateAppropriateMeaningfulMeaningfulUsefulUseful
Meaningful Meaningful InferenceInferenceSays something about Says something about the meaning of the the meaning of the information obtained information obtained through an instrumentthrough an instrument
Useful InferenceUseful InferenceHelps researchers Helps researchers make decisions related make decisions related to what they studiedto what they studied
Do we get useful Do we get useful information about information about the topic/variables?the topic/variables?
Validity depends on:Validity depends on:The amount and type The amount and type of evidence supporting of evidence supporting the interpretationsthe interpretations
Types of EvidenceTypes of EvidenceContent-related Content-related Criterion-relatedCriterion-relatedConstruct-relatedConstruct-related
Content-relatedContent-relatedAppropriate?Appropriate?Comprehensive?Comprehensive?Logical?Logical?Adequate?Adequate?
Adequacy of the samplingAdequacy of the sampling• representative of content?representative of content?
Format of the instrumentFormat of the instrument• language, directions, etc.language, directions, etc.
Who judges the validity of Who judges the validity of the content?the content?• InterpretationInterpretation
• judgementjudgement
Criterion-relatedCriterion-relatedRelationship between scores Relationship between scores
from different instrumentsfrom different instrumentsStrong relationship?Strong relationship?Accurate?Accurate?
Instrument results compared Instrument results compared with independent criterionwith independent criterion• a second test or measurementa second test or measurement
Predictive validityPredictive validity• time between measurementstime between measurements
Concurrent validityConcurrent validity• instrument and criterion instrument and criterion
measurements at same timemeasurements at same time
Validity CoefficientValidity Coefficient• r r = correlation coefficient= correlation coefficient
• degree of relationship degree of relationship between scores from between scores from instrument and criterioninstrument and criterion
• Between -1.00 and +1.00Between -1.00 and +1.00
• .00 = no relationship.00 = no relationship
Construct-relatedConstruct-relatedHow well does the How well does the
theoretical construct account theoretical construct account for or explain differences?for or explain differences?
Is the construct sufficient?Is the construct sufficient?
Three steps:Three steps:• variable measured clearly variable measured clearly
defineddefined
• hypotheses (based on theory)hypotheses (based on theory)
• hypotheses tested logically hypotheses tested logically and empiricallyand empirically
Collect variety of different Collect variety of different types of evidencetypes of evidence
ReliabilityReliabilityConsistency of the scoresConsistency of the scores
• among individualsamong individuals
• among itemsamong items
Reliable -- valid or not Reliable -- valid or not validvalid
Not reliable -- not validNot reliable -- not validValidity and reliability Validity and reliability
dependent on the context dependent on the context in which the instrument is in which the instrument is used.used.
Measurement ErrorMeasurement ErrorSame test, different scoresSame test, different scoresCaused by variety of factors:Caused by variety of factors:
• motivationmotivation
• anxietyanxiety• environmentenvironment
• etc.etc.
Reliability CoefficientReliability CoefficientReliability estimatesReliability estimatesRelationship between scores Relationship between scores
at two different times, orat two different times, orRelationship between two Relationship between two
parts of the same instrumentparts of the same instrument
Test-RetestTest-RetestSame test, same group, two Same test, same group, two
timestimesReliability coefficient Reliability coefficient
indicates relationship indicates relationship between the two scoresbetween the two scores
Equivalent-FormsEquivalent-FormsTwo different, equivalent Two different, equivalent
tests, same grouptests, same groupReliability coefficient Reliability coefficient
calculatedcalculated
Internal-ConsistencyInternal-ConsistencySingle administration of the Single administration of the
instrumentinstrumentSplit-half procedureSplit-half procedureKuder-Richardson approachKuder-Richardson approachAlpha coefficientAlpha coefficient
Split-Half ProcedureSplit-Half ProcedureScore two halves of test Score two halves of test
separatelyseparatelyDegree to which the two halves Degree to which the two halves
provide same resultsprovide same resultsReliability increases with length Reliability increases with length
of the test/instrumentof the test/instrument