reliability iop 301-t mr. rajesh gunesh reliability reliability means repeatability or consistency ...
TRANSCRIPT
![Page 1: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/1.jpg)
Reliability
![Page 2: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/2.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability Reliability means repeatability
or consistencyA measure is considered
reliable if it would give us the same result over and over again (assuming that what we are measuring isn’t changing!)
![Page 3: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/3.jpg)
Mr. Rajesh GuneshIOP 301-T
Definition of ReliabilityReliability usually “refers to the
consistency of scores obtained by the same persons when they are reexamined with the same test on different occasions, or with different sets of equivalent items, or under other variable examining conditions (Anastasi & Urbina, 1997).
Dependable, consistent, stable, constant
Gives the same result over and over again
![Page 4: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/4.jpg)
Mr. Rajesh GuneshIOP 301-T
Validity vs Reliability
![Page 5: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/5.jpg)
Mr. Rajesh GuneshIOP 301-T
Variability and reliability
What is the acceptable range of error in measurement– Bathroom scale ±1 kg– Body thermometer ±0.2 C– Baby weight scale ±20 g – Clock with hands ±5 min– Outside thermometer ±1 C
![Page 6: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/6.jpg)
Mr. Rajesh GuneshIOP 301-T
Variability and reliability
We are completely comfortable with a bathroom scale accurate to ±1 kg, since we know that individual weights vary over far greater ranges than this, and typical changes from day to day are about the same order of magnitude.
![Page 7: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/7.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability
True Score TheoryMeasurement ErrorTheory of reliabilityTypes of reliabilityStandard error of
measurement
![Page 8: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/8.jpg)
Mr. Rajesh GuneshIOP 301-T
True Score Theory
![Page 9: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/9.jpg)
Mr. Rajesh GuneshIOP 301-T
True Score Theory
Every measurement is an additive composite of two components:
1. True ability (or the true level) of the respondent on that measure
2. Measurement error
![Page 10: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/10.jpg)
Mr. Rajesh GuneshIOP 301-T
True Score Theory
Individual differences in test scores
– “True” differences in characteristic being assessed
– “Chance” or random errors.
![Page 11: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/11.jpg)
Mr. Rajesh GuneshIOP 301-T
True Score Theory
What might be considered error variance in one situation may be true variance in another (e.g Anxiety)
![Page 12: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/12.jpg)
Mr. Rajesh GuneshIOP 301-T
Can we observe the true score?
X = T + ex
We only observe the measurement, we don’t observe what’s on the right side of equation (only God knows what those values are)
![Page 13: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/13.jpg)
Mr. Rajesh GuneshIOP 301-T
True Score Theory
var(X) = var(T) + var(ex)
The variability of the measure is the sum of the variability due to true score and the variability due to random error
![Page 14: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/14.jpg)
Mr. Rajesh GuneshIOP 301-T
What is error variance?
Conditions irrelevant to purpose of the test– Environment (e.g., quiet v. noisy)– Instructions (e.g., written v. verbal)– Time limits (e.g., limited v.
unlimited)– Rapport with test taker
All test scores have error variance.
![Page 15: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/15.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement Error
Measurement error:
–Random–Systematic
![Page 16: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/16.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement Error
![Page 17: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/17.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement Error
Random error: effects are NOT consistent across the whole sample, they elevate some scores and depress others– Only adds noise; does not
affect mean score
![Page 18: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/18.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement ErrorSystematic error: effects are
generally consistent across a whole sample– Example: environmental
conditions for group testing (e.g., temperature of the room)
– Generally either consistently positive (elevate scores) or negative (depress scores)
![Page 19: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/19.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement Error
![Page 20: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/20.jpg)
Mr. Rajesh GuneshIOP 301-T
Measurement Error
![Page 21: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/21.jpg)
Mr. Rajesh GuneshIOP 301-T
Theory of Reliability
![Page 22: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/22.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability
Reliability = The variance of the true scoreThe variance of the measure
Reliability = Var(T)
Var(X)
![Page 23: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/23.jpg)
Mr. Rajesh GuneshIOP 301-T
How big is an estimate of Reliability?
Var(T)Reliability =
Var(T)
Var(X) =
Var(T) + Var(e)
Reliability =Subject variability
Subject variability + measurement error
![Page 24: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/24.jpg)
Mr. Rajesh GuneshIOP 301-T
We can’t compute reliability because we can’t calculate the variance of the true score; but we can get an estimate of the variability.
![Page 25: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/25.jpg)
Mr. Rajesh GuneshIOP 301-T
Estimate of Reliability
Observations would be related to each other to the degree that they share true scores. For example consider the correlation between X1 and X2:
)(var)(var
),(covariance
21
21
XX
XX
![Page 26: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/26.jpg)
Mr. Rajesh GuneshIOP 301-T
Test-Retest
Stability
Alternate-form
Inter-scorer
Equivalence
Split-half
Kuder-R ichardson
Cronbach Alpha
Internal consistency
RELIABILITY
![Page 27: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/27.jpg)
Mr. Rajesh GuneshIOP 301-T
Types of Reliability1. Test-Retest Reliability
Used to assess the consistency of a measure from one time to another
2. Alternate-form ReliabilityUsed to assess the consistency of the results of two tests constructed the same way from the same content domain
![Page 28: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/28.jpg)
Mr. Rajesh GuneshIOP 301-T
Types of Reliability3. Split-half Reliability
Used to assess the consistency of results across items within a test by splitting them into two equivalent halves
Kuder-Richardson Reliability Used to assess the extent to which items are homogenous when items have a dichotomous response, e.g. “yes/no” items.
![Page 29: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/29.jpg)
Mr. Rajesh GuneshIOP 301-T
Types of Reliability Cronbach’s alpha (α) Reliability
Compares the consistency of response of all items on the scale (Likert scale or linear graphic response format)
4. Inter-Rater or Inter-Scorer ReliabilityUsed to assess the concordance between two or more observers scores of the same event or phenomenon for observational data
![Page 30: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/30.jpg)
Mr. Rajesh GuneshIOP 301-T
Test-Retest Reliability
Definition: When the same test is administered to the same individual (or sample) on two different occasions
![Page 31: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/31.jpg)
Mr. Rajesh GuneshIOP 301-T
Test-Retest Reliability:Used to assess the consistency of a measure from one time to another
![Page 32: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/32.jpg)
Mr. Rajesh GuneshIOP 301-T
Test-Retest ReliabilityStatistics used
– Pearson r or Spearman rhoWarning
– Correlation decreases over time because error variance INCREASES (and may change in nature)
– Closer in time the two scores were obtained, the more the factors which contribute to error variance are the same
![Page 33: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/33.jpg)
Mr. Rajesh GuneshIOP 301-T
Test-Retest Reliability
Warning– Circumstances may be
different for both test-taker and physical environment.
– Transfer effects like practice and memory might play a role on the second testing occasion
![Page 34: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/34.jpg)
Mr. Rajesh GuneshIOP 301-T
Alternate-form Reliability
Definition: Two equivalent forms of the same measure are administered to the same group on two different occasions
![Page 35: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/35.jpg)
Mr. Rajesh GuneshIOP 301-T
Alternate-form Reliability:Used to assess the consistency of the results of two tests constructed same way from the same content domain
![Page 36: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/36.jpg)
Mr. Rajesh GuneshIOP 301-T
Alternate-form Reliability
Statistic used– Pearson r or Spearman rho
Warning– Even when randomly
chosen, the two forms may not be truly parallel
– It is difficult to construct equivalent tests
![Page 37: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/37.jpg)
Mr. Rajesh GuneshIOP 301-T
Alternate-form Reliability
Warning– Even when randomly chosen,
the two forms may not be truly parallel
– It is difficult to construct equivalent tests
– The tests should have the same number of items, same scoring procedure, uniform content and item difficulty level
![Page 38: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/38.jpg)
Mr. Rajesh GuneshIOP 301-T
Split-half ReliabilityDefinition: Randomly divide the test into two forms; calculate scores for Form A, B; calculate Pearson r as index of reliability
![Page 39: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/39.jpg)
Mr. Rajesh GuneshIOP 301-T
Split-half Reliability
![Page 40: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/40.jpg)
Mr. Rajesh GuneshIOP 301-T
Split-half Reliability
hh
hhtt r
rr
1
2
(Spearman-Brown formula)
![Page 41: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/41.jpg)
Mr. Rajesh GuneshIOP 301-T
Split-half ReliabilityWarning The correlation between the odd
and even scores are generally an underestimation of the reliability coefficient because it is based only on half the test.
![Page 42: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/42.jpg)
Mr. Rajesh GuneshIOP 301-T
Cronbach’s alpha & Kuder-Richardson-20
Measures the extent to which items on a test are homogeneous; mean of all possible split-half combinations– Kuder-Richardson-20 (KR-
20): for dichotomous data– Cronbach’s alpha: for non-
dichotomous data
![Page 43: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/43.jpg)
Mr. Rajesh GuneshIOP 301-T
Cronbach’s alpha (α)
![Page 44: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/44.jpg)
Mr. Rajesh GuneshIOP 301-T
Cronbach’s alpha (α)
2
22
1 t
it
s
ss
n
n
(Coefficient
alpha)
![Page 45: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/45.jpg)
Mr. Rajesh GuneshIOP 301-T
Kuder-Richardson (KR-20)
2
2
1 t
ttt
s
pqs
n
nr
![Page 46: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/46.jpg)
Mr. Rajesh GuneshIOP 301-T
Inter-Rater or Inter-Observer Reliability:
Used to assess the degree to which different raters or observers give consistent estimates of the same phenomenon
![Page 47: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/47.jpg)
Mr. Rajesh GuneshIOP 301-T
Inter-rater Reliability
Definition Measures the extent to
which multiple raters or judges agree when providing a rating of behavior
![Page 48: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/48.jpg)
Mr. Rajesh GuneshIOP 301-T
Inter-rater ReliabilityStatistics used
– Nominal/categorical data• Kappa statistic
– Ordinal data• Kendall’s tau to see if pairs of
ranks for each of several individuals are related– Two judges rate 20
elementary school children on an index of hyperactivity and rank order them
![Page 49: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/49.jpg)
Mr. Rajesh GuneshIOP 301-T
Inter-rater ReliabilityStatistics used
– Interval or ratio data•Pearson r using data obtained from the hyperactivity index
![Page 50: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/50.jpg)
Mr. Rajesh GuneshIOP 301-T
Factors affecting Reliability
Whether a measure is speeded
Variability in individual scores
Ability level
![Page 51: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/51.jpg)
Mr. Rajesh GuneshIOP 301-T
Whether a measure is speeded
For speeded measures, test-retest and equivalent-form reliability are more appropriate. Split-half techniques may be considered if the split occurs according to time rather than number of items.
![Page 52: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/52.jpg)
Mr. Rajesh GuneshIOP 301-T
Variability in individual scores
Correlation is normally affected by the range of individual differences in a group. Sometimes, smaller subgroups display correlation coefficients which are completely different from that of the whole group. This phenomenon is known as range restriction.
![Page 53: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/53.jpg)
Mr. Rajesh GuneshIOP 301-T
Ability level
One must also consider the variability and ability levels of samples. It is advisable to compute separate reliability coefficients for homogeneous and heterogeneous subgroups.
![Page 54: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/54.jpg)
Mr. Rajesh GuneshIOP 301-T
Interpretation of Reliability
One must ask oneself the following questions:How high must the coefficient of reliability be?
How is it interpreted?What is the standard error of measurement?
![Page 55: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/55.jpg)
Mr. Rajesh GuneshIOP 301-T
Magnitude of reliability coefficient
Anastasi & Urbina (1997) 0.8 – 0.9 Huysamen (1996)
at least 0.85 for individualsat least 0.65 for groups
Smit (1996)0.8 – 0.85 for personality &
interest at least 0.9 for aptitude
![Page 56: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/56.jpg)
Mr. Rajesh GuneshIOP 301-T
Standard Error of the Measurement
Definition: Estimate of the amount of error usually attached to an individual’s obtained test score– As SEM ↑, test reliability ↓– As SEM ↓, test reliability ↑
![Page 57: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/57.jpg)
Mr. Rajesh GuneshIOP 301-T
Standard Error of the Measurement
ttt rsSEM 1
![Page 58: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/58.jpg)
Mr. Rajesh GuneshIOP 301-T
Standard Error of the Measurement
Confidence Interval: Uses SEM to calculate a band or range of scores that has a high probability of including the person’s true score.
Example: 95% confidence interval means only 5 times in 100 will the person’s TRUE score lie outside this range of scores.
![Page 59: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/59.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability
Formula: CI = Obtained score + z(SEM)z = 1.0 for 68% levelz = 1.44 for 85% levelz = 1.65 for 90% levelz = 1.96 for 95% levelz = 2.58 for 99% level
![Page 60: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/60.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability of standardized tests
An acceptable standardized test should have reliability coefficients of at least:
0.95 for internal consistency0.90 for test-retest (stability)0.85 for alternate-forms
(equivalency)
![Page 61: Reliability IOP 301-T Mr. Rajesh Gunesh Reliability Reliability means repeatability or consistency A measure is considered reliable if it would give](https://reader036.vdocument.in/reader036/viewer/2022062407/56649c715503460f94922749/html5/thumbnails/61.jpg)
Mr. Rajesh GuneshIOP 301-T
Reliability: Implications
Evaluating a test– What types of reliability have
been calculated and with what samples?
– What are the strengths of the reliability coefficients?
– What is the SEM for a test score– How does this information
influence decision to use and interpret test scores?