![Page 1: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/1.jpg)
Chapter 5.
Reliability of
Assessment Device
![Page 2: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/2.jpg)
Contents
Concept of reliability
Stability reliability
Alternate-form reliability
![Page 3: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/3.jpg)
Assessment Reliability
The consistency of results produced by measurement
devices.
Relationship between reliability and validity
Test reliability is a necessary, but insufficient. Ex) vocabulary test & Pole-Vaulting aptitude test
Concept Of Reliability
![Page 4: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/4.jpg)
Administer a test to students
Readministe-r the same test to the same students
Correlate students’ scores on the two tests
Stability Reliability
The consistency of assessment results over time.
Stability Reliability
Result reliability coefficient
Appropriate interval
![Page 5: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/5.jpg)
Stability Reliability
test retest
A 7 7
B 5 6
C 9 9
D 8 7
E 6 6
Test-retest
Coefficient Stability reliability
![Page 6: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/6.jpg)
Attention !
The length of the interval between two test
The selection of the between-testing interval’s length should be
made so as to reduce the influence of the first testing on the second, but also to reduce the likelihood that intervening events in the lives of the students will distort the second set of test results.
Stability Reliability
‘two or three weeks’ is reasonable *
![Page 7: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/7.jpg)
Attention !
Same motive, attitude, atmosphere
Care must be taken to create an atmosphere in which students will
devote comparable zeal to both test administrations.
The length of the test
the longer the test, the more reliable the test tends to be.
Stability Reliability
![Page 8: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/8.jpg)
Classification Consistency
If the test were administered at a different time, how many students would be classified in the same manner.
A representation of the proportion of students who are placed in the same category on two testing occasions or two test forms.
Stability Reliability
![Page 9: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/9.jpg)
Classification Consistency
Stability Reliability
0 x=25 50 0 x=25 50
Cutoff score = 27 cutoff score = 45
Situation X Situation Z
![Page 10: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/10.jpg)
Classification Consistency
False positive
Classifying a student as having mastered a given skill or knowledge domain, but the student actually had not mastered it.
False Negative
Classifying a student as not having mastered a given skill or knowledge domain, but the student actually had mastered it.
Stability Reliability
![Page 11: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/11.jpg)
Alternate-Form Reliability
The consistency of measured results yielded by
different forms of the same test.
Alternate-Form Reliability
Administer two forms of the same test
Correlate student’s scores on the two test forms
Result coefficient
![Page 12: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/12.jpg)
Reduce remembrance effect (+)
Difficulty of making different forms of the same test (-)
Alternate-Form Reliability
![Page 13: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/13.jpg)
Stability and Alternate-Form Reliability
The consistency of measured results over time using two different test forms.
has low reliability coefficient
☞ Because of two factors: time delay and
differences between the two forms
Alternate-Form Reliability
![Page 14: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/14.jpg)
Post Facto Form Equating two approaches to determining item difficulties
Field-testing
: create large numbers of items → administer those items in a formal field test → consequence of field tests → determine the difficulty level of each item
Embedding
: embedding of new test items in administered versions of an ongoing testing program
Alternate-Form Reliability
![Page 15: Chapter.5 Reliability of Assessment Devicecontents.kocw.net/document/Ch.5 Reliability of Assessment... · 2013-12-30 · Assessment Reliability The consistency of results produced](https://reader036.vdocument.in/reader036/viewer/2022062402/5e7b82721bb62f1d23688925/html5/thumbnails/15.jpg)
Thank you.