aei summer institute - educational assessment
TRANSCRIPT
Educational Assessment
Stephen Parks, Learning Specialist - Mathematics
Assessment and Evaluation Branch (Anglophone Division)
Department of Education and Early Childhood Development
Atlantic Education International
July 21, 2015
0
0 .2
0 .4
0 .6
0 .8
1 .0
-3 -2 -1 0 1 2 3
b
Ab i l i ty
Pro
bab
ilit
y
Item Characteristic Curve: ITEM 0001
a = 0 .5 3 9 b = 0 .2 0 2
1. Educational Assessment
2. New Brunswick Assessment Program - NBAP (Current)
3. Realignment of the NBAP
4. Grade 9 ELPA – Grade 11/12 ELPA/R
5. Protocols for Accommodations and Exemptions
6. Assessment and Item Development Process (Grade 6)
7. Provincial Mathematics Assessment at Grade 8
TOPICS
© 2005, Dynamic Measurement Group 3
GPS On Where are we?
You are Here
Port
Where do we want to be ?
How do we get there?
How are we doing?
ActualCourse
DesiredCourse
Assessment is the GPS for Educators
10
20
30
40
Dec.Scores
Feb.Scores
Jan.Scores
M archScores
AprilScores
MayScores
JuneScores
60
50
Rea
ding
Aimline
New Brunswick Assessment Program (NBAP)
Mission StatementThe mission for the Provincial Assessment and Evaluation Branch is to inform all stakeholders of student achievement by constructing valid, educationally beneficial, evaluations at critical points in the system. The purpose of the New Brunswick Assessment Program (NBAP) is to: determine if students demonstrate the skills and concepts outlined in New Brunswick curriculum documents; report provincially on how well students have achieved provincial standards at given points in their schooling; assist schools, districts, and the province in monitoring student learning and; coordinate the administration of national and international assessments.
Current Assessment Program
•Pan-Canadian Assessment Program (PCAP) as well as the Programme for International Student Assessment (PISA).
•Scientific literacy
•Administering assessments as single units.
•Feedback for high schools.
•The current regime of provincial assessments
•French Language Programs
Realignment of the Provincial Assessment Program
Recommendation # 1
• To create a New Brunswick Balanced Assessment Framework document incorporating assessment as a holistic concept from formative to benchmarking to large scale and to develop a systemic implementation plan that includes accountability measures.
Balanced Assessment
• No single assessment approach is 'better' than another as each serves a slightly different role.
• All types of assessment must be planned with its purpose in mind.
Standardized testing, particularly, is justified as one resource to aid learning and should not be mistaken for a comprehensive verdict on learning or school quality. ~ Peter Grimmett & Paul Shaker
Recommendation #2To realign the Provincial Assessment Program to correspond with PCAP and PISA as well as respond to the current challenges.
“New” New Brunswick Assessment Program (NBAP)
Grade 6 and Grade 10 (Census) and Grade 4 (Field Test)
•Reading, Mathematics and Scientific Literacy Grade 6 Reading will be by program of instruction; PIF booklets will include a short section in French Reading; Math and Scientific Literacy will include bilingual booklets for students in the FI Program.
Grade 10 Math and Scientific Literacy will include all students and booklets will be bi-lingual where appropriate. English literacy will be assessed in the ELPA/R at Grade 9 and will not be included.
Grade 2 Reading (annual, census, administered in May)Grade 2 Reading will be in English to ensure the gains in early literacy are sustained and continued growth is monitored.
Grade 9 ELPA (Graduation Requirement) - Grade 11/12 ELPA/R
Grade 6, 10 French Oral Proficiency (10% stratified by Program)
Grade 12 French Oral Proficiency CertificationThe existing Grade 12 will become mandatory for all students in French Immersion.
English Language Proficiency Assessment – ELPA/R
ACCOMMODATIONS AND EXEMPTIONS
Provincial assessments are compulsory for all students enrolled in New Brunswick schools.
In certain circumstances, it is necessary and appropriate for students to be exempted from participating in provincial assessments if the testing format does not allow demonstration of their learning, even with accommodations.
Exemptions fall into three broad categories: academic, social/emotional and unforeseen. Typically, academic exemptions are sought for students following an individualized Special Education Plan (SEP-I). Students who are exempt due to an SEP-I should be assessed on those goals that are set out in their Special Education Plan(SEP).
ACCOMMODATIONSAccommodations are those supports, methods and practices that are used in the classroom enabling a student to participate as independently as possible.
There are many ways in which accommodations can be used to support students when they are participating in a provincial assessment. Within the context of provincial assessments, students may require either universal or justifiable accommodation.
For the purpose of large scale assessment, accommodations are alterations that do not jeopardize the integrity or content of the test, yet give students an equal opportunity to demonstrate what they know and can do as part of the assessment process.
Types of Accommodations
UNIVERSAL
Universal accommodations are those strategies, technologies or adjustments that enable a student to reach prescribed outcomes and can be used at the teacher’s or student’s discretion. Universal accommodations do not necessitate an SEP or prior approval for provincial assessments.
JUSTIFIABLE
Justifiable accommodations are documented strategies, technologies or adjustments without which the student would not be able to demonstrate knowledge. These accommodations are documented within the SEP and require prior approval to be accessed during provincial assessments.
EXEMPTIONS
In certain circumstances, it is not appropriate for students to participate in provincial assessments. Any decision to exempt a student from any provincial assessment must be made in collaboration with the student’s parent(s) or guardian(s).
Within the context of provincial assessments, students may require
an exemption for academic, social/emotional or unforeseen reasons.
Types of Exemptions
ACADEMICThe student has been identified with exceptionalities and has a current individualized Special Education Plan (SEP-I) that identifies the need for the requested exemption in the assessed subject area.
Typically, these students are following an SEP-I and are not expected to attain the curricular outcomes and standards. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the SEP-I.
Typically, this constitutes the largest category of exemption requests.
SOCIAL/EMOTIONAL
If the student has been identified with a significant social, emotional or behavioural impairment and cannot be accommodated by such means as alternate location, extra time, etc., an exemption may be sought.
Typically, these students are following an Individual Behaviour Support Plan (IBSP) and are not expected to attain the curricular outcomes and standards at their grade level. This has been communicated to parent(s) or guardian(s) and an alternate evaluation of progress has been determined within the IBSP.
.
UNFORESEEN
In the event of serious illness, bereavement or other exceptional and unforeseen circumstance, an on-the-spot exemption can be granted.
These situations require that the director of education be contacted immediately..
New Brunswick Assessment Program
Operational Standards Processes How items are chosen
Standards for Educational and Psychological Testing
The Standards for Educational and Psychological Testing, 2008 edition are a collaborative publication from the American Educational Research Association, the American Psychological Association and the National Council on Measurement in Education. The intent of the Standards is to promote the sound and ethical use of tests and to provide a basis for evaluating the quality of testing practices.
The New Brunswick Assessment and Evaluation Branch adheres to the Standards when developing, administering, scoring and analysing large scale assessments within the scope of our mandate.
Operational Standards
Standards for Educational and Psychological Testing
The Assessment Cycle
There are four (4) stages within the overall Assessment Cycle for each of the 13 assessments; the (1) Development Stage, (2) Administration Stage, (3) Data Stage and the (4) Communication Stage.
Within each of the stages, there are several phases which require the expertise from a variety of staff.
Each of these stages require strict adherence to the Standards for Educational and Psychological Testing and the Principles for Fair Student Assessment in Canada.. These standards and principles will be referenced where appropriate.
Processes
0
0 .2
0 .4
0 .6
0 .8
1 .0
-3 -2 -1 0 1 2 3
b
Ab i l i ty
Pro
bab
ilit
y
Item Characteristic Curve: ITEM0001
a = 0 .5 3 9 b = 0 .2 0 2
1 - 7
8 - 14
15 - 21
22 - 28
29 - 35
36 - 42
43 - 48
Matrix Plot of Item Characteristic Curves for Book A
1 - 7
8 - 14
15 - 21
22 - 28
29 - 35
36 - 42
43 - 48
Matrix Plot of Item Characteristic Curves for Book A
Development of the Provincial Assessment at Grade 6
Blue PrintAssessment Blue Prints are developed to guide the item development process. The Blue Print may include: (Mathematics Example)
Assessment Type:
Items Per Booklet: Mathematics
Assessment Format – 4 Booklet Matrix SampleScience and Mathematics will be written in the same booklet by Prime and French Immersion students. Each booklet will be bilingual (English on the left-hand side and French on the right-hand side).
•Mathematics section•Scientific Literacy section•Literacy (English) section•Literacy (French) section
Booklet AScience
and Math
Booklet BScience
and Math
Booklet CScience
and Math
Booklet DScience
and Math
2014-2015 Provincial Mathematics Assessment at Grade 8
Items and Rubrics