Martha Thurlow and Laurene ChristensenNational Center on Educational Outcomes

CEC Preconvention Workshop #4April 21, 2010

Martha ThurlowLaurene Christensen Introduce yourself –

Name State One more thing about you

2

Review five-step process for monitoring accommodations decision making and implementation

Identify strategies for improving the use of accommodations in instruction and assessment

Share tools and techniques for checking on decision making and use

Discuss how to address validity issues related to accommodations

3

Accommodations – Changes in materials or procedures that enable students to meaningfully access instruction and assessment. Assessment accommodations do not change the construct that is being measured.

Accommodations mediate the effects of a student’s disability and do not reduce learning expectations.

4

Modifications – Changes in materials or procedures that enable students to access instruction and assessment. Assessment modifications do change the construct that is being measured.

Modifications create challenges for assessment validityAccommodations mediate the effects of a student’s disability and do not reduce learning expectations.

5

6

Another way to talk about this -------

Accommodations remove construct irrelevant variance – that is, variability that may be systematic (e.g., the effect of the disability), but that is not what the test is intended to measure.

Modifications may also remove construct relevant variance – that is, variability that may be systematic (e.g., the effect of the disability) – but in this case, it is also related to what the test is intended to measure.

7

Accommodations are intended to produce valid measures of what a student knows and is able to do. To determine whether each is valid requires relating the accommodation to the standards and content that is being assessed.

Standard accommodations ?

Nonstandard accommodations ?

Conditional accommodations ?

Adaptations ?

Other terms ? Share your terminology – and what might be confusing about the terms.

8

Binder with agenda, powerpoints, resources, and exercises

Accommodations Monitoring Tool, with examples, checklists, additional resources, and glossary

Accommodations Manual, with tools Hints and Tips Accommodations Video Flash Drive with materials

9

Analyses of accommodation policies (1992-2009) and effects (1999-2008)

Accommodations for computer-based tests (2002)

Accommodations manuals projects (2005, 2009) Principled Approach to Accountability

Assessments Peer review project

◦ Hints and Tips Tool◦ Technical Report on Peer Review of Accommodations

10

1. Selection of accommodations2. Linking of instructional and assessment

accommodations3. Monitoring accommodations4. Accommodations use allows for valid

and meaningful scores

Big Issue that Emerged: Monitoring

accommodations11

Developed in collaboration with CCSSO Reviewed extensively by ASES SCASS group Based on a review of publicly available materials 5 Step process Each step includes examples, a checklist,

additional resources Appendices include additional examples from

states Extensive glossary at the end

12

Federal Laws that Pertain to Monitoring Accommodations◦IDEA◦ESEA

What is monitoring?

13

IDEA (Individuals with Disabilities Education Act)◦Requires students with disabilities who need

accommodations to have them◦Document accommodations on an IEP◦Monitor the educational results for students◦Ensure States follow through with the

requirements of IDEA

Nothing specific about monitoring accommodations.

14

ESEA (Elementary and Secondary Education Act)◦Requires accommodations as necessary in

order to measure student achievement◦Requires accommodations use to result in a

valid score

Monitoring is identified in the standards and assessment peer review materials.

15

So, what is meant by: “monitoring accommodations”?

16

Monitoring accommodations is a process of evaluating policies, procedures, and practices in order to ensure that a state’s assessment system is inclusive of students with disabilities and English language learners.

17

The State has analyzed the use of specific accommodations for different groups of students with disabilities and has provided training to support sound decisions by IEP teams.

The State routinely monitors the extent to which test accommodations are consistent with those provided during instruction.

But, it is about more than compliance . . .18

Systematic attention to the provision of accommodations can ensure that students are able to show what they know and can do

19

Training for decision-making teams Evaluating IEP team decisions about

accommodations Tracking accommodations students

use for instruction and assessment Tracking whether the use of

accommodations results in a valid score for the student

Reporting the use of accommodations

20

21

Break!

22

Keeping track of training Keeping track of IEP decisions on

accommodations Are accommodations used for

instruction consistent with those used for assessments?

Keeping track of requests for unique accommodations

23

Questions to ask: What kinds of training on

accommodations are offered by the state or district?

Who attends the trainings? How are materials made available to

participants and others who might need them?

Are materials available online?

24

IEP Team Making Decisions about Accommodations

25

Your ideas? Summarize on Flip Chart

26

Are there model IEP forms for schools to use?

Does the IEP form reference current state policy in some way so that teams are aware of the consequences of their decisions?

27

Do IEP forms indicate instructional and assessment accommodations separately?

Does the electronic data system (if one exists) allow for matching instructional and assessment accommodations?

28

Documentation of accommodations on IEPs?

Connection between instructional and assessment accommodations?

Ease of access to accommodations information?

29

30

1. Do decision-making teams know how to make special requests for accommodations?

Do they know who to contact? Do they know what information should be provided to help the state make a determination?

2. Is there a clear process in place so that teams know how and when decisions on special requests will be made?

3. How are special requests tracked?

31

32

33

What we have done so far: Background Step 1 – Know rules and regulations Step 2 – Document decisions about accommodations What is still to come: Step 3 – Document use of accommodations Step 4 – Review accommodations decisions and

use Step 5 – Evaluate and report on accommodations

34

Documenting accommodations on student demographic sheets

Documenting inappropriate use of accommodations on test day

Documenting that students received their IEP accommodations on test day

35

What information is asked for? How are accommodations listed? Are invalidating accommodations

(modifications) noted on the bubble sheet? What about “Other” accommodations?

36

Information about the Student

37

Information on accommodations

38

Accommodations – Provided or Used?

39

Bubble Sheets:

40

Strengths

Needed Improvements

Testing irregularity forms

Testing certification forms

How are these forms used? How are they reviewed? What is done with

the information?

41

42

43

Direct observation on test day Record reviews On-site visits on a day other than test

day Interviews with students, teachers, and

administrators about the use of accommodations

44

Is there an established process? How are schools chosen? Are visits announced or unannounced?

◦ Are there requirements to prepare in advance?

Example: Arkansas

45

How will records be reviewed?

How will records be selected?

How will information on accommodations be tracked?

46

1. Are the accommodations/modifications appropriate for the skill area affected by the disability (no oral testing for math disability)?

Yes No If no, example:

2. Are the accommodations identified in the IEP for state and district-wide assessment provided in their instructional program? (Do they match?)

Yes No If no, example:

3. Were the accommodations identified in the IEP for state and district-wide assessment “USED” during assessment administration? (compare the coding on the assessment data sheet with the assessment accommodations listed inthe IEP)

Yes No If no, example:

4. If the student is identified as taking an alternative assessment, does he or she meet the criterion and has it been documented on the IEP?

Yes No If no, example:

Monitoring Questions from South Dakota

47

Can be for compliance, or for technical assistance and professional development

May serve as a needs assessment May or may not include interviews with

stakeholders, such as administrators, teachers, and students

48

Utah Utah ExampleExample

49

South South Carolina Carolina ExampleExample

50

1. What accommodations do you use for instruction? How do they help you learn?

2. What accommodations do you use for assessments? How do they help you do your best?

3. Do you have any questions about the accommodations you use for instruction or assessments?

4. How do your teachers make sure you have the accommodations you need on test day?

5. Is there anything else about the accommodations you use that you want to share?

51

1. How do you ensure that students receive the necessary accommodations as indicated on the IEP?

2. How do IEP teams make decisions about what assessment a student will be given (i.e., what data are used to make the determination)?

3. How do IEP teams make decisions about what assessment accommodations a student needs? How do you ensure that accommodations happen?

4. How are families/parents involved in assessment and accommodations selection?

5. How is the provision of accommodations monitored by the school?

52

1. How do you ensure that students receive the necessary accommodations as indicated on the IEP?

2. How do IEP teams make decisions about what assessment a student will be given (i.e., what data are used to make the determination)?

3. How do IEP teams make decisions about what assessment accommodations a student needs? How do you ensure that accommodations happen?

4. How are families/parents involved in assessment and accommodations selection?

5. How is the provision of accommodations monitored by the school?

6. As an administrator, how do you ensure that decision- making teams receive appropriate training on accommodations?

53

54

Arkansas Case Study: Discussion

55

56

Break!

Analyze accommodations

Report on accommodations

Revise accommodations policies

57

Alternatives to a formal research study:

1. Document process of collecting information on accommodations

2. Look at existing research on accommodations

3. Conduct extant data analyses

4. Triangulation: formal lit review, expert judgment, and empirical evidence

58

59

Hawai’i Example

Discrepancy Analysis

60

Accommodations Report

Delaware Example

61

Resource: NCEO Dataviewer

http://data.nceo.info

62

63

64

Next Steps!

65

66

Contact Information

Martha Thurlow: THURL001@umn.edu

Laurene Christensen: chri1010@umn.edu

National Center on Educational Outcomes (NCEO):

www.nceo.info