New England Common Assessment Program

Bias-Sensitivity Review Committee Meeting

December 5 & 6, 2005

Manchester, NH

What Is Item Bias?

Bias is the presence of some characteristic of an assessment item that results in the differential performance of two individuals of the same ability but from different student subgroups

Bias is not the same thing as stereotyping (but we don’t want stereotypes in NECAP either)

We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills

Reading Passages Selected by Item Developers

Bias-Sensitivity Review of Reading Passages Reading and Math Items are Developed Bias-Sensitivity Review of Reading & Math

Items Item Review of Reading and Math Items Field-Testing Feedback Pilot-Testing Data Analysis (DIF)

How Do We Prevent Item Bias?

Review reading passages, and then both math and reading items

Complete independent ratings of each passage or item and record comments

Be prepared to discuss “Mixed Reviews” and “Problem Passages” with your review team; Finalize independent ratings

Please Note: The Bias-Sensitivity Review Committee plays an advisory role. Final decisions on which passages and items to

include in NECAP will be made by the NH, RI and VT DOEs

Role of the Bias-Sensitivity Review Committee

Sensitivity to different cultures, religions, ethnic and socio-economic groups, and disabilities

Balanced gender roles that are free of gender stereotypes

Use of positive language, situations and images Overall, that the test is free of items and text that

may elicit strong emotions in specific groups of students, and as a result, prevent those groups of students from accurately demonstrating their skills and knowledge

Role of the Bias-Sensitivity Review Committee

The Bias-Sensitivity Review Committee DOES need to make recommendations that will help ensure…

Reading Level Grade Level Appropriateness GLE Alignment Instructional Relevance Language Structure and Complexity Accessibility Overall Item Design

Role of the Bias-Sensitivity Review Committee

The Bias-Sensitivity Review Committee DOES NOT need to make recommendations concerning…

The Item Review Committees Will

Address These Issues and Factors

December 5th and 6th – Review All Reading Passages

April 10th and 11th – Review All Reading and Mathematics Test Items

Today – (1) Training, (2) Practice, (3) Large Group Review and Discussion of the First Passage Set, (4) Clarification of Guidelines and Procedures, (5) Break Into Small Groups for Independent Review of Remaining Passage Sets

Tomorrow – (1) Break into Small Groups for Re-Evaluation and Discussion of “Mixed Review” and “Problem Passages,” (2) Finalize Independent Ratings, (3) Evaluate the Review Process and Provide Feedback

The Bias-Sensitivity Review Process

Avoid issues and topics considered sensitive by the general public

Keep in mind that some topics that are appropriate for classroom discussion may not be appropriate for assessment

Consider context and setting; Bias-Sensitivity Review is not a “literal” process

Think of “bias” as an assessment issue as well as a cultural issue, that is, a specific topic or feature of a passage or item that may disadvantage one of two otherwise equally matched groups of students

General Guidelines

Abortion Birth Control Child Abuse/Neglect Creationism Divorce Incest Occult/Witchcraft Questioning

Parental Authority

Rape Religious Doctrine Sex/Sexuality Sexual Orientation Weight Suicide Sexually Transmitted

Diseases

Topics to Avoid in Reading Passages & Test Items

Death Drugs, Alcohol &

Tobacco Family Issues Guns/ Gun Control Handicapping

Conditions Homelessness Holidays

Murder Racism, Sexism &

Ageism Pregnancy Violence Misuse of Animals/

Animal Rights Religion (Except in

Historical Context)

Topics Which May or May Not Be Acceptable

Avoid economic, regional, cultural or gender bias in questions, prompts, identified situations, graphics or reading selections

Avoid profanity unless it has a clear literary or historical context

Keep in mind that the ability to answer a test question should reflect learning opportunities not cultural opportunities or life experiences.

Remember that we are proud of our states and region and expect to see that pride reflected in assessment passages and items

Awareness Issues

Passage Review Rating FormNew England Common Assessment Program

Bias/Sensitivity Committee Meeting Passage Review

Grad Level:

Passage # Title

Passage Evaluation

Read the statement below and select the rating that best describes your evaluation of the passage for potential bias and sensitivity concerns.

“This passage does not raise bias and/or sensitivity concerns that will interfere with the performance of a group of students.” Rating Description

4 Strongly Agree 3 Agree 2 Disagree 1 Strongly Disagree

Comments/Explanation

Please provide specific comments to explain your evaluation. Your comments should address your particular bias/sensitivity concerns, the subgroup(s) that you feel may be impacted, and the extent of the potential impact within the subgroup(s)

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

“You’re going straight to hell, Jimmy Smith,” Donna yelled at the boy who had just knocked her books out of her hands.

President Truman, who rarely backed down from a confrontation with other world leaders, was known as “Give ‘em Hell Harry.”

Bias-Sensitivity Examples and Non-Examples

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Avoid profanity unless it has a clear literary or historical context”

Not okay because itdoesn’t meet the “general public” standard for how 3-8 graders should talk

Okay because ofhistorical context

for profanity

“You’re going straight to hell, J immy Smith,”Donna yelled at the boy who had just knocked her books out of her hands.

President Truman, who rarely backed down from a confrontation with other world leaders, was known as “Give ‘em Hell Harry.”

Bias-Sensitivity Examples and Non-Examples

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

My sister met Kyle at a high school dance and nine months later I was an uncle.

When dad brought the broken-down mare home from the livestock auction, we all thought he had thrown his money away. Now there’s a handsome colt in the next stall, so I guess the old horse was a good deal after all.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Topics to Avoid: Sex/Sexuality”

Not okay because thepassage alludes to

extra-marital sex betweenhigh school students

Okay because our generalpublic standard generally

allows for the notion ofanimal parents and babies

My sister met Kyle at a high school dance and nine months later I was an uncle.

When dad brought the broken-down mare home from the livestock auction, we all thought he had thrown his money away. Now there’s a handsome colt in the next stall, so I guess the old horse was a good deal after all.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

The Christmas Tree Fern is a common sight in the New England woodlands.

Christmas is the most wonderful time of the year.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Topics Which May or May Not Be Acceptable: Holidays, Religion”

Okay because “Christmas” refers to the name of the

plant; Not a specific reference to religion

Not okay because thereference is based on the

writer’s cultural and religiousvalues (not shared by all)

The Christmas Tree Fern is a common sight in the New England woodlands.

Christmas is the most wonderful time of the year.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Bill was blind as a bat.

Judith Heumann was the first person with a disability to be licensed to teach in New York State.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Topics Which May or May Not Be Acceptable: Handicapping Conditions”

Not okay because it makesa degrading reference to

a person with a disability

Okay because thereference highlights the

positive accomplishmentsof a person with a disability

Bill was blind as a bat.

J udith Heumann was the first person with a disability to be licensed to teach in New York State.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives.

Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.”

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Avoid ECONOMIC, regional, CULTURAL or gender bias in questions, prompts, identified situations…”

Okay because the writer’sculture and economic status

are not factors in the passage

Not okay because the passageassumes a cultural and

economic status that won’tbe shared by all test-takers

Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives.

Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.”

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

It was ten years after Uncle Ned’s death when we found his old sea trunk in the attic.

Mary cried and cried when she was told Uncle Ned had died.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Topics Which May or May Not Be Acceptable: Death”

Okay because the deathreference is used to set the

passage’s time and place but is not a key plot feature

Not okay because death isa key plot feature; deals

with emotions that may beupsetting to some test-takers

It was ten years after Uncle Ned’s death when we found his old sea trunk in the attic.

Mary cried and cried when she was told Uncle Ned had died.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Pa and Uncle Jimmy drove the final nails into the new barn roof while Ma and I laid out supper. For settlers on the Great Plains in the 1840’s, a barn raising was hard work, but also a time for celebration

Never give a girl a hammer ‘cause she’s liable to clobber herself or anybody in reach.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

“Avoid economic, regional, cultural or GENDER BIAS in questions, prompts, identified situations…”

Okay because the genderspecific roles in the passageare historically accurate and

do not, per se, represent bias

Not okay because thewriter’s perspective

represents a clear and unacceptable gender bias

Pa and Uncle J immy drove the final nails into the new barn roof while Ma and I laid out supper. For settlers on the Great Plains in the 1840’s, a barn raising was hard work, but also a time for celebration

Never give a girl a hammer ‘cause she’s liable to clobber herself or anybody in reach.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

“Never give a girl a hammer ‘cause she’s liable to clobber herself or anybody in reach.” That’s what I used to think until the day I spied Mary Jean McCubbins up in a tree, building the coolest tree fort I ever saw.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree

What’s the Point?

Okay because the writeridentifies the gender bias inprior beliefs and attempts to correct the stereotype

“Avoid economic, regional, cultural or GENDER BIAS in questions, prompts, identified situations…”

“Never give a girl a hammer ‘cause she’s liable to clobber herself or anybody in reach.”That’s what I used to think until the day I spied Mary J ean McCubbins up in a tree, building the coolest tree fort I ever saw.

Bias-Sensitivity Examples and Non-Examples

“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”

Rating Descriptions

4 Strongly Agree

3 Agree

2 Disagree

1 Strongly Disagree