new england common assessment program grades 3-8 & 11 bias/sensitivity review- part 2 march 26...
TRANSCRIPT
New England Common Assessment Program
Grades 3-8 & 11
Bias/Sensitivity Review- Part 2
March 26 and 27, 2006
Quechee, VT
What Is Item Bias?
Bias is the presence of some characteristic of an assessment item that results in the differential performance of two individuals of the same ability but from different student subgroups
Bias is not the same thing as stereotyping (but we don’t want stereotypes in NECAP either)
We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills
The “Big Idea” of Bias-Sensitivity Review
NECAP should measure the results of learning opportunities NOT life experiences
Reading Passages Selected by Item Developers
Bias-Sensitivity Review of Reading Passages – Focus on Sensitivity Issues
Reading and Math Items are Developed Bias-Sensitivity Review of Reading & Math
Items – Focus on Item Bias Item Review of Reading and Math Items Field-Testing Feedback Pilot-Testing Data Analysis (DIF)
How Do We Prevent Item Bias?
Review reading passages, and then both math and reading items
Complete independent ratings of each passage or item and record comments
Be prepared to discuss “Mixed Reviews” and “Problem Items” with your review team; Finalize independent ratings
Please Note: The Bias-Sensitivity Review Committee plays an advisory role. Final decisions on which passages and items to
include in NECAP will be made by the NH, RI and VT DOEs
Role of the Bias-Sensitivity Review Committee
Sensitivity to different cultures, religions, ethnic and socio-economic groups, and disabilities
Balanced gender roles that are free of gender stereotypes Use of positive language, situations and images Use of context, situations, images and examples that are
universally understood across groups and cultures Overall, that the test is free of items and text that may elicit
strong emotions in specific groups of students, and as a result, prevent those groups of students from accurately demonstrating their skills and knowledge
Overall, that the test is free of items that measure life experiences instead of learning experiences
Role of the Bias-Sensitivity Review Committee
The Bias-Sensitivity Review Committee DOES need to make recommendations that will help ensure…
Reading Level Grade Level Appropriateness GE Alignment Instructional Relevance Language Structure and Complexity Accessibility Overall Item Design
Role of the Bias-Sensitivity Review Committee
The Bias-Sensitivity Review Committee DOES NOT need to make recommendations concerning…
The Item Review Committees Are
Addressing These Issues and Factors
December 12th and 13th – Review All Reading Passages
March 26th and 27th – Review All Reading and Mathematics Test Items
Process: Today – (1) Training, (2) Practice, (3) Clarification of
Guidelines and Procedures, (4) Break Into Two Groups for Independent Review of Item Sets – Group 1 will review reading and math for grades 3, 5 and 7 and math for grade 11, and Group 2 will review reading and math for grades 4, 6 and 8 and reading/writing for grade 11.
Tomorrow – (1) Break into Small Groups for Re-Evaluation and Discussion of “Problem Items” (2) Finalize Independent Ratings, (3) Evaluate the Review Process and Provide Feedback
The Bias-Sensitivity Review Process
Avoid issues and topics considered sensitive by the general public
Keep in mind that some topics that are appropriate for classroom discussion may not be appropriate for assessment
Consider context and setting; Bias-Sensitivity Review is not a “literal” process
Think of “bias” as an assessment issue as well as a cultural issue, that is, a specific topic or feature of a passage or item that may disadvantage one of two otherwise equally matched groups of students
General Guidelines
Abortion Birth Control Child Abuse/Neglect Creationism Divorce Incest Occult/Witchcraft Questioning
Parental Authority
Rape Religious Doctrine Sex/Sexuality Sexual Orientation Sexually Transmitted
Diseases Suicide Weight
Topics to Avoid in Reading Passages & Test Items
Death Drugs, Alcohol &
Tobacco Family Issues Guns/ Gun Control Handicapping
Conditions Holidays Homelessness
Misuse of Animals/ Animal Rights
Murder Pregnancy Racism, Sexism &
Ageism Religion (Except in
Historical Context) Violence
Topics Which May or May Not Be Acceptable
Avoid economic, regional, cultural or gender bias in questions, prompts, identified situations, graphics or reading selections
Avoid profanity unless it has a clear literary or historical context
Keep in mind that the ability to answer a test question should reflect learning opportunities not cultural opportunities or life experience
Remember that we are proud of our states and region and expect to see that pride reflected in assessment passages and items
Awareness Issues
New England Common Assessment ProgramBias & Sensitivity Review Committee
Item Review Evaluation Form
Read the statement below and select the rating that best describes your evaluation of the item for potential bias and sensitivity concerns.
“This item does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students.”
Rating Description
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Comments
Please provide specific comments to explain your evaluation. Your comments should address your particular bias/sensitivity concerns, the subgroup(s) that you feel may be
impacted, and the extent of the potential impact within the subgroup(s).
Item Review Rating Form
Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives.
Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.”
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
What’s the Point?
“Avoid ECONOMIC, regional, CULTURAL or gender bias in questions, prompts, identified situations…”
Okay because the writer’sculture and economic status
are not factors in the passage
Not okay because the passageassumes a cultural and
economic status that won’tbe shared by all test-takers
Once, I got my head stuck in our porch railing. If that man with a cell phone hadn’t stopped and called the fire department, I’d probably still be there. Cell phones can save lives.
Everybody at school has a cell phone. Once, during an English test, somebody actually got a call and in a regular speaking voice, said, “Sorry, I can’t talk now because I’m taking an English test.”
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Larry buys an iPod ® that has a regular price of $120. The iPod ® is on sale for 30% off. What is the sale price of the iPod ®?
A. $150
B. $84
C. $36
D. $90
Larry buys a sweater that has a regular price of $40. The sweater is on sale for 30% off. What is the sale price of the sweater? Show your work or explain how you know.
What’s the Point?
“Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”
Not okay because many students can’t afford luxury items such as
IPods
Okay because clothing isa necessity and $40 is a
more reasonable amount
“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Larry buys an iPod ® that has a regular price of $120. The iPod ® is on sale for 30% off. What is the sale price of the iPod ®?
A. $150
B. $84
C. $36
D. $90
Larry buys a sweater that has a regular price of $40. The sweater is on sale for 30% off. What is the sale price of the sweater? Show your work or explain how you know.
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
② Explain how the Model T differed from earlier attempts to build a “horseless carriage.” Use examples from the story.
(Related to a reading passage called “Henry’s Horseless Carriage.”)
② Explain how the antique cars described in the story are different than modern cars. Provide at least 3 examples.
(Related to a reading passage called “Henry’s Horseless Carriage.”)
What’s the Point?
“Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”
Okay because student isonly required to use
information contained inthe passage.
Not okay because the studentneeds both information
from the story AND informationfrom personal experience
“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Explain how the Model T differed from earlier attempts to build a “horseless carriage.” Use examples from the story.
(Related to a reading passage called “Henry’s Horseless Carriage.”)
Explain how the antique cars described in the story are different than modern cars. Provide at least 3 examples.
(Related to a reading passage called “Henry’s Horseless Carriage.”)
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Mount Tim is 3 miles from bottom to top and has a vertical rise of 1800 feet.
A mountain goat can travel up a mountain at an approximate speed of 2 miles per hour, plus an additional ½ hour per thousand feet of vertical rise. How long will it take a mountaingoat to travel from the bottom to the top of Mount Tim? Show your work or explain how you know.
1800 feet3 miles
The Mount Tim Hiking Trail ascends 1800 feet in 3 miles.
The Appalachian Mountain Club Trail Guide estimates hiking time using ½ hour per mile, plus an additional ½ hour per thousand feet of vertical rise. Approximately how long will it take to hike the Mount Tim Trail? Show your work or explain how you know.
1800 feet3 miles
What’s the Point?
“We need to ensure that ALL students have an equal opportunity to demonstrate their knowledge and skills”
Okay because the context isgenerally understood by
all students.
Not okay because somestudents (particularly VT & NH
students) may have experience using the trail guide formula
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Mount Tim is 3 miles from bottom to top and has a vertical rise of 1800 feet.
A mountain goat can travel up a mountain at an approximate speed of 2 miles per hour, plus an additional ½ hour per thousand feet of vertical rise. How long will it take a mountaingoat to travel from the bottom to the top of Mount Tim? Show your work or explain how you know.
1800 feet3 miles
The Mount Tim Hiking Trail ascends 1800 feet in 3 miles.
The Appalachian Mountain Club Trail Guide estimates hiking time using ½ hour per mile, plus an additional ½ hour per thousand feet of vertical rise. Approximately how long will it take to hike the Mount Tim Trail? Show your work or explain how you know.
1800 feet3 miles
“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
① The word hear belongs in which sentence?
A. People come ____ for vacation.
B. I want to sit _____ by the window.
C. Take cover when you _____ thunder.
D. Please put my books over _____.
① The word hear belongs in which sentence?
A. People come ____ for vacation.
B. I want to sit _____ by the window.
C. She can _____ the loud thunder.
D. Please put my books over _____.
?
?
What’s the Point?
“Avoid economic, regional, CULTURAL or gender bias in questions, prompts, identified situations…”
Is this NOT okay becausethunder may not be
“universally understood” bypersons who are deaf?
Is this better? Is itokay because the sentencerefers to another person’shearing, not the reader’s
Bias-Sensitivity Examples and Non-Examples
“This passage does not raise bias and/or sensitivity concerns that would interfere with the performance of a group of students”
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
Rating Descriptions
4 Strongly Agree
3 Agree
2 Disagree
1 Strongly Disagree
The word hear belongs in which sentence?
A. People come ____ for vacation.
B. I want to sit _____ by the window.
C. Take cover when you _____ thunder.
D. Please put my books over _____.
The word hear belongs in which sentence?
A. People come ____ for vacation.
B. I want to sit _____ by the window.
C. She can _____ hear the loud thunder.
D. Please put my books over _____.
“(Use) context, situations, images and examples that are UNIVERSALLY UNDERSTOOD across groups and cultures”
The “Big Idea” of Bias-Sensitivity Review
NECAP should measure the results of learning opportunities NOT life experiences