how do we know what they know? fass meeting orlando, fl april 5, 2004 arthur eisenkraft...
TRANSCRIPT
![Page 1: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/1.jpg)
How do we know what they know?
FASS MeetingOrlando, FLApril 5, 2004
Arthur Eisenkraft ([email protected])
![Page 2: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/2.jpg)
Are we listening?
• The optometrist
• The Duracell competition
• Two objects falling in a vacuum
• The cord of wood
![Page 3: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/3.jpg)
No Child Left Behind
• Enormous concern about NCLB and other high stakes assessments.
• NCLB – a potential nightmare – – AYP - “need of improvement”– public embarrassment– best students leaving and the scores dropping more,
closing of schools. • What can we do?
– This year alone 26,100 of the nation’s 91,400 have been labeled “schools that need improvement.” (Sam Dillon, 1 in 4 Schools Fall Short Under Bush Law, N.Y. Times, January 27, 2004 at A21)
![Page 4: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/4.jpg)
NCLB - Advice• Assessment in Support of
Instruction and Learning: Bridging the Gap Between Large-Scale and Classroom Assessment - Workshop Report (2003)Board on Testing and Assessment, Mathematical Sciences Education Board, Center for Education
• www.nap.edu
![Page 5: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/5.jpg)
The Deborah Meier Amendment
• A basic test should first be taken by the folks we honor by electing to office.
• The people who legislate or mandate a test should be required first to take it themselves to ensure that it's measuring what they think it is. It's a form of validity checking.
• They might even have their scores posted!
• Seeking Alternatives to Standardized Testing • By Jay Mathews, Washington Post Staff Writer
Tuesday, February 17, 2004; 10:46 AM
![Page 6: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/6.jpg)
Today’s Discussion
• Formative classroom assessment can positively impact instruction and therefore is our best approach to students performing better on all tests.
• What can teachers do?
• What can we do to support teachers?
![Page 7: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/7.jpg)
Brief History of Assessment
• When did it all begin?
– No Child Left Behind– FCAS– New York State Regents
![Page 8: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/8.jpg)
![Page 9: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/9.jpg)
![Page 10: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/10.jpg)
How do we assess?
• Please discuss– Observation– Portfolio– Projects– Questioning– Paper and pencil– Interview– Presentation– Checklist– Skills– Self assessment– Quizzes– conferences
![Page 11: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/11.jpg)
Classroom Assessment The Grade Book
TestsQuizzesHomeworkClass participation (?)Lab reportsAttendance (X)Projects
The Final ExamLocalState – High stakes
These are often treated as summative though they do inform as formative.
Other formative assessments include:–Questions in class–Practice tests
![Page 12: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/12.jpg)
Get tests back immediately
• They can then be used for formative assessment.
• How can anyone continue instruction when you have a tool that informs you of student understanding?
• Easily measured by supervisors and students alike
![Page 13: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/13.jpg)
Formative Assessment
• The value of formative assessment (Paul Black): students often have limited opportunities to understand or make sense of topics because many curricula have emphasized memory rather than understanding. Textbooks are filled with facts that students are expected to memorize, and most tests assess students’ abilities to remember the facts.
![Page 14: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/14.jpg)
What Goes Wrong?
Tests that do not correlate with understanding•Force Concept Inventory (FCI)•Regents exam question on moving galaxies•Private Universe videotapes
![Page 15: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/15.jpg)
What Goes Wrong?
Tests that do not correlate with understanding•Force Concept Inventory (FCI)•Regents exam question on moving galaxies•Private Universe videotapes
We’re not testing what we teach•Harris cartoon of mouse and maze
![Page 16: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/16.jpg)
![Page 17: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/17.jpg)
What Goes Wrong?
Tests that do not correlate with understanding•Force Concept Inventory (FCI)•Regents exam question on moving galaxies
We’re not testing what we teach•Harris cartoon of mouse and maze
We’re not teaching what we test•“Waldo” phenomenon
![Page 18: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/18.jpg)
![Page 19: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/19.jpg)
![Page 20: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/20.jpg)
![Page 21: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/21.jpg)
![Page 22: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/22.jpg)
![Page 23: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/23.jpg)
Improvements
Rubrics•Clearly defined grading schema – matrix•B++++ and A-----
•Have students help create rubric•Ownership•Motivation
•Have students self – evaluate with rubric
![Page 24: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/24.jpg)
The grading rubric +/-
Student Grade Teacher Grade
A A
A C
C C
![Page 25: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/25.jpg)
Improvements
Rubrics•Clearly defined grading schema – matrix•B++++ and A-----•student and teacher comparisons: A,A or C,C or A,C or C,A
•all require very different discussions
Saphier effective instruction•testing for understanding
•how do you know what the students know?
![Page 26: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/26.jpg)
Cognitive Empathy
With references from
The Skillful Teacher
-Jon Saphier
![Page 27: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/27.jpg)
Checking for Understanding
Knowing when students don’t understand suggests that teachers have means for checking for understanding.
What means do we have for checking for understanding?
![Page 28: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/28.jpg)
Checking for understanding
• Presses on– No test return for 3 weeks– After math lesson, here’s your 25 problems– Take a clean sheet, we’re going on– No clue there are kids in the room– Never asks students to explain– Incorrect response, - can anyone else answer
![Page 29: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/29.jpg)
Checking for understanding
• Presses on
• Reads cues– Their looks– Eye contact– Nodding heads– Asleep or awake– Misbehavior– I can see it in their eyes
![Page 30: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/30.jpg)
Checking for understanding
• Presses on• Reads cues• “dipsticks”
– White-boarding– Short quiz– Raise the hand– Choral answers– Cards with A or B– Raise fingers with 1(index) or 2– List answers on board – which answer is the best
![Page 31: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/31.jpg)
Checking for understanding
• Presses on• Reads cues• “dipsticks”• Uses recall questions
– What do we already know– List examples– Who invented– Where’s waldo– Definitions– Name the parts of the microscope– The scientific method– Restating what is already known
![Page 32: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/32.jpg)
Checking for understanding
• Presses on• Reads cues• “dipsticks”• Uses recall questions• Uses comprehension questions
– Explain– Justify– Compare– Apply– Calculate– Why– Summarize
![Page 33: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/33.jpg)
Checking for understanding
• Presses on• Reads cues• “dipsticks”• Uses recall questions• Uses comprehension questions• Anticipates confusion
– Photosynthesis, Kreb’s cycle– Understanding that some kids are literal
• Underground railroad
– Misconceptions research– Look at prior knowledge– Teacher examining their own assumptions
![Page 34: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/34.jpg)
A TEST for Checking for Understanding
How do you know that a student understands?
What evidence do you have?
How often should you be able to answer this question?
![Page 35: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/35.jpg)
The National Science Education Standards (NSES)
Less Emphasis On: Assessing what is easily measured
More Emphasis On:Assessing what is most highly valued
Less Emphasis On: Assessing to learn what students do not know
More Emphasis On:Assessing to learn what students understand
![Page 36: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/36.jpg)
Instructional Models
• Karplus– three-phrase learning cycle
• exploration, invention and discovery
• Lawson– exploration, term introduction, and concept
application
• Bybee 5E– Engage, explore, explain, elaborate, evaluate
• 7E clarification of 5E
![Page 37: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/37.jpg)
5 E
Engage
Explore
Explain
Elaborate
Evaluate
![Page 38: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/38.jpg)
5 E 7 E
--------------------------- Elicit | |
Engage ----------------- | |
---------------------------- Engage
![Page 39: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/39.jpg)
5 E 7 E
--------------------------- Elicit | |
Engage ----------------- | |
---------------------------- Engage
Explore ------------------------------------------- Explore
![Page 40: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/40.jpg)
5 E 7 E
--------------------------- Elicit | |
Engage ----------------- | |
---------------------------- Engage
Explore ------------------------------------------- Explore
Explain ---------------------------------------------- Explain
![Page 41: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/41.jpg)
5 E 7 E
--------------------------- Elicit | |
Engage ----------------- | |
---------------------------- Engage
Explore ------------------------------------------- Explore
Explain ---------------------------------------------- Explain
-------------------------- Elaborate |
Elaborate --------------- |
-------------------------- Evaluate |
Evaluate ----------------- |
--------------------------- Extend
![Page 42: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/42.jpg)
4 Q Assessment Model
• What does it mean?
![Page 43: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/43.jpg)
4 Q Assessment Model
• What does it mean?
• How do we know?
![Page 44: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/44.jpg)
4 Q Assessment Model
• What does it mean?
• How do we know?
• Why should I believe?
![Page 45: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/45.jpg)
4 Q Assessment Model
• What does it mean?
• How do we know?
• Why should I believe?
• Why should I care?
![Page 46: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/46.jpg)
The other four questions
• What did you say?
• Should we take notes?
• When is class over?
• Will this be on the test?
![Page 47: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/47.jpg)
Challenges
Identify who are we testingoStudentsoTeachersoSchools and districts
![Page 48: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/48.jpg)
Identify for what purpose (from Classroom Assessment and the NSES, NRC)
oHelp students learn oTo illustrate and articulate the standards for quality workoTo inform teaching oTo guide curriculum selectionoTo monitor programsoTo provide a basis for reporting concrete accomplishments to interested partiesoFor accountabilityoCertification
Reporting individual achievementGradingPlacementPromotionAccountability
oParents to taxpayers(from High Stakes Assessments, NRC)
Challenges
![Page 49: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/49.jpg)
Are we trying to use ONE instrument for all (students, teachers, schools)? for all purposes?
Understanding vs. beliefoMazur student taking FCI
Challenges
![Page 50: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/50.jpg)
Classroom Assessment and the National Science Education
Standards (2001)Center for Education
Knowing What Students Know: The Science and Design of Educational Assessment (2001)Center for Education
www.nap.edu
![Page 51: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/51.jpg)
The Assessment Triangle
cognition, observation, and interpretation—must be explicitly connected and designed as a coordinated whole. If not, the meaningfulness of inferences drawn from the assessment will be compromised.
C O
I
![Page 52: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/52.jpg)
Assessment #1
• Question: What was the date of the battle of the Spanish Armada?– Answer: 1588 [correct].
• Question: What can you tell me about what this meant?– Answer: Not much. It was one of the dates I
memorized for the exam. Want to hear the others?
![Page 53: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/53.jpg)
Assessment #2
• Question: What was the date of the battle of the Spanish Armada?– Answer: It must have been around 1590.
• Question: Why do you say that?– Answer: I know the English began to settle in Virginia
just after 1600, not sure of the exact date. They wouldn’t have dared start overseas explorations if Spain still had control of the seas. It would take a little while to get expeditions organized, so England must have gained naval supremacy somewhere in the late 1500s.
![Page 54: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/54.jpg)
Comparison
• Most people would agree that the second student showed a better understanding of the Age of Colonization than the first, but too many examinations would assign the first student a better score.
• When assessing knowledge, one needs to understand how the student connects pieces of knowledge to one another. Once this is known, the teacher may want to improve the connections, showing the student how to expand his or her knowledge.
![Page 55: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/55.jpg)
Formative Assessment Research
• Black and Wiliam (1998) provide an extensive review of more than 250 books and articles presenting research evidence on the effects of classroom assessment.
• They conclude that ongoing assessment by teachers, combined with appropriate feedback to students, can have powerful and positive effects on achievement.
• They also report, however, that the characteristics of high-quality formative assessment are not well understood by teachers and that formative assessment is weak in practice. High-quality classroom assessment is a complex process
![Page 56: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/56.jpg)
Return to bad practice
• We revert back to old, failed strategies when under pressure
• Comfort food
• In dealing with our children• You’ll poke your eye out• What if everybody jumped off the Empire State
Building?
• Prison, drug addiction
![Page 57: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/57.jpg)
Responding to the pressure (NCLB, FCAS)
• If you go back to the worksheets and it doesn’t work, you won’t be blamed in the same way because this is what everybody does.
• If you continue with something that is closer to the edge, you are more open to criticism
• This could be the reason for the blame• Fail forward as Thomas Edison did
![Page 58: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/58.jpg)
Support our teachers
• Help them to not succumb to the pressures.
• Encourage them to improve their instruction, not to revert to poor pedagogy.
• Give them permission to be better teachers.
![Page 59: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/59.jpg)
Questions that foster deep understanding rather than questions that ask for repetition of memorized information
and conclusions. (Jim Minstrell and Emily van Zee)
• Ask or promote questions to open an inquiry and elicit students’ initial understanding and reasoning.
• Ask or promote questions to interpret and make sense of data in order to generate new knowledge and understanding.
• Ask or promote questions to clarify or elaborate on observations and inferences.
• Ask or promote questions to encourage learners to justify their answers and conclusions or to explain their reasoning to go beyond a mere stating of an answer.
• Ask or promote questions to extend or apply learned ides.• Ask or promote questions that help learners monitor their own
learning.
![Page 60: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/60.jpg)
Understanding by Design
• When do we generate test questions
• UbD (Grant Wiggins, Jay McTighe)– Enduring understandings– Evidence– Instruction
![Page 61: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/61.jpg)
Not in vain
• When studying about veins and arteries, for example, students may be expected to remember that – arteries are
• thicker than veins,• more elastic, and • carry blood from the heart;
– veins carry blood back to the heart.
![Page 62: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/62.jpg)
Sample test item
• Arteries– a. Are more elastic than veins– b. Carry blood that is pumped from the heart– c. Are less elastic than veins– d. Both a and b– e. Both b and c
![Page 63: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/63.jpg)
The new science of learning
• does not deny that facts are important for thinking and problem solving.
• Research on expertise in areas such as chess, history, science,and mathematics demonstrate that experts ’ abilities to think and solve problems depend strongly on a rich body of knowledge about subject matter (e.g.,Chase and Simon,1973;Chi et al.,1981;deGroot,1965).
![Page 64: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/64.jpg)
Facts are not enough
• However,the research also shows clearly that “usable knowledge ” is not the same as a mere list of disconnected facts.
• Experts’ knowledge is – connected and organized around important concepts
(e.g., Newton ’s second law of motion); – “conditionalized” to specify the contexts in which it is
applicable;– supports understanding and transfer (to other
contexts) rather than only the ability to remember.
![Page 65: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/65.jpg)
Vein and artery experts
• Know the facts in the mc question• also understand why veins and arteries have particular
properties.– They know that blood pumped from the heart exits in spurts – That the elasticity of the arteries helps accommodate pressure
changes.– They know that blood from the heart needs to move upward (to
the brain) as well as downward and that the elasticity of an artery permits it to function as a one-way valve that closes at the end of each spurt and prevents the blood from flowing backward.
• They are better able to transfer– Design an artificial artery strong enough to handle pressure with
or without elasticity (Bransford and Stein,1993).
![Page 66: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/66.jpg)
Classroom environmentsFormative assessments—ongoing assessments designed to make
students’ thinking visible to both teachers and students are essential.
They permit the teacher to • grasp the students’ preconceptions,• understand where the students are in the “developmental cor-
ridor” from informal to formal thinking, and • design instruction accordingly.
In the assessment-centered classroom environment, formative assessments help both teachers and students monitor progress.
![Page 67: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/67.jpg)
Learner friendly assessments
• Provide students with an opportunity to revise and improve their thinking (Vye et al.,1998b),
• help students see their own progress over the course of weeks or months
• help teachers identify problems that need to be remedied (problems that may not be visible without the assessments).
• Problem based learning model– Sport for the moon
![Page 68: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/68.jpg)
Defining Science Content
• Facts, process, knowledge– Order of the planets
• I went to a two day history seminar and there was no content – I didn’t learn any new dates.
• Where is the knowledge we have lost in information. (T.S. Eliot)
![Page 69: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/69.jpg)
Wait time study at King’s College (Black & Wiliam, 2000)
• Questions opened up discussion that helped expose and explore students’ assumptions and reasoning.
• At the same time, wrong answers became useful input, and the students realized that the teacher was interested in knowing what they thought, not in evaluating whether they were right or wrong.
• As a consequence, teachers asked fewer questions, spending more time on each.
![Page 70: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/70.jpg)
Wait time study at King’s College (Black & Wiliam, 2000)
• In addition, teachers realized that their lesson planning had to include careful thought about the selection of informative questions.
• They discovered that they had to consider very carefully the aspects of student thinking that any given question might serve to explore.
• This discovery led them to work further on developing criteria for the quality of their questions.
![Page 71: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/71.jpg)
Grades vs. comments
• Simply giving grades on written work can be counterproductive for learning (Butler, 1988)
• In response, teachers began instead to concentrate on providing comments without grades—feedback designed to guide students’ further learning.
• Students also took part in self-assessment and peer-assessment activities, which required that they understand the goals for learning and the criteria for quality that applied to their work.
• In these ways, assessment situations became opportunities for learning, rather than activities divorced from learning.
![Page 72: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/72.jpg)
Sliding toward Socrates
• Follow-up research– Grades vs. comments vs. grades & comments
• No distinction between instruction and assessment
![Page 73: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/73.jpg)
Beliefs and Practice
1 2 3 4 What you value
(e.g. inquiry, content)
Yo
ur
text
an
d p
rog
ram
1
2
3
4
+
+ +
![Page 74: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/74.jpg)
Beliefs and Practice
1 2 3 4 What you value
(e.g. “telling and memorizing”)
You
r te
xt a
nd p
rogr
am
1
2
3
4
+
![Page 75: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/75.jpg)
Beliefs and Practice
1 2 3 4 What you value
You
r te
xt a
nd p
rogr
am
1
2
3
4 +
+
+
+
![Page 76: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/76.jpg)
Beliefs and Practice
1 2 3 4 What you value(e.g. inquiry, content)
You
r te
xt a
nd p
rogr
am
1
2
3
4
o o
o o
![Page 77: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/77.jpg)
Beliefs and Practice
1 2 3 4 What we value
(e.g. telling and memorizing)
You
r te
xt a
nd p
rogr
am
1
2
3
4 o o
o
![Page 78: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/78.jpg)
Beliefs and Practice
1 2 3 4 Content, pedagogy, assessment factors
Our
tex
t an
d pr
ogra
m
1
2
3
4 +
+
+
+
![Page 79: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/79.jpg)
Summary
• Return tests promptly• Rubrics• Understanding by Design
– Enduring undestandings, evidence, instruction• Testing for undestanding
– Know what each student knows at the end of every class• 7E learning model• Assessment triangle: Cognition, Observation,
Interpretation• Don’t revert to non-productive behavior• Define science content to include inquiry and process
– Facts in meaningful context
![Page 80: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/80.jpg)
Assessment in the movies
• Do we recognize this quality of assessment?
![Page 81: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/81.jpg)
In Conclusion
• The best way in which to deal with NCLB is to improve instruction in the classroom.
• This can be done with improvements in formative assessment– Formative assessment has been shown to be effective at
improving student achievement.– Help teachers learn “Testing for understanding” techniques.
• Provide teachers with the confidence and support to continue to improve their teaching and resist the temptation to revert to poor pedagogy and worksheets in attempts to deal with NCLB.
• Give them permission to be good teachers.
![Page 82: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/82.jpg)
Common Advice
• ….the wisdom to know what I can change
• My husband handles the important issues
![Page 83: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/83.jpg)
“We must be the change we want to see in the world.”
- Gandhi
![Page 84: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/84.jpg)
![Page 85: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/85.jpg)
![Page 86: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/86.jpg)
Get inside a student’s head.
Cognitive empathy for the workings of the learners’ minds -- an ability to put themselves in the learners’ shoes
![Page 87: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/87.jpg)
Unscrambling Confusion
Determining what students don’t understand implies that teachers have ways of unscrambling confusions that identify the specific point(s) of misunderstanding and deal with them.
What means do we have for unscrambling confusions?
![Page 88: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/88.jpg)
Unscrambling Confusions
• None
![Page 89: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/89.jpg)
Unscrambling Confusions
• None
• Re-explains
![Page 90: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/90.jpg)
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint questions
![Page 91: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/91.jpg)
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint questions
• Perseveres and returns
![Page 92: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/92.jpg)
Unscrambling Confusions
• None
• Re-explains
• Isolates point of confusion with pinpoint questions
• Perseveres and returns
• Has student explain own current thinking
![Page 93: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/93.jpg)
Three pillars of assessment
• a model of how students represent knowledge and develop competence in the subject domain
• tasks or situations that allow one to observe students’ performance,
• an interpretation method for drawing inferences from the performance evidence thus obtained.
![Page 94: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/94.jpg)
Improvements
Rubrics•Clearly defined grading schema – matrix•B++++ and A-----•student and teacher comparisons: A,A or C,C or A,C or C,A
•all require very different discussions
Eliciting prior understandings•Fish is fish
![Page 95: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/95.jpg)
![Page 96: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/96.jpg)
![Page 97: How do we know what they know? FASS Meeting Orlando, FL April 5, 2004 Arthur Eisenkraft (eisenkraft@att.net)](https://reader035.vdocument.in/reader035/viewer/2022062516/56649e175503460f94b03623/html5/thumbnails/97.jpg)
Standards State exams (good instruction) (good preparation)