understanding by design stage 2: evidence think like an assessor
TRANSCRIPT
Understanding by DesignStage 2: Evidence
Think Like an Assessor
Backward Design: Stage 2
• Performance Tasks and Rubrics• Other Evidence• Self-Assessment
• To what extent do the assessments provide valid, reliable, and sufficient measures of the desired results?
Pre-Assessment
Reflection Questions
• What patterns do you notice?• What generalizations can you make about
your own classroom? Your grade-level or department? Your school? Your district?
• What may be an area for improvement?
Heidi Hayes Jacobs: What year are we preparing our students for?
http://goo.gl/udjmU
Video Viewing Guide
Think-Pair-Share
• What observations did you have while watching this talk?
• Paradigm Shift
• What will it take to get to the point of developing assessments as Jacobs suggests?
Assessment vs. Evidence
• What’s the difference?• Think-Pair-Share
Assessment vs. Evidence
• What’s the difference?• Think-Pair-Share
• Evidence is…– collected over time
instead of a single event– ongoing and frequent to
catch misunderstandings in the early stages
– Includes a wide range of different types of assessments
Three Basic Questions
1) What kinds of evidence do we need to find hallmarks of our goals, including that of understanding?
2) What specific characteristics in the student responses, products, or performances should we examine to determine the extent to which the desired results were achieved?
3) Does the proposed evidence enable us to infer a student’s knowledge or understanding?
Exercises to Support Deep Understanding
• Ensure that teachers consider the intent of the standards
• Facilitate conversations with other teachers to help reduce misconceptions
• How should each standard or group of standards be assessed?
http://www.google.com/imgres?q=collaboration&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=iWwNZpAHR8ZSyM:&imgrefurl=http://www.thevarguy.com/2011/04/13/visual-collaboration-the-next-var-opportunity-arrives/&docid=caP-BUJSvM5wOM&imgurl=http://www.thevarguy.com/wp-content/uploads/2011/04/collaboration.jpg&w=371&h=323&ei=IpgET9bZMYeXtwf82f20CA&zoom=1&iact=hc&vpx=331&vpy=322&dur=320&hovh=209&hovw=241&tx=129&ty=96&sig=114730900127101918374&page=1&tbnh=139&tbnw=143&start=0&ndsp=19&ved=1t:429,r:7,s:0
Collection of Evidence Assessment
Understanding Stage 2: Evidence
http://www.google.com/imgres?q=think+clipart&start=484&num=10&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=4IlRKZhxKjkC6M:&imgrefurl=http://fido-lew.blogspot.com/2010_08_01_archive.html&docid=APUIdeRwTl6jrM&imgurl=http://2.bp.blogspot.com/_bP5JHTTWjIM/THnkNQvZDhI/AAAAAAAAArA/rLbUpG06h4s/s1600/6a00d8341e626f53ef0115712117e1970c-800wi.jpg&w=800&h=551&ei=lLwJT9jEF82ltwe6ttneAQ&zoom=1&chk=sbg&iact=hc&vpx=822&vpy=203&dur=154&hovh=186&hovw=271&tx=145&ty=85&sig=114730900127101918374&sqi=2&page=23&tbnh=109&tbnw=158&ndsp=22&ved=1t:429,r:12,s:484
Changing the Paradigm about Quality Evidence and Assessment
http://pics.blameitonthevoices.com/s.php?f=102010&p=test.jpg
Brain Dump
• What are the characteristics of a quality assessment?
http://www.google.com/imgres?q=brain+dump&hl=en&client=firefox-a&hs=27E&sa=X&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&prmd=imvns&tbnid=uYAemdwwqbvTkM:&imgrefurl=http://knektus.wordpress.com/2011/03/06/not-alzheimers-just-overload/&docid=cuOLDoEoKCVR4M&imgurl=http://knektus.files.wordpress.com/2011/03/brain-dump.jpg&w=300&h=200&ei=H9kKT8fRFNCutweUs7TQBg&zoom=1&iact=rc&dur=524&sig=114730900127101918374&page=3&tbnh=129&tbnw=193&start=39&ndsp=19&ved=1t:429,r:3,s:39&tx=97&ty=28
Quality Assessments
• Tasks that require students to demonstrate their understanding of the Transfer and Meaning elements of Stage 1
Stage 2: Assessment and Evidence• To what extent do the assessments provide
valid, reliable and sufficient measures of the desired results?
– What will students do to show they understand?– What is the most appropriate assessment(s) method?
Reflect and Share
• What does this quote mean to you?
“Think like an assessor, not an activity designer.”
Thinking Like an Assessor Thinking Like a Activity Designer
What would be sufficient evidence of understanding?
What would be interesting and engaging activities on this topic?
What performance tasks must anchor the unit and focus the instructional work?
What resources and materials are available on this topic?
How will I be able to distinguish between those who really understand and those who don’t (though they may seem to)?
What will students be doing in and out of class? What assignments will be given?
Against what criteria will I distinguish work?
How will I give students a grade and justify it to parents?
What misunderstandings are likely? How will I check for those?
Did the activities work? Why or why not?
Review
• Why is it important to think like an assessor and not an activity designer?
Mach 2005
Six Facets of UnderstandingUbD Facet Description
Explanation Sophisticated explanations and theories
Interpretation Interpretations, narratives, and translations
Application Use knowledge in new situations and contexts
Perspective Critical and insightful points of view
Empathy Ability to get inside another person's feelings
Self-knowledge To know one's ignorance, prejudice, and understanding
Understanding Stage 2: Evidence
http://www.google.com/imgres?q=think+clipart&start=484&num=10&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=4IlRKZhxKjkC6M:&imgrefurl=http://fido-lew.blogspot.com/2010_08_01_archive.html&docid=APUIdeRwTl6jrM&imgurl=http://2.bp.blogspot.com/_bP5JHTTWjIM/THnkNQvZDhI/AAAAAAAAArA/rLbUpG06h4s/s1600/6a00d8341e626f53ef0115712117e1970c-800wi.jpg&w=800&h=551&ei=lLwJT9jEF82ltwe6ttneAQ&zoom=1&chk=sbg&iact=hc&vpx=822&vpy=203&dur=154&hovh=186&hovw=271&tx=145&ty=85&sig=114730900127101918374&sqi=2&page=23&tbnh=109&tbnw=158&ndsp=22&ved=1t:429,r:12,s:484
Facet 1: Explanation
Impact on Instruction• Students need to express
the “big picture” or “big idea” in their own words.”
Criteria• Accurate• Coherent • Justified• Systematic• Predictive
http://www.google.com/imgres?q=explain+clipart&hl=en&client=firefox-a&hs=9jf&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=j84zFgquNWOCFM:&imgrefurl=http://ticketbud.com/blog/category/organizing-people/&docid=dcEEMAAGyz27yM&imgurl=http://blogs.msdn.com/blogfiles/willy-peter_schaub/windowslivewriter/vstsrangersprojectsmigrationguidance_9a3c/clipart_of_27038_smjpg_2.jpg&w=480&h=480&ei=IfwJT7CFJNGgtwfN9PAQ&zoom=1&iact=rc&dur=311&sig=114730900127101918374&page=9&tbnh=138&tbnw=146&start=174&ndsp=21&ved=1t:429,r:20,s:174&tx=71&ty=86
Facet 2: Interpretation
Impact on Instruction• Students need to have
opportunities to make sense of stories, art, data, situations, etc.
Criteria• Meaningful• Insightful• Significant• Illustrative• Illuminating
http://www.google.com/imgres?q=student+thinking+clipart&hl=en&client=firefox-a&hs=sDL&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=vdMfbMhxAOZrKM:&imgrefurl=http://problemfinding.labanca.net/tag/situated-cognition/&docid=UvUkQjNt2b6dZM&imgurl=http://problemfinding.labanca.net/wp-content/uploads/2009/12/sbpuzzled-300x276.jpg&w=300&h=276&ei=Uf4JT4NAgp-3B7rZ3c8G&zoom=1&iact=rc&dur=628&sig=114730900127101918374&page=4&tbnh=141&tbnw=153&start=71&ndsp=22&ved=1t:429,r:3,s:71&tx=58&ty=81
Facet 3: Application
Impact on Instruction• Students must use their
knowledge and skills in new situations.
Criteria• Effective• Efficient• Fluent• Adaptive• Graceful
http://www.google.com/imgres?q=insightful+clipart&hl=en&client=firefox-a&hs=t8K&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=GtoVA3MKcpsPSM:&imgrefurl=http://bpmfundamentals.wordpress.com/&docid=MSI7nCE5i6ehIM&imgurl=http://blogs.msdn.com/blogfiles/willy-peter_schaub/WindowsLiveWriter/WhatisALMandaretherelevantpublicationsto_F2DE/CLIPART_OF_26862_SMJPG_2.jpg&w=600&h=480&ei=HP0JT8mPCMiWtwfE65XRBg&zoom=1&iact=hc&vpx=176&vpy=145&dur=629&hovh=201&hovw=251&tx=135&ty=98&sig=114730900127101918374&page=1&tbnh=132&tbnw=163&start=0&ndsp=22&ved=1t:429,r:0,s:0
Facet 4: Perspective
Impact on Instruction• Students need
opportunities to see things from different points-of-view, articulate the opposing side of an issue, or take a critical stance.
Criteria• Credible• Revealing• Insightful• Plausible• Tactful
http://www.google.com/imgres?q=global+perspective+clipart&hl=en&client=firefox-a&hs=1IL&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=AupLc76xhZtzsM:&imgrefurl=http://blogs.msdn.com/b/willy-peter_schaub/archive/2010/02/23/mvp-global-summit-from-a-rangers-perspective.aspx&docid=vOHJRMFvOtSPHM&imgurl=http://blogs.msdn.com/blogfiles/willy-peter_schaub/WindowsLiveWriter/MVPGlobalSummitfromaRangersperspective_D4A9/CLIPART_OF_15186_SM_2.jpg&w=480&h=480&ei=kP8JT9XkI8yltweprM3PBg&zoom=1&iact=hc&vpx=399&vpy=151&dur=465&hovh=152&hovw=148&tx=159&ty=102&sig=114730900127101918374&page=1&tbnh=152&tbnw=148&start=0&ndsp=19&ved=1t:429,r:1,s:0
Facet 5: Empathy
Impact on Instruction• Students demonstrate
empathy when they can appreciate people who think and act differently from themselves.
Criteria• Sensitive• Open• Receptive• Perceptive• Tactful
http:// http://www.google.com/imgres?q=personal+knowledge+clip+art&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=adimAXTmnZonpM:&imgrefurl=http://blogs.msdn.com/b/willy-peter_schaub/archive/2011/03/31/ahe-and-ate-in-the-final-phase-of-review-a-huge-thank-you-to-the-phenomenal-teams.aspx&docid=KDsB17A4Q_7bKM&imgurl=http://public.blu.livefilestore.com/y1p-97mb5WvgK9wNdFgRMS0Ez0wp2QGzCT8r2KMafvfXkwnKhQYY4vR12TlWOnOzlxnl6fhlFwv_wABL1xfcNHD9Q/CLIPART_OF_29895_SMJPG.jpg%253Fpsid%253D1&w=1500&h=1125&ei=1QEKT7iCBsu3tweS-PnPBg&zoom=1&iact=hc&vpx=608&vpy=345&dur=206&hovh=194&hovw=259&tx=142&ty=97&sig=114730900127101918374&page=1&tbnh=123&tbnw=120&start=0&ndsp=21&ved=1t:429,r:10,s:0
Facet 6: Self-Knowledge
Impact on Instruction• Students self-assess their
past as well as their future.
Criteria• Self-aware• Meta-cognitive• Self-adjusting• Reflective• Wise
http://www.google.com/imgres?q=aura+clipart&hl=en&client=firefox-a&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&tbnid=JJhtqyOrALRrLM:&imgrefurl=http://cute-pictures.blogspot.com/2011/03/spiritual-illustrations-meditation.html&docid=DF-UxyNilMwuNM&imgurl=https://lh6.googleusercontent.com/-jCx5hgjtZ4s/TXemchAIKiI/AAAAAAAAG-s/TVCBuSO39iU/21%252Blearn_to_see_your_aura_%252B%25252528www.cute-pictures.blogspot.com%25252529.jpg&w=300&h=343&ei=5gIKT7bLIM2ItwfDsrCSBQ&zoom=1&iact=hc&vpx=696&vpy=311&dur=2926&hovh=240&hovw=210&tx=116&ty=153&sig=114730900127101918374&page=9&tbnh=134&tbnw=117&start=173&ndsp=22&ved=1t:429,r:10,s:173
Electricity Activity
• Use the six facets of understanding and the student tasks to complete the hexagon graphic organizer.
http://www.google.com/imgres?q=electricity&hl=en&client=firefox-a&hs=IN1&sa=X&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&prmd=imvnsb&tbnid=rESrye1eU8O4EM:&imgrefurl=http://sowrock.com/electricity/&docid=yq2H__lbUPzl9M&imgurl=http://sowrock.com/electricity/electricity3.gif&w=550&h=500&ei=nwoKT4z6CInAtgeC3KDQBg&zoom=1&iact=rc&dur=310&sig=114730900127101918374&page=1&tbnh=140&tbnw=154&start=0&ndsp=20&ved=1t:429,r:2,s:0&tx=61&ty=82
Matching Activity• Explain to the class how a battery causes a
light bulb to glow.• Give a pre-test and a post-test to assess
common misconceptions and have students reflect on their deepening understanding.
• Interpret a schematic diagram and predict the outcome.
• Design an electrical circuit to accomplish a specific task.
• Describe an electron’s experience as it passes through a simple current.
• Why does the United States use AC instead of DC current? What are the strengths and limitations of each type?
• Explanation • Application• Perspective• Self-Knowledge• Empathy• Interpretation
Questions?
A Continuum of Assessments
• Informal Checks for Understanding
• Dialogue and Conversations
• Tests and Quizzes• Academic Prompts• Performance Tasks
• Define: What does it look like?
• What are some examples?
• Strengths• Limitations
Continuum of Assessments
Informal Checks• Ongoing assessments used as part of the
instructional process.
Observations and Dialogue
Tests and Quizzes
Academic Prompts
Performance Tasks…• Are not secure: The task, evaluative criteria,
and performance standards are known in advance and guide student work.
• Involve a real or simulated setting and the kind of constraints, background noise, incentives, and opportunities an adult would find in a similar situation.
Why Performance Tasks?
http://www.google.com/imgres?q=performance+task&hl=en&client=firefox-a&hs=tnE&sa=X&rls=org.mozilla:en-US:official&biw=1366&bih=664&tbm=isch&prmd=imvns&tbnid=YrUE8DqBoM79CM:&imgrefurl=http://wiki.bssd.org/index.php/August_6-10,_2007_UBD/DI_Teacher_Workshop&docid=qk9PRH01Ybz4gM&imgurl=http://wiki.bssd.org/images_up/f/f4/Learning_pyramid.png&w=684&h=491&ei=7p0JT-jBG4_rtgeG-NnRBg&zoom=1&iact=hc&vpx=1060&vpy=152&dur=27&hovh=190&hovw=265&tx=187&ty=68&sig=114730900127101918374&page=1&tbnh=136&tbnw=198&start=0&ndsp=20&ved=1t:429,r:5,s:0
Performance Tasks
Developing Performance Tasks
45
Construct a Performance Task Scenario Using GRASPSGoal:• Your task is ____________________________________________________• Your goal is to __________________________________________________• The problem or challenge is _________________________________________• The obstacles to overcome are _______________________________________
Role:• You are________________________________________________________• You have been asked to ____________________________________________• Your job is _____________________________________________________
Audience:• Your clients are___________________________________________________• The target audience is _______ _______________________________________• You need to convince_______________________________________________
Situation:• The context you find your self in is ______________________________________• The challenge involves dealing with ______________________________________
Product Performance and Purpose:• You will create a ___________________________ in order to show__________________________• You need to develop _________________________so that _________________________________
Standards and Criteria for Success:• Your performance needs to ______________________________________• Your work will be judged by ______________________________________• Your product must meet the following standards: ___________________________________
• A successful result will __________________________________________
McTighe and Wiggins
Conversation
• How can using the GRASPS and Six Facets guide the development of quality assessments?
47
According to Grant Wiggins…
• “What is to be assessed must be clear and explicit to all students.”
• Rubrics must accompany all major assignments and assessments.
• Students should be able to chart their progress and self-assess throughout the duration of their work.
Evaluation Criteria
• Another Key to Quality Assessment• Quality Rubrics• Beginning with the End in Mind is Crucial
49
A Rubric…
• Defines levels of quality• Communicates standards• Provides expectations for assessment tasks• Is NOT a checklist (yes or no answers)• Includes dimensions (criteria), indicators, and
a rating scale.
50
Advantages of Using a Rubric:
• Lowers students’ anxiety• Provides specific feedback about the quality of their
work• Provides a way to communicate expectations and
progress• Ensures all student work is judged by the same
standard• Leads students toward quality work.
51
Parts of a Rubric:
• Dimensions: Sometimes referred to as criteria - encompass the knowledge, skills, and understanding to be assessed
• Indicators: Specify the evidence used to judge the degree to which the dimension is mastered
• Rating Scales: Discriminate among the various levels of performance
52
Basic Rubric Template: Scale
Criteria
Indicator Indicator Indicator Indicator
Indicator Indicator Indicator Indicator
Indicator Indicator Indicator Indicator
53
Template for Holistic Rubrics:Score Description
5 Demonstrates complete understanding of the problem. All requirements of task are included in response.
4 Demonstrates considerable understanding of the problem. All requirements of task are included.
3 Demonstrates partial understanding of the problem. Most requirements of task are included.
2 Demonstrates little understanding of the problem. Many requirements of task are missing.
1 Demonstrates no understanding of the problem
0 No response/task not attempted
54
Template for Analytic Rubrics:Beginning
1Developing
2Accomplishe
d3
Exemplary4
Score
Criteria #1
Description reflecting beginning level of performance
Description reflecting movement toward mastery level of performance
Description reflecting achievement of mastery level of performance
Description reflecting highest level of performance
Criteria #2
Description reflecting beginning level of performance
Description reflecting movement toward mastery level of performance
Description reflecting achievement of mastery level of performance
Description reflecting highest level of performance
Criteria #3
Description reflecting beginning level of performance
Description reflecting movement toward mastery level of performance
Description reflecting achievement of mastery level of performance
Description reflecting highest level of performance
Criteria #4
Description reflecting beginning level of performance
Description reflecting movement toward mastery level of performance
Description reflecting achievement of mastery level of performance
Description reflecting highest level of performance
55
Ineffective Rubrics
• Wordy• Checklists• Judge each work against other items of work• Heavy on Criteria not inherent in the
standards
56
Good Rubrics:• Are tools • Show level of quality of a performance or task• Communicate standards clearly and specifically• Are provided to students to set expectations• Are consistent and reliable• Use content that matches standards and
instructional emphasis
57
Rubric Resources
• www.rubrics4educators.com
• http://school.discovery.com/schrockguide/assess.html
• http://rubistar.4teachers.org/
• http://www.teach-nology.com/web_tools/rubrics/
Implementing Stage 2• What understandings or goals
will be assessed through this task?
• Through what authentic performance task will students demonstrate understanding?
• What student products and performances will provide evidence of desired understandings?
• By what criteria will student products and performances be evaluated?
http://www.google.com/imgres?hl=en&client=firefox-a&hs=WlV&sa=X&rls=org.mozilla:en-US:official&biw=1346&bih=620&tbm=isch&prmd=imvnsl&tbnid=5tQoDDRo0MgreM:&imgrefurl=http://www.techfume.com/registry-cleaner-reviews/&docid=pTg3JnTRo5pEVM&imgurl=http://www.techfume.com/wp-content/uploads/2011/03/review.jpg&w=400&h=300&ei=zn0mT7P7KdTMtgeGr4CiCw&zoom=1&iact=hc&vpx=314&vpy=177&dur=1196&hovh=194&hovw=259&tx=136&ty=83&sig=114730900127101918374&page=1&tbnh=117&tbnw=152&start=0&ndsp=21&ved=1t:429,r:1,s:0
Stage 2 Feedback Tool
Planning Time
http://www.google.com/imgres?q=stop&hl=en&gbv=2&biw=1366&bih=664&tbm=isch&tbnid=Vgyx3BKJV1ajDM:&imgrefurl=http://sociablesusan.blogspot.com/2011/01/stop-stopping-yourself.html&docid=6AVY8XnUSNWA9M&imgurl=http://4.bp.blogspot.com/_-mQqQAcrZaA/TTOZWTLFYHI/AAAAAAAACUo/AanVLLJ6XYA/s1600/Stop_hand.png&w=512&h=512&ei=wDcLT5bkEIultwft4PD4CQ&zoom=1&iact=rc&dur=405&sig=114730900127101918374&page=1&tbnh=132&tbnw=132&start=0&ndsp=22&ved=1t:429,r:1,s:0&tx=71&ty=62