power of 7 eduphoria staar · 2018. 9. 1. · norms honor all start times use electronic devices to...

© 2014 Region One Education Service CenterDivision of Instructional Support

Curriculum, Instruction & Assessment

Power of 7 – Eduphoria STAARSeptember 16th, 2016

Office of CIA- Learning Resources Integration

Norms

Honor all start times

Use electronic devices to enhance your

learning experiences

Those that do, learn, so participate actively

Share best practices and Network

Incorporate what you have learned into your

own best practices

Agenda

• Triangle of Data Driven Instruction

– Percent Score Data

– Reporting Category Data

– Student Expectation Data

– Factor Analysis Data

• Degree of Mastery Patterns

• Implications for Curriculum and Instruction

• Trend Reports

Objectives

• Access Eduphoria Reports for instructional purposes

• Engage in the Degree of Mastery process for student expectations

• Analyze the implications for curriculum and instruction

• View trend reports as an ongoing process to monitor student progress

Performance/Accountability

https://youtu.be/3nbjhpcZ9_g

https://youtu.be/3nbjhpcZ9_g

The Real Story – Behind the Scenes

https://youtu.be/LEIxpcYukqc

https://youtu.be/LEIxpcYukqc

Ice Breaker

Data Analysis is like…….

Data Analysis

RootProblem

CorrectiveInstruction

Implementation

CalibrationTool

Systemic Analysis and Instructional Practices for Educators

Percent Score Data

Report Category Data

Student Expectation

Data

Root Cause

Percent Score DataReports: Percent Score View &

Percent Score by Students View

Reporting Category DataReports: Reporting Category View &

Reporting Category Student View

How do you define

?

Reflection activityThink-Write-Pair-Share

What is rigor?

Individually: write down some characteristics of rigor that will help to define it

Pair up with a person next to you to compare notes

Share out with large group.

© 2014Division of Instructional Support

Curriculum, Instruction & Assessment16(TEA, 2010, HB3 Sec1 Chapter 2 p. I-28)


Curriculum, Instruction & Assessment17

RIGOR

FOCUS

Content

Complexity

Learning Standards

TEKS

CLARITY DEPTH

Cognitive

Complexity

Assessments

DOK

Performance Standards

TEKS/STAAR

What is Rigor?

Performance Level Descriptions



What is Rigor?

Content Complexity

Cognitive Complexity

Specificity PLDs

Student Expectations

Content Complexity


Specificity DOK


Instruction Assessment



Content Complexity Cognitive Complexity

Specificity Blooms Taxonomy


Curriculum

What is Rigor?



STAAR Performance Labels

• Advanced Academic PerformanceLevel III

• Satisfactory Academic PerformanceLevel II

• Unsatisfactory Academic PerformanceLevel I

Cut Score

Cut Score



Purpose of the PLDs• Provide a snapshot of student’s academic

characteristics based on assessment performance

• Describe a student’s knowledge and skills by levels of performance:• Level III- Advanced• Level II - Satisfactory • Level I - Unsatisfactory

• Translate into grade or course specific descriptions of student achievement



Purpose of the PLDs

Level III

Level II

Level I



Performance Level Descriptors (PLD): Progressions of Learning

Performance Level

Descriptors

Curriculum

Instruction

Assessment



Reflection Question

• What are the implications of Level I, Level II, Level III for Instruction?



Thinking about data: Digging Deeper



Student Expectation Data

Reports: Student Expectation View &Student Expectation by Student View

Starting the Road Map

What is the rigor of the standards?

What do students need to know and be able do?

Content Complexity Cognitive Complexity

Specificity Blooms Taxonomy


Curriculum

What is Rigor?



What is Rigor?

Content Complexity


Specificity PLDs


Content Complexity


Specificity DOK


Instruction Assessment



STAAR is assessing content and skills at a greater depth

and higher level of cognitive complexity.

(TEA, 2010, p. I-4)



Cognitive Complexity and

The Revised Bloom’s Taxonomy

A New Framework



Revised Bloom’s Taxonomy

Knowledge Dimension

knowledge that shares a consensus of acceptance within the

discipline

there are four types of knowledge: factual.

conceptual, procedural, and metacognitive

Cognitive Process Dimension

verbs which reflect the underlying thinking

processes in order of increasing complexity

there are six major cognitive process

categories



OLD

Evaluation

Synthesis

Analysis

Application

Comprehension

Knowledge

NEW

Create

Evaluate

Analyze

Apply

Understand

Remember

N

O

U

N

S

V

E

R

B

S

Adapted form Anderson & Krathwohl, 2001)

Revised Bloom’s Taxonomy



Let’s analyze the TEKS



Cognitive Expectation

• Cognitive

– The level at which students are

expected to perform in order to

adequately meet the standard.

– Determined by the verbs used in both

the knowledge and skills statements

and student expectation(s).



• TEKS Format – Essential Knowledge and Skills

• Grade 8

• (8) History. The student understands individuals,

issues, and events of the Civil War.

TEKS Breakdown: Cognitive



• TEKS Format – Student Expectation

• 8B - explain the causes of the Civil War, including sectionalism, states' rights, and slavery, and significant events of the Civil War, including the firing on Fort Sumter; the battles of Antietam, Gettysburg, and Vicksburg; the announcement of the Emancipation Proclamation; Lee's surrender at Appomattox Court House; and the assassination of Abraham Lincoln; and (R)

Cognitive Expectations



Content Expectations

• Content

– The content items for which

students must demonstrate

understanding at the appropriate

cognitive level in order to

adequately meet the standard.



• TEKS Format – Essential Knowledge and Skills

• Grade 8

• 8.8 History. The student understands individuals, issues,

and events of the Civil War.

TEKS Breakdown: Content



• TEKS Format – Student Expectations

• 8B - explain the causes of the Civil War, including sectionalism, states' rights, and slavery, and significant events of the Civil War, including the firing on Fort Sumter; the battles of Antietam, Gettysburg, and Vicksburg; the announcement of the Emancipation Proclamation; Lee's surrender at Appomattox Court House; and the assassination of Abraham Lincoln; and (R)

Content Expectations



Finding the Root Cause

Root CauseAnalysis



Let’s Find the Standard

Specificity of assessment

items



(4)(B) explain the roles played by significant individuals during the American Revolution, including Abigail Adams, John Adams, Wentworth Cheswell, Samuel Adams, Mercy Otis Warren, James Armistead, Benjamin Franklin, Bernardo de Gálvez, Crispus Attucks, King George III, Haym Salomon, Patrick Henry, Thomas Jefferson, the Marquis de Lafayette, Thomas Paine, and George Washington; (16 names)

Standard Specificity!



• SE: 1A - identify the major eras and events in U.S. history through 1877, including colonization, revolution, drafting of the Declaration of Independence, creation and ratification of the Constitution, religious revivals such as the Second Great Awakening, early republic, the Age of Jackson, westward expansion, reform movements, sectionalism, Civil War, and Reconstruction, and describe their causes and effects; (R)



SE: 19D - identify examples of responsible citizenship, including obeying rules and laws, staying informed on public issues, voting, and serving on juries; (S)DUAL: 29C - organize and interpret information from outlines, reports, databases, and visuals, including graphs, charts, timelines, and maps; (P)



Depth of Knowledge

What item levels are being assessed?



“A primary feature of STAAR’s test design is a focus on preparedness for success in subsequent grades or courses and, ultimately for college and career. This requires the test to emphasize depth rather than breadth in assessing student expectations.”-2010, House Bill 3: Transition to STAAR



Dr. Norman Webb & Depth of Knowledge

Norman Webb’s work has been applied to different content areas and used in test item development, as well as in alignment studies to determine the degree of alignment (match) between states’ standards and the tests used by states for accountability purposes.

2005, 2006 Cognitive complexity: Applying Webb DOK Levels of Bloom’s Taxonomy, Karin K. Hess, National Center for Assessment,

Dover, NH © Karin K. Hess permission to reproduce is given when authorship is fully cited [email protected]

mailto:[email protected]



What is Dr. Norman Webb’s Depth of Knowledge?

A common language for discussing rigor

A tool for alignment in educational assessment

A way to revise assessment practices

More informed dialogue for instruction

A part of reflective teaching



Content Complexity• What is content complexity in assessment?

Differentiates learning expectations and outcomes by considering the amount of prior knowledge, processing of concepts and skills, sophistication, number of parts, and application of content structure required to meet an expectation or to attain an outcome.

-Dr. Norman Webb-



Depth of Knowledge Tool

Root Cause

Analysis

Root Cause Analysis View

Data-Driven Instruction..what does it mean?

Data-driven instruction is the philosophy that schools

constantly focus on one simple question:

Are our students learning?

Quick Share out

How do you know when your students have mastered your content and when adjustments may be needed in your instructional timeline?

Data AnalysisLevel 1:

Wide Angle View of Student Performance

• How well did the class do as a whole?

• What are the strengths and weaknesses in the standards: where do we need to work the most?

• How did the class do on previously taught versus currently taught standards? Are they forgetting or improving on previous material?

Data Analysis

Level 2:

Close-up View of Student Performance

• Did students all choose same wrong answer? Why or why not?

• Break down each standard: Did students do similarly on every question within the standard or were some questions harder? Why?

• Compare similar standards: Do results in one influence the other?



Sample Questions

20%

11%

38%

31%

62% Chose the wrong answer!



Sample Question

22%

10%

50%

18%

50% Chose the wrong answer

• 2016 STAAR Item Analysis- Demographic w/Responses by Reporting Category

• Did students all choose same wrong answer? Why or why not?

• Did students do similarly on every question within the standard or were some questions harder? Why?

• Compare similar standards: Do results in one influence the other?

Zooming in..

Degree of Mastery• The DOM Analysis is to support teachers in

monitoring the effectiveness of instruction with the end result of greater teacher effectiveness in the classroom.

• The DOM Analysis is to provide Leadership with data to internalize--the need for quality curriculum, evaluation of learning, effective intervention, and effective instructional approaches.

Error Response Patterns

Percent of student group selecting individual distractor options.

• Type I• No Distractor options selected at a rate of 10% or more respectively

• Type II• Only One Distractor option selected at a rate of 10% or more respectively

• Type III • Only two distractor options selected at a rate of 10% or more respectively

• Type IV• Three or more distractor options selected at a rate of 10% or more respectively

Type I


• No distractor options selected at a rate of 10% or more respectively; no distractor options are plausible to the examinee group.

Instructional Condition Instructional Alignment Rationale

Highly Aligned InstructionInstruction is highly aligned to knowledge and skills being assessed and mastery learning is taking place.

Type II

Percent of student group selecting individual distractoroptions.

• Only one distractor option is highly plausible to the examinee group


Aligned InstructionInstruction is aligned to knowledge and skills being assessed.

Type III


• Only two distractor options are plausible to the examinee group


Minimally Aligned InstructionInstruction is minimally aligned to Knowledge and skills being assessed.

Type IV

Percent of student group selecting individual distractoroptions.

• All three distractor options are plausible to the examinee group.


No Aligned InstructionInstruction is not aligned to knowledge and skills

being assessed or item content not taught.



DOM TaxonomyDistractor Pattern Functioning Taxonomy

Distractor

PatternDistractor Rationale Performance Standard Instructional Condition

Instructional Alignment

Rationale

IV

All three distractor options

are plausible to the

examinee group.

Knowledge and Skills

Assessed

Ineffective or

Lack of Instruction

Instruction is not aligned to

knowledge and skills being

assessed

or item content not taught.

III

Only two distractor options

are plausible to the

examinee group.


AssessedModerate Instruction

Instruction is minimally aligned to

Knowledge and skills being

assessed.

II

Only one distractor option

is highly plausible to the

examinee group.


AssessedEffective Instruction

Instruction is aligned to knowledge

and skills being assessed.

I

No distractors options are

plausible to the examinee

group.


AssessedHighly Effective Instruction

Instruction is highly aligned to

knowledge and skills being

assessed

and mastery learning is taking

place.



Item Analysis Summary ReportSocial Studies

PERCENT RESPONDING

ITEM (A,F) (B,G) (C,H) (D,J)

1. 2 1 1 96*

2. 62* 20 6 12

3. 4 81* 2 13

4. 14 18 56* 12

5. 93* 3 2 2

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

50. 22 13 15 38*

51. 37 53* 4 1

52. 3 3 86* 4

Type I

Type III

Type II

Type IV

Type I

Type IV

Type II

Type I



Your Turn!

• Find the Degree of Mastery for your assessment items.

• What did you discover in the Degree of Mastery Analysis?



• Based on the information what are the implications for curriculum, instruction and assessment?

Data Analysis

RootProblem


Implementation

CalibrationTool



Percent Score Data

Report Category Data

Student Expectation

Data

Root Cause



Planning for High Quality Instruction

• What needs to be addressed all year long?

• Which content needs more time?



Effective Instructional Approaches



Data Analysis

RootProblem


Implementation

CalibrationTool



Eduphoria Assessment

• TEKSbank – Online Testing

• Eduphoria – Online Student Response System



Systemic Analysis and Instructional Practices for Educators



Recommended Trend Reports



As the teachers continue to analysis their local data by systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.

Progress monitoring will follow the same concept of the triangle which includes percent score, reporting category and expectation data. In this example, we are checking how the District is performing.



As the teachers continue to analysis their local data by

systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.

Progress monitoring will follow same concept of the triangle which includes percent score, reporting category and

expectation data. In this example, we are checking how our campus is performing.




systemically practicing the process of the factor analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.


expectation data. In this example, it will allow us to check how our teacher is performing.




systemically practicing the process of the root cause analysis, another key component will be progress monitoring. It’s imperative to monitor the progress of the district, campus, teacher, class, students and focus groups.


expectation data. In this example, we are checking how our students are performing.



Connect/Extend/Challenge

• How are the ideas/information presented connected to what you already knew?

• What ideas did you get that extended or broadened your thinking?

• What challenges remain from the ideas/information presented?

Sources

© 2014 Region One Education Service CenterDivision of Instructional Support

Curriculum, Instruction & Assessment

Greenstein, L. (2010). What teachers really need to know about formative assessment.Alexandria, VA: ASCD.

Ritchhart, R., Church, M., & Morrison, K. (2010). Making thinking visible. San Francisco, CA: Jossey-Bass.

TEA, Curriculum Division. (2007-2012). Texas essential knowledge and skills. Retrieved April 30, 2013, from Texas Education Agency: Online

TEA, Student Assessment Division. (2011-2012, June 8). STAAR assessed curriculum. Retrieved May 5, 2013, from Texas Education Agency: Online

TEA, Student Assessment Division. (2013, January 11). STAAR performance level descriptors. Retrieved January 12, 2013, from Texas Education Agency.

TEA, Student Assessment Division. (2013, January 11). STAAR Grade 8 Social Studies April 2013 Released. Retrieved August 5, 2013, from Texas Education Agency.

Walsh, J. A., & Sattes, B. D. (2011). Thinking through quality questioning. Thousand Oaks, CA: Corwin.

power of 7 eduphoria staar · 2018. 9. 1. · norms honor all start times use electronic devices to...

Documents