![Page 1: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/1.jpg)
MEASUREMENT AND MODELING OF HIGHER ORDER SKILLS IN STEM TALENT DEVELOPMENT USING THE CWRAChristopher Kolar
NCSSSMST/NAGC Annual Convention 2013, Indianapolis
![Page 2: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/2.jpg)
The challenge
Exceptional students
Exceptional claims of exceptional results
Exceptional Evidence
![Page 3: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/3.jpg)
Standards of Significant Learning (SSLs)
Assessment at IMSA
![Page 4: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/4.jpg)
Competence In Inquiry
Making accurate observations.
Evaluating evidence.
Challenging assumptions.
Making connections. Making sound
judgments. Communicating
powerfully.
![Page 5: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/5.jpg)
CWRA: Performance task and document
![Page 6: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/6.jpg)
In addition to an overall Performance Task score, subscores include:
Analytic Reasoning
Writing Effectiveness
Writing Mechanics
Problem Solving
![Page 7: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/7.jpg)
![Page 8: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/8.jpg)
MEASURING UP
Admissions data
![Page 9: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/9.jpg)
Admissions Benchmarking
Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc
![Page 10: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/10.jpg)
Admissions Benchmarking
CWRA performance task score significantly correlated with all admissions metrics.
Prior GPA SAT-M SAT-CR SAT-We SAT-Wmc
![Page 11: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/11.jpg)
MEASURING UP
Initial academic performance
![Page 12: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/12.jpg)
First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology)
Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+
First Year Grade Point Average
![Page 13: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/13.jpg)
First year grades in writing intensive courses: Literary Explorations, American Studies, and Scientific Inquiries (Biology): 23% of variance explained.
Prior GPA SAT-CR SAT-We SAT-Wmc CWRA+
First Year Grade Point Average
![Page 14: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/14.jpg)
MEASURING UP
English core assessment
![Page 15: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/15.jpg)
No reliability as outcome measures
Thesis Paragraphs Quote Usage
Analytic Reasoning
Writing Effectiveness
Writing Mechanics Problem Solving
IMSA CWRA+
![Page 16: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/16.jpg)
THEREFORE
What this means in practice
![Page 17: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/17.jpg)
Student Support
CWRA+ individual results as a diagnostic Further grade and retention modeling using this
data
![Page 18: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/18.jpg)
Assessment Development
Performance task academy School improvement plan Revision of core assessment
![Page 19: NAGC 2013 Measurement and Modeling of Higher Order Thinking Skills](https://reader035.vdocument.in/reader035/viewer/2022070317/5563be96d8b42ac70d8b5716/html5/thumbnails/19.jpg)
Value added
“Small” effect size for time by junior year. Senior exit testing next.