Assessing Computational Thinking

Eric SnowCenter for Technology in LearningSRI International

Sangeeta BhatnagarEvergreen Valley High School

Overview

• What is computational thinking (CT)?• Key CS/CT assessment efforts to date. • Current CT Assessment Work at SRI

International:• Planning for the Assessment of Computational

Thinking (PACT 1)• ECS-Standards Alignment

• Principled Assessment of Computational Thinking (PACT 2)

What is Computational Thinking?

• Popularized by Jeanette Wing in ACM Viewpoints Article (2006)

• “Universally applicable attitude and skill set everyone, not just computer scientists, would be eager to learn and use.”

• Characteristics of CT:• Conceptualizing, not (just) programming• A way that humans, not computers, think• Ideas, not (just) artifacts• Draws on existing conceptions of mathematical, engineering

(systems), and inquiry-based thinking• Not just for computer science/scientists

Key CS/CT Assessment Efforts to Date• CS Principles Computational Thinking Practices (College

Board, 2011)• Connecting computing• Developing computational artifacts• Abstracting• Analyzing problems and artifacts• Communicating• Working effectively in teams• Aligned with new CS AP principles course

• CS Concept Inventory (Herman, Loui, & Zilles, 2010)• FCS1 Assessment (Elliott Tew, 2010)• SUNY CS1-2 Exam (Decker, 2007)

Planning for the Assessment of Computational Thinking (PACT 1)

NSF-CE21 Planning Grant (9/2011 – 6/2012)Principal Investigators: Eric Snow, Marie Bienkowski

Project GoalsAdvance field of assessment in computer science and computational thinking.

Leverage interdisciplinary expertise to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum.

Planning for the Assessment of Computational Thinking (PACT 1)

Project Plan

December 2011 Core expert group meets to begin creating an assessment framework and design templates for computational thinking (CT) aligned with the Exploring Computer Science curriculum

January 2012Debrief core expert group meeting and circulate draft framework and design templates for internal review. Develop project dissemination web site. Begin planning for next stage of CT assessment design and development work

February – September 2012 Release project dissemination website, draft standards alignment document, invite CE21 community feedback on assessment framework, design templates and standards alignment work. Plan for next stage of CT assessment design and development work

Focal KSAs form the core of design templates that specify what we want to measure, the behaviors that will count as evidence of what we want to measure, and the types of tasks we can use to elicit the desired behaviors.

Planning for the Assessment of Computational Thinking (PACT 1)

ECS - CS/CT Standards Alignment• Objectives• Gain a better understanding of how ECS CT practices and

learning objectives were related to relevant national and state standards (e.g., CSTA, Common Core).

• Provide value to ECS team, especially teachers and others who had to convince stakeholders that ECS helps students learn valuable knowledge and skills

• Worked with local ECS teacher to develop draft document that shows the alignment between ECS Unit learning objectives and CT practices, and CSTA, ISTE, Common Core, Next Generation Science/Engineering, and CA and IL state learning and CTE standards.

• Currently under review/revision

ECS - CS/CT Standards Alignment

Planning for the Assessment of Computational Thinking (PACT 1)

Web Site URL: http://pact.sri.com

Please note:• The site is still under construction.• PACT materials will continue to be posted over coming

months. Please email me if you would like to get an update on when materials are posted.

• Feel free to send comments and feedback.

Principled Assessment of Computational Thinking (PACT 2)

NSF-CE21 Special Project (9/2012 – 8/2014)Principal Investigators: Eric Snow, Marie Bienkowski*proposal currently “Recommended” for funding

Project GoalsAdvance field of assessment in computer science and computational thinking.

Leverage materials and partnerships developed under PACT 1 to design, develop and validate assessments of computational thinking for the ECS curriculum and, eventually for the larger CT community.

Principled Assessment of Computational Thinking (PACT 2)

Year 1• Extend and finalize domain analysis/modeling started

in PACT 1; focus on ECS units 1-4, possibly unit 5 (“big data”).

• Develop pilot assessment tasks for unit and summative assessments

• Expert panel review of pilot tasks• Pilot summative assessment (include subset of unit

tasks), including think-aloud interviews• Data analysis and assessment task revisions• Advisory panel

Principled Assessment of Computational Thinking (PACT 2)

Year 2• Expert panel review of summative and unit assessment

tasks• Field test summative assessment, pilot unit

assessments, including think-aloud interviews• Data analysis and final assessment task revisions• Advisory panel

Methodological approach focused on assessment design/development, and on collecting validity evidence based on test content, internal structure, response processes, and relations to other variables.

Computational Thinking Resources

• National Research Council. (2004). Computer science: Reflections on the field. Washington, DC: The National Academies Press.

• National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the scope and nature of computational thinking. Washington, DC: National Academy Press.

• National Research Council Committee on the Workshops on Computational Thinking. (2010). Report of a workshop on the pedagogical aspects of computational thinking. Washington, DC: National Academy Press.

• CS Standards from International Society for Technology in Education (ISTE) and CS Teacher’s Association (CSTA) (2011)

THANK YOU!