outcomes assessment 1 classroom assessment joseph a. shaeiwitz west virginia university...

Outcomes Assessment 1

Classroom Assessment

Joseph A. ShaeiwitzWest Virginia

[email protected]

u

Daina M. BriedisMichigan State

[email protected]

Outline Rationale: intro to assessment Types of assessment Classroom goals & objectives Classroom assessment methods

Using technology Linking classroom assessment to

program assessment Review and Recommendations

Assessment

Assessment feedback on what, how much, and how well students are learning.

Goal of assessment

Highest possible

quality of student learning

Assessment One of the “Two Guiding Principles

of Effective Teaching” (Felder & Brent): Practice and FEEDBACK (assessment)

encompassing cognitive, affective and psychomotor domains

learning competencies reasoning attitudes work habits

oneclass

oneclass

education process

one course

one course

enteringcollege

graduate alumnus

Feedback Model

Assessment

Learner

InstructorLearner

Instructor

Learner-Centered Instructor-Directed

Mutually Beneficial

Assessment

Learner

InstructorLearner

Instructor

Assumptions Student learning ∞ Teaching

Effectiveness Course objectives and goals need to be

made clear Learning Policies

To improve learning, students need feedback

To improve teaching, instructors need feedback

Assessment

Instructor teaches based on learning objectives (or goals) Students are supposed to learn Assessment helps minimize gap

Make it manageable May be part of program assessment

not sufficient for program assessment program assessment in 2nd session

Assessment – what it is NOT

NOT active learning although some overlap is possible

(and desirable) NOT for the purpose of grading

students Almost never graded Almost always anonymous

NO special training needed

Example

Be thinking of a “focus” course for the next time you will be teaching

Throughout this session, think of Classroom Assessment Techniques (CATs) that you can apply in this course

Outline Rationale: intro to assessment Types of assessment Classroom goals & objectives Classroom assessment methods

Using technology Linking classroom assessment to


Types of Assessment

Formal and Informal Formative Summative Diagnostic

Types of Assessment

Formal and Informal Formal: quizzes, term papers, lab

reports, homework, examinations Informal: questions in class, body

language, facial expressions, CATs (classroom assessment techniques)

Repertoire of successful methods

Formative vs. Summative

A Z

A M Z

A D L M R Z

~ Ongoing ~

Types of Assessment Formative

for improving a process (learning here) occurs during process often informal in detail in this session

Summative final, summative judgment about

effectiveness of process achievement of objectives and outcomes in detail in 2nd session

Types of Assessment

Diagnostic Assessment of metacognitive skills Identification of misconceptions Generally more difficult to analyze

(and administer)

Outline Introduction to assessment Types of assessment Classroom goals Classroom assessment methods Using technology Linking classroom assessment to


Classroom goals To assess and improve instruction,

first clarify what students are to learn!

Course objectives & goals need to be made clear Learning-oriented

Teaching Goals Inventory Teaching-oriented

Use these to direct classroom assessment

Teaching Goals Self-assessment of instructional goals Develop, Improve, Learn e.g.,

Develop ability to synthesize and integrate information and ideas

Develop problem-solving skills Improve writing skills Learn computing skills relevant to

problem solving

Course Learning Objectives (Goals)

Student-based What students will know and/or be

able to do at the conclusion of the course

Set expectations for both students and instructor

Ties to pre-requisites and post-requisites in curriculum

Learning Objectives

At the conclusion of this workshop, the participants will be able to:

define learning objectives, write and classify them in terms of Bloom’s Taxonomy levels, and list pedagogical and curricular benefits of writing them for courses.

generate a set of handouts for the first day of a course (course syllabus, learning objectives, statement of policies and procedures) that provide the students with a full understanding of the course structure and ground rules.

devise preliminary course activities that capture interest and motivate learning. identify characteristics of effective learners and techniques for obtaining active

participation from most or all students in attendance. define inductive teaching and learning and give examples of inductive teaching

methods and identify benefits of this instructional approach. define and give examples of the higher-level (analytical, creative, critical) thinking skills

of Bloom’s Taxonomy, identify instructional conditions that induce students to develop and exercise these skills, and formulate exercises and problems that provide practice in these skills.

design tests that are both challenging and fair and a grading system that provides positive motivation for learning without lowering standards.

deal effectively with a variety of common classroom management and other student-related problems.

identify problems associated with the teaching profession having to do with time management, starting an maintaining research programs, and assessing and improving teaching, and formulate plans to overcome these problems.

Example

Identify three teaching goals for your focus course

Write three learning objectives for this course

Preparing for a Successful Start

Plan Implement

Respond


1. Start small – one course to begin in which you are confident

2. Reserve 5-10 minutes of class time3. Let students know ahead of time (why,

historical successes, anonymity)4. Easy method5. Review/analyze asap

a. 1-2 min per responseb. Speed of analysis increases (divide “o.k.,”

“sort of o.k.,” “not o.k.”6. Respond!!!


1. Don’t make it a burden2. Don’t use methods inappropriate at

your institution3. Research projects

Teaching as research

Classroom Assessment Techniques (CATs)

Select the “right” technique Suits context of course Suits class and instructor personality Effective Efficient Integrates into course flow

Exhaustive CAT Catalog

Angelo, T.A., and K. P. Cross, “Classroom Assessment Techniques: A Handbook for College Teachers,” 2nd edition, Jossey-Bass Publishers, 1993.

Classroom Assessment Techniques (CATs) Minute paper and related exercises Pre- and post-tests Surveys Application cards Learning logs/journals Portfolios Coaching students working on examples Test is to make up test Open outlines Students defend test to instructor

Time & Energy Requirements

Prep: L, M, HResponse: L, M, HAnalysis: L, M, H

Minute Paper and Related Exercises “Minute paper” or “Half-Sheet

Response” USED MOST OFTEN! At end of lecture, ask one or two

questions related to what students should have learned

not graded/no names instructor learns about success or failure

of lecture/lab/video only useful if provide necessary

review/feedback next lecture

Prep: LResponse: LAnalysis: L

Minute Paper and Related Exercises

Questions: “What was the most important thing you

learned today?” “What important question was unanswered

for you today? Determines if focus of session was understood Evaluates ability to synthesize, integrate, pay

attention, listen, learn concepts and theories Respond: summarize next time; review

learning, answer unanswered questions

Minute Paper and Related Exercises Clearest vs. Muddiest point

At end of lecture, ask what is clearest point learned/what is muddiest point

List key knowledge skills learned today List a few words that define what xxx

means to you (xxx based on lecture content)

Summarize what you have learned in a few sentences so that you can explain to a friend


Example

What is the clearest point in this workshop thus far?

What is the muddiest point?

Pro & Con Grid

+/Δ done two slides ago Provides overview of class

analysis of advantages/disadvantages Examples:

two proposed designs ethical issues others?

Prep: LResponse: LAnalysis: L-M

Pro & Con Grid

Evaluates ability to apply analytical skills, capacity for ethical choices, use of judgment, capacity to think for oneself

Prep: LResponse: LAnalysis: L-M

Pre- and Post-tests Diagnostic Pre-test given first day of class

provides information on student skills/background

learn what students know about subject adjust teaching/syllabus based on

results Post-test given last day of class

able to assess what students have learned

may be almost identical to pre-test

Prep: MResponse: MAnalysis: M-H

Pre-test

Variation: “readiness” test Based on assigned reading or

homework E.g., Flashcard method

Done in groups with a recorder

Surveys Class management tools

Study habits Class opinion surveys Information seeking

Web-based surveys CATME (Comprehensive

Assessment of Team Member Effectiveness) www.catme.org

Available “Institutional Research”

Prep: LResponse: L-MAnalysis: L-H

Application Cards Students write down one application of

a theory or principle learned in class Connect newly learned concepts with

prior knowledge May provide new material for

instructor! Assesses ability to apply principles,

think creatively, learn concepts/ theories, think for oneself

Prep: LResponse: L-MAnalysis: L-M

Application Cards Develop one application card

question for your focus course. Example:

Provide an example of how statistical significance testing could be applied to your unit operations lab data.

Prep: LResponse: L-MAnalysis: L-M

Application Cards

This method may also be used in pairs and groups to expand generation and sharing of ideas

Diagnostic Learning Logs Journal of homework, assignments,

exams Student records two lists:

Main points that were understood Points that were unclear

Students reflect upon own learning Diagnose strengths & weaknesses

Prep: MResponse: HAnalysis: H

Diagnostic Learning Logs

Opportunity for reflective learning and self-assessment

Works best in course with frequent assignments

Be sure to focus on positives as well Develops analytical skills, problem-

solving skills, study skills, responsibility for one’s own behavior

Portfolios

Reflective journal Often a collection of student’s best

work May be posted electronically Portfolio for grad school and

employers Some institutions use for program

assessmentPrep: HResponse: HAnalysis: H

Classroom Assessment Methods Minute paper and related exercises Pre- and post-tests Standard approaches Surveys Learning logs/journals Portfolios Coaching students working on examples Test is to make up test Open outlines Students defend test to instructor

Coaching Students Working on Examples

Example problems are good Better if students work them (or at least

begin to work them) rather than just see them

best way to learn to solve problems is to solve problems, not to watch others solve problems

Pair-and-share method Advantages: walk around room, encourage

group work, see how students work problems, identify misconceptions, answer questions

Disadvantages: not all students work at same speed, some students do not like “looking over shoulder,” some students do not accept criticism well


Classroom Assessment Methods Minute paper and related exercises Pre- and post-tests Surveys Application cards Learning logs/journals Portfolios Coaching students working on examples Test is to make up test Open outlines Students defend test to instructor

Test is to Make Up Test Rich Felder calls this the “generic

quiz” CEE, Fall 1985

Student-Generated Test Questions Allows assessment of

Memorable course content Student concept of fair & reasonable How well they answer their own question

Demonstrates in-depth learning Probably works best in elective or

graduate classPrep: MResponse: MAnalysis: M- H

Test is to Make Up Test

What teaching goals would this tool assess?

Open Outlines Instructor provides students with a

partially completed lecture or presentation outline

Very few instructors collect the outlines for assessment

Good for courses with large amount of content Evaluates listening skills, learning of terms

and factsPrep: MResponse: LAnalysis: M

Students Defend Test to Instructor One-on-one meeting Student explains solutions Test not graded until after meeting Instructor really learns what student

knows Student gets individual feedback Probably best in elective/graduate class

with smaller enrollmentPrep: LResponse: MAnalysis: M-H

Using Technology

Class management software utilities

Surveys available from institution Personal Response System

Attendance Quizzes Diagnostics

Linking to Program Assessment

The Program Assessment session will address evaluation of student learning relative to eleven ABET outcomes.

Some classroom assessment methods may be adapted for outcomes assessment. Use overlap where possible

Effect on Students Increase their involvement in learning

they are telling you what you need to emphasize/review

Promotion of metacognition Students’ understanding of their own learning

Increased cooperation & satisfaction Camaraderie between students and

instructor; win-win Students learn more; instructors get feedback Students believe that instructor wanted their

input/feedback

Example

Propose a CA process and description of CAT for your “focus” course.

Consider realistic constraints of class sizes, instructor resources, TA resources (?), technology

Review Introduction to assessment Types of assessment Classroom goals Classroom assessment methods Using technology Linking classroom assessment to

program assessment Effect on students

Recommendations

Try a few simple methods the first time

Add more as become comfortable with first few

Different methods for different size classes

After you develop experience, teach your colleagues

Questions

?

outcomes assessment 1 classroom assessment joseph a. shaeiwitz west virginia university...

Documents

feedback assessment

outcomes assessment

program assessmentreview

classroom assessmentjoseph

learning objectives

summativea za

facial expressions

body language