Assessing Learner

ProgressJim Marteney

“Teaching

without learning

is just talking.”

“Teaching

without learning

is just talking.”

“I’d really enjoy

teaching if it

wasn’t for

grading.”

What do you want your

students to learn?

How do you know your

students have learned?

Traditional Testing?

How do you evaluate

your students?

What is Quality

Matters?

A set of quality standards for

online course design

A faculty-centered peer

review process for online

courses

What is Quality

Matters?

A tool used by instructional

design staff their work with

faculty

A professional development

opportunity for both faculty

and staff

Faculty-driven

Peer Reviewed

Review Process that is

Collaborative Continuous

CollegialCentered

Quality Matters Rubric

Rubric Standards

Course Overview and Introduction

Learning Objectives

Assessment and Measurement

Instructional Materials

Learner Interaction and Engagement

Course Technology

Learner Support

Accessibility

Rubric Standards

Course Overview and Introduction

Learning Objectives

Assessment and Measurement

Instructional Materials

Learner Interaction and Engagement

Course Technology

Learner Support

Accessibility

General Standard 3

Assessment and Measurement

“Assessments are integral to the learning process and are designed to evaluate learner progress in achieving the stated learning objectives or mastering the competencies.”

General Standard 3

Assessment and Measurement

The assessments measure the stated

learning objectives or competencies.

The course grading policy is stated

clearly.

Specific and descriptive criteria are

provided for the evaluation of learners’

work and are tied to the course grading

policy.

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

The course provides learners with multiple

opportunities to track their learning progress.

General Standard 3

General Standard 3

Assessment and Measurement

General Standard 3

The Key:

“Critical course

components

reinforcer one

another”

The assessments measure the stated

learning objectives or competencies.

Alignment

General Standard 3

The Key:

with your activities?

The assessments measure the stated

learning objectives or competencies.

Alignment

Do your assignments line up

with your objectives?

with your modules?

General Standard 3

“Course assessments are consistent with the course and module learning objectives or competencies.”

The assessments measure the stated

learning objectives or competencies.

“From types of assessments chosen, it is clear that learners can successfully complete the assessments if they have met the unit’s learning objectives.”

General Standard 3

The assessments measure the stated

learning objectives or competencies.

General Standard 3

What are your objectives?

The assessments measure the stated

learning objectives or competencies.

To know if you have arrived, you

need to know where you’re going.

Creating Course Objectives

Should we add

our

module

objectives?

Do we even have

module objectives?

We have

course objectives

Creating Course Objectives

Standard 2

The module/unit learning objectives

describe outcomes that are measurable

and consistent with the course-level

objectivesAll learning objectives are stated

clearly and written from the student’s

perspective

Instructions to students on how to

meet the learning objectives are

adequate and stated clearly.

Creating Course Objectives

Create a Stem

Add a

Verb

Determine the Outcome

After completing the lesson, the student will be able to . . .

After this unit, the student will have . . .

By completing the activities, the student will . . .

At the conclusion of the course/unit/study the student will . .

re-tell in his/her own words _____.

summarize the plot of _____.

identify states of matter . . .

create a timeline of events . . .

identify the causes and effects of . . .

create Venn Diagrams which compare and contrast . . .

recall information about the reading . . .

After this unit, the student will be able to create

a timeline of events leading up to the Civil War.

General Standard 3

The course grading policy is stated

clearly.

“A clear, written statement fully explains how the course grades are calculated.”

“The relationship(s) between points, percentages, weights, and letter grades are explained.”

“The instructor’s policy on late submissions is clearly stated.”

General Standard 3

The course grading policy is stated

clearly.

Where can you post your grading policy?

Syllabus

Introduction letter

Modules

Quiz

Discussion

Announcement

General Standard 3

The Key:

Specific and descriptive criteria are

provided for the evaluation of learners’

work and are tied to the course grading

policy.

Rubrics

General Standard 3

Heidi Andrade

“A document that articulates

the expectations for an

assignment by listing the

criteria, or what counts, and

describing levels of quality from

excellent to poor.”

General Standard 3

General Standard 3

General Standard 3

Let’s not be this guy

Let’s create a Rubric

General Standard 3

Goal 1

Goal 2

Goal 3

Goal 4

Level 1 Level 2 Level 3

General Standard 3

Followed format: 5

Missed 2 aspects: 3

No relation to format: 0

Used 5 sources: 5

Used 3-4 sources: 3

Used 0-2 sources: 0

No mistakes: 51-5 mistakes:

36 or more

mistakes: 0

Format

Sources

Grammar

Stapled

Hot Medium Cold

Used staples: 2

Not use staples: 0

General Standard 3

My Rubric For Discussions

General Standard 3

The Key:

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

Variety that works with online learning

General Standard 3

Are Traditional Tests The Best Evaluative

Methods?

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

General Standard 3

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

“Multiple assessment strategies are used in both the online and face-to-face settings, including alternative assessments that require learners to apply what they learn and to think critically.”

General Standard 3

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

http://cft.vanderbilt.edu/guides-sub-pages/writing-

good-multiple-choice-test-questions/

Vanderbilt UniversityCenter for Teaching

Writing a Multiple Choice Test

General Standard 3

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

Writing a Multiple Choice Test

A multiple choice item consists of a problem, known

as the stem, and a list of suggested solutions, known

as alternatives. The alternatives consist of one correct

or best alternative, which is the answer, and incorrect

or inferior alternatives, known as distractors.

General Standard 3

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

The stem should be meaningful by itself and should

present a definite problem. A stem that presents a

definite problem allows a focus on the learning

outcome.

Writing a Multiple Choice Test

The assessment instruments selected

are sequenced, varied, and appropriate to

the learner work being assessed.

General Standard 3

“In traditional assessments, such as those that use multiple choice, true-false, or matching, learners are asked to select a response from different options, and tests often are self-scoring.”

The assessment instruments selected

are sequenced, varied, and appropriate to

the learner work being assessed.

General Standard 3

“In alternative assessments, also commonly called performance or authentic assessments, learners are asked to develop an answer in response to a prompt or stimulus, which is graded by the instructor. Such assessments may include interviews, journals, portfolios, observations, demonstrations, performance tasks, and exhibits.”

The assessment instruments selected

are sequenced, varied, and appropriate to

the learner work being assessed.

General Standard 3

“Assessments are varied in order to provide multiple ways for learners to demonstrate mastery, and to accommodate diverse learners.”

The assessment instruments selected

are sequenced, varied, and appropriate to

the learner work being assessed.

General Standard 3

“The assessments are sequenced so as to promote the learning process and to build on previously mastered knowledge and skills gained in this course and prerequisite courses. Assessments are paced to give learners adequate time to achieve mastery and complete the work in a thoughtful manner.”

Scaffolding

Using Online Technologies Beyond Traditional Tests

What do You Use to

Evaluate Your Learners?

What have you ever

thought of trying?

Using Online Technologies Beyond Traditional Tests

Discussions: more than Participation

Peer to Peer Evaluation

Journaling

Individual Projects

Group Projects

Sharing Homework with peers vs. just the instructor

Using Online Technologies Beyond Traditional Tests

CATS in tests and quizzes

??

Feedback from testing

Creating content like videos/powerpoints

Moderating their own discussion

The course provides learners with multiple

opportunities to track their own learning

progress.

General Standard 3

“Learning is more effective if learners receive frequent, substantive, and timely feedback. The feedback may come from the instructor directly, from assignments and assessments that have feedback built into them, or even from other learners.”

The course provides learners with multiple

opportunities to track their own learning

progress.

General Standard 3

“Look for examples of self-check quizzes and activities, as well as other types of practice opportunities that provide timely feedback. Such assignments may be voluntary and/or allow multiple attempts.”

The assessments measure the stated

learning objectives or competencies.

The course grading policy is stated

clearly.

Specific and descriptive criteria are

provided for the evaluation of learners’

work and are tied to the course grading

policy.

The assessment instruments selected

are sequenced, varied, and suited to the

learner work being assessed.

The course provides learners with multiple

opportunities to track their learning progress.

General Standard 3

Thank you

Now Go Forth and Assess