Empowering Faculty to Improve Students’ Learning through Collaborative Assessment:

An Implementation Workshop for Faculty and Program Administrators

Come with a challenge

Work together 2:00 – 2:30 Presenters

2:30 – 3:30 Tables Work

3:30 – 3:45 Break, Independence Foyer

3:50 – 4:30 Tables Work

4:30 – 4:45 Whole Group Convenes

4:45 – 4:50 Conclusion, Next Steps

Leave with a plan

Faculty/instructional development tool …

• By, of, for instructors: formative vs. summative

Appendix B

• Patterns of beneficial teaching/learning methods

– Discipline, level, size, demographics (Appdx A)

Example: transparency around assignment design

… broader uses?

Discuss assignments' learning goals and design rationale before students begin work

• Chart out the skills students will practice in each assignment example

• Begin each assignment by defining the learning benefits to students: skills practiced, content knowledge gained

example

• Provide criteria for success in advance example 1 Business U of Illinois;

example 2 research paper U of Baltimore;

example 3 lab report Iowa State U

• Offer examples of successful work, and annotate them to indicate how criteria apply

example, plant Bio, U of Illinois

TEMPLATE: SAMPLE BUSINESS COURSE ASSIGNMENT due November 6

Purpose: Skills: The purpose of this assignment is to help you practice the following skills that are essential for your success in this course and in any business context:

– using information technology – analysis of diverse, multicultural, global economic contexts – assessing professional, ethical and legal aspects – judging and selecting best solutions

Knowledge: This assignment will also help you to become familiar with a particular

multicultural business context.

Task: Use the material in the video and the website to illustrate what you learned about subsistence

contexts. In particular, discuss the types and nature of challenges that individuals have to face in different areas such as food, etc. Suggest some ideas that businesses could pursue to help people living in poverty while also making a profit.

Criteria for Success: A successful assignment will have the following characteristics: – 250 written words maximum with at least 3 video clips (under 4 minutes total) – Comparison of at least two contexts – Identification of ethical dilemma or ethical considerations – Evidence to support why your suggestion could work

Project Scope

Since 2010, this initiative has involved

31000+ students

200+ courses (varied disciplines, levels)

24 higher ed institutions

(cc, public/private, large research, small college)

6 countries US, Mexico, Scotland, Australia, Indonesia, India

Voluntary Response Rates

Response rates for US students only (2012)

Course Size

category

Total

Enrollmt

Total N Avg Rate

of Return

(1)= or < 30 629 320 50.9%

(2) 31-65 1223 448 36.6%

(3) 66-299 3416 787 23.0%

(4) = or > 300 9841 862 8.8%

Benchmarks:

Our Race/Ethnicity distribution ≅ representative

White 61%

Black 4%

Hispanic 5%

Asian 17%

Other 7%

Unknown 6%

US Institutions in Transparency Sample 2009-2012

White 62%

Black 15%

Hispanic 14%

Asina/Pac Isander

6%

Two or more 2%

Native Am 1%

US Institutions in 2010

Benchmarks:

Our UG control group doesn’t overestimate

its ability to learn

Question Total Women Men

PSRI

Strongly agree: I developed

increased ability to learn from

diverse perspectives during college

52.6% 56.6% 48.1%

Current (TILT) Study

More likely (Much more likely): to

consider different opinions or points

of view as result of taking this

course

50.7%

(11.4%)

48.5%

(9.1%)

52.2%

(13.1%)

Benchmarks: Our UG control group doesn’t over-estimate

its learning

Question Scales (NESSE 4-pt / TILT 5-pt)

NSSE During the current

year, how much has your

coursework emphasized:

Very

Little Some

Quite

a Bit

Very

Much

Analyzing 2% 16% 44% 38%

Application 4% 20% 39% 38%

TILT How much has this

course helped you in: Not at

All

A

Little

Moderate

Amt A Lot

Great

Deal

Analyzing 6% 18% 36% 28% 12%

Application 6% 18% 32% 30% 14%

Benefits for underrepresented students: Are you better or worse at recognizing when you need help

with your academic work?

Mean Responses for students in intro Physical Science courses

Enrollment Size 66-299

1

2

3

4

5

Caucasian Non-Caucasian

Me

an R

esp

on

se

Race/Ethnicity

Control "Understanding" "Assignments" + "How Learn"

Faculty development tool … broader uses?

Institutions:

First-year student success/retention

Gregory Brown, Vice Provost for Faculty, Policy & Research

Advising / Student Success

Administrative and faculty advisors

Gayle Juneau, Executive Director, Academic Advising

Programs:

General Education first-year seminars

Jennifer Keene, Associate Dean, College of Liberal Arts

Worksheet

• Challenge / goal

• Existing data / future measure of success

• Strategic planning

– Timeframe, plan what happens when

– Barriers

– Resources you have

– Resources you need

Gregory Brown

Vice Provost for Faculty, Policy and Research

Challenge / goal

Retain more first-year students

Existing data: over 1/5 of 1st year class stops (700+ students)

Assignment design rationale

benefits 1st-yr all disciplines

Student Progression

Nu

mb

er

of

Stu

de

nts

0

500

1000

1500

2000

2500

3000

3500

EnteringFreshmen

Progressionrates

GraduatingCohort

24Credits

78%

60Credits

74%

75Credits

64%

90Credits

91%

Goal: Retain More First-year Students

Data: future measures of success

• Numbers of 1st year students; rising sophomores

• Transparency survey questions re: 1st year UULOs

5-10, 16. communication, application, evaluation

19. recognizing when you need help

17. confidence to succeed in school

Strategic planning

– Timeframe: 2d sem large enrollment and FYS instructors modify assignment design in test group

– Barriers: survey via FYS course evaluations

– Resources: workshops, data gathering, analysis

Gayle Juneau,

Executive Director of Academic Advising

Challenge / goal

Administrative and academic advisors collaborate more

Connect more with students’ classroom experiences

Existing data

1st-year student retention rate

Did advisor direct you to helpful resources? (sooner?)

Assignment design rationale benefits 1st-yr all disciplines

Measures of Future success

Transparency survey ?s:

19, 21. are you better/worse recognizing when you need help

23. understand what constitutes successful work

17, 18. confidence to succeed in school/field

Goal: advisors collaborate, connect more

with students’ classroom experiences

Strategic planning

Timeframe: 2d semester, ongoing

College-based workshops for advisors: Coaching students: strategies to ID successful course work

Coaching students: recognize sooner when need help / how to get it

Barriers:

Lack precedent for prof’l & academic advisors studying classroom data

Advantage?: Equal footing (unfamiliar) for both groups

Resources: workshops, data gathering, analysis

Jennifer Keene,

Associate Dean, College of Liberal Arts

Challenge / goal In non-standardized, discipline-based First Year Seminars,

improve students’ writing

Existing data

1st-year retention rate in College of Liberal Arts

Grades, baseline assignments: varied

Measures of Future success

Assignment design rationale benefits 1st-yr all disciplines

Survey ?s: 4. How much has this course helped you in writing effectively

8. How much … improving your ability to separate and examine

the pieces of an idea, experience or theory (Bean, Colomb)

23. How well do you understand what constitutes successful work …

Goal: improve students’ writing

Strategic planning

Timeframe: 2d sem, ongoing

workshops for instructors to modify assignment design

gather, analyze data

Barriers:

Adjunct population includes eager and unwilling

Advantage?: test group and control group

Scheduling: blended online/onsite format for workshops

Resources: workshops, data gathering, analysis

Table Groups

10 minutes

• Introductions (1 minute per person)

Pairs discuss, record on worksheet:

GOALS, MEASURES

20 minutes

• Your challenge / goal (5 m)

• Existing data, future measures of success (5m)

– National Transparency data that could inform (5m)

– Specific survey questions that could inform (5m)

Pairs discuss, record on worksheet:

STRATEGIC PLANNING

15 minutes

• Timeframe, simple plan (what happens when) (5 m)

• Barriers to success (5m)

• Resources you have / need (5m)

Break: Independence Foyer

3:45 pm Table groups reconvene

Individual Feedback

35 – 40 minutes (5 min per table member)

Briefly identify, gather feedback on:

– Goal

– Measures/data

– Timeframe, plan (what happens when)

– Resources

– How Transparency project can adapt to help

General Advice

Common insights from groups:

– Goal

– Measures/data

– Timeframe, plan (what happens when)

– Resources

– How Transparency project can adapt to help