transparent instruction and its impact on learning...transparent instruction and its impact on...

Transparent Instruction

and its Impact on Learning

Mary-Ann Winkelmes, Ph.D.

Executive Director, Center for Teaching and Learning, Brandeis University

Founder and Principal Investigator,

PURPOSE:

– Understand how transparently designed assignments

can offer equitable opportunities for all college

students to succeed; and consider applications

TASKS:

– Review: summary of research findings, cases

– Apply: to sample assignments

CRITERIA:

You’ll leave with

– Understanding of our research

– Strategies for applying TILT Framework to your

teaching materials

Overview

What is Transparency?

• Transparent teaching and learning methods

explicitly focus on how and why students are

learning course content in particular ways, and

how they’ll use that learning in their lives after

college.

Gaining Attention: 3 main reasons

Two Research Studies

• National Study: AAC&U and TILT

• UNLV Student Retention Study

• National Study: Association of American Colleges & Universities and TILT

• University of Nevada, Las Vegas, Student Retention Study

CONTEXT

for these studies

Faculty ID Challenges to Students’

Successful Completion of Academic Work

PreparationUnclear re: expectationsUnderprepared: language,

math, writing, research skills

Don't see connection to prior knowledge

Time managementProcrastinationUnderestimate time required

Little planning experienceCompeting commitments

MotivationAnxiety, Feel overwhelmedDon’t see relevanceLow self- confidence Reluctant to ask for help

Access to ResourcesDon't know how to get helpNo time outside class to seek help

Center for Teaching & Learning

Equity Crisis: Access ≠ Equity

• Underrepresented, first generation, low

income: half as likely to complete college in 4

years as white/Asian peers

• Gatekeepers stunt research

• High-achievement in HS can frustrate college

success

• Well-prepared novices don’t think like experts

1st Study: 2014-2016 AAC&U

• Co-PIs: Tia Brown McNair, Ashley Finley, AAC&UMary-Ann Winkelmes, TILT Higher Ed

• Schools:– Community College of Philadelphia– Queensborough Community College, Bayside, NY– St Edward's Univ. Austin, TX– Univ. of Houston – Downtown, TX– California State University, LA– Winston-Salem State University, NC– Heritage University, Toppenish, WA

• Publication: Peer Review (Spring 2016)

TILT Higher Ed Research Team:

Matthew Bernacki, Ph.D. (consultant)

Jeffrey Butler, Ph.D. (research, analysis)

David Copeland (consultant)

Jennifer Golanics, J.D., M. Ed. (analysis)

Sherry Marks (budget)

MaryKay Orgill (consultant)

Kati Harriss Weavil Ph.D. candidate (analysis)

Michelle Zochowski, M. Ed. candidate (analysis)

1st Study: Implementation

2014-2016 AAC&U study funded by

“Transparency and Problem-centered Learning”

– 7 MSIs, 1800 students, 35 faculty

• 425 First generation students

• 402 non-white students

• 479 low-income students

• 297 multiracial students

– 2 x small teaching intervention

Transparent Assignment Design

TemplatePurpose

• Skills practiced long-term relevance to students’ lives

• Knowledge gained. relation to stated learning outcomes•Task

• What students will do • How to do it (steps to follow, avoid)

•Criteria for success• Checklist or rubric in advance so students can self-evaluate• What excellence looks like (annotated examples where

students/faculty apply those criteria)

}

Research Findings:

1) Association of American Colleges &

Universities Study

Results

• Boosted students’ learning in 3 important ways (medium-large effect for underserved students):

• Academic confidence

• Sense of belonging

• Metacognitive awareness of skill development

(employer-valued skills, Hart Research Assoc.)

Impact: Boosted Success Predictors for ALL

Baseline Equivalence

Impact: Greater Gains for Underserved Students

Research Findings:

2) University of Nevada, Las Vegas Study

69.98% N = 1951/2788

77.10% N = 2821/3658

85.5% N = 744/870

50.00% 60.00% 70.00% 80.00% 90.00%

Cohort 2158 w/o PT

Cohort 2158

Primarily Transparent

Blue: UNLV first-time, full-time 1st year students in 2015-2016 enrolled in "primarily transparent" courses in Fall 2015 or Spring 2016,

retained 10/2016

Red: All UNLV first-time, full-time 1st year students in 2015-2016, including those in "primarily transparent" courses, retained 10/2016

Purple: UNLV first-time, full-time 1st year students in 2015-2016, excluding those in "primarily transparent" courses, retained 10/2016

(Sources: TILT Survey; UNLV Data Warehouse/Office of Decision Support, 10/23/2017)

2nd Study: 1-year UNLV Retention Rates Increase with TILT

63.88% N = 1781/2788

67.20% N = 2458/3658

77.80% N = 677/870

30.00% 40.00% 50.00% 60.00% 70.00% 80.00%

Cohort 2158 w/o PT

Cohort 2158

Primarily Transparent

Blue: UNLV first-time, full-time 1st year students in 2015-2016 enrolled in "primarily transparent" courses in Fall 2015 or Spring 2016,

retained 10/2017

Red: All UNLV first-time, full-time 1st year students in 2015-2016, including those in 2015-2016 "primarily transparent" courses,

retained 10/2017

Purple: UNLV first-time, full-time 1st year students in 2015-2016, excluding those in 2015-2016 "primarily transparent" courses,

retained 10/2017

(Sources: TILT Survey; UNLV Data Warehouse/Office of Decision Support, 10/23/2017 and 03/02/2018)

2nd Study: 2-year UNLV Retention Rates Increase with TILT

2nd Study: UNLV Students See Increased Skill Development

Purpose• Skills practiced long-term relevance to• Knowledge gained students’ lives

•Task• What students will do • How to do it (steps to follow, avoid)

•Criteria for success• Checklist or rubric in advance so students can

self-evaluate• What excellence looks like (annotated

examples where students/faculty apply those criteria)

}

Two Studies: The TILT Intervention (2 assignments, 1 course, 1 term)

(OPTIONAL)

Questions / Comments

STEM Students, End of Term

Perceived Transparency in the Course36. In this course, I knew the purpose of each assignment.

37.Each assignment included a section that explained how the assignment was related to the objectives of the course.

38.My instructor identified a specific learning goal for each assignment.-------------------------------------------------------------------------------------------------------------------

36. In this course, I knew the steps required to complete my assignments.

37.Each assignment included a detailed set of instructions for completing it.

38.My instructor provided detailed directions for each learning activity that was assigned.

-------------------------------------------------------------------------------------------------------------------

36. In this course, I knew how my work would be evaluated.

37.My instructor provided students with annotated examples of past students’ work.

38.My instructor provided tools I could use to assess the quality of my and others' work. ----Never, Sometimes, Often, Always

Learning Outcomes that 4 in 5 Employers Rate as

Very Important

– Oral communication

– Working effectively with others in teams

– Written communication

– Ethical judgment and decision-making

– Critical/analytical thinking

– Applying knowledge/skills to real world

– Analyzing/solving complex problems

4. How much has this course helped you in writing effectively?5. How much has this course helped you in communicating your ideas effectively in your spoken statements?6. How much has this course helped you in collaborating effectively with others?8. How much has this course helped you in improving your ability to separate and examine the pieces of an idea,

experience, or theory?9. How much has this course helped you in learning how to connect information from a variety of sources?10. How much has this course helped you in learning how to apply concepts to practical problems or in new

situations?11. How much has this course helped you in considering the ethical implications of your actions?

Not at all, A little, A moderate amount, A lot, A great deal22. As a result of taking this course are you a better or worse judge of the strengths and weaknesses of ideas, or

has the course made no difference?24. As a result of taking this course are you a better or worse judge of the reliability of information from various

sources, or has the course made no difference?Much worse, Somewhat worse, No difference, Somewhat Better, Much Better

32. Are you likely to apply knowledge and skills you gained from this course in contexts outside of this course?

Not likely, Slightly likely, Moderately likely, Very likely, Extremely likelyNew STEM-focused skills questions:How much has this course helped you in designing experiments or processes to address a problem? How much has this course helped you in analyzing and interpreting data and/or problems? How much has this course helped you in choosing methods appropriate to solving a problem?

Response options: Not at all, A little, A moderate amount, A lot, A great deal

Awareness of Improvement of Employer-valued Skills

Skills: Beginning and End of CourseThe following 10 questions are asked at the beginning and end of term:

I can express my ideas effectively when I write.

I can communicate effectively when I speak.

I collaborate well with others on academic work.

I am good at breaking down theories, ideas and experiences into pieces so I can

consider them.

When I am given information from multiple sources, I have an easy time making

connections between them.

I am able to apply the things I have learned to new problems and situations.

I tend to consider the ethical implications of my actions.

I am capable of learning on my own.

Response options: Never, Sometimes, Often, Always

Please rate your confidence about your ability to succeed in school.

Please rate your confidence about your ability to succeed in this field.

Response options: Low, Moderate, High

Confidence

30. Please rate your confidence about your ability to succeed in school.

31. Please rate your confidence about your ability to succeed in this field.

Low, Moderate, High

25. As a result of taking this course are you more or less confident about your

ability to succeed in school, or has the course made no difference?

26. As a result of taking this course are you more or less confident about your

ability to succeed in this field, or has the course made no difference?

Much less confident, Somewhat less, No difference, Somewhat more, Much more

confident

Belonging

34. How much did class meetings incorporate the students' suggestions and

interests?

35. How much did the instructor value you as a student?

New: How much did this course help you feel that you are a member of your school’s

community?

Not at all, A little, A moderate amount, A lot, A great deal

New: I feel that I am a member of my school’s community. Never, Sometimes, Often,

Always

Academic Confidence & Sense of Belonging

Foundational Research that

Supports Transparent Instruction

Research on Learning Implications for

Transparent Instruction

2. Show students your plan for building their skills in sequence

HIST 251 / ART 495 Dr. Mary-Ann Winkelmes Tuesday / Thursday 8:30 – 9:45 am University of Nevada, Las Vegas

INTRODUCTION TO HISTORICAL METHODS / SPECIAL TOPICS IN ART HISTORY:

Art and Politics in Renaissance Italy This course focuses on strengthening the intellectual skills that are essential to History and Art History, as well as successful professional careers. In the four units of this course, we will consciously practice these skills:

UNIT 1) Understanding the frameworks: contexts, techniques, terms, artifacts as history, UNIT 2) Analysis and Synthesis: using artifacts, primary and secondary sources to construct the story, UNIT 3) Evaluating reliability of artifacts and sources, perceptions of value and achievement, evolving contexts, UNIT 4) Creating new contexts for examining Renaissance artifacts and events.

As we hone these skills, we’ll study painting, sculpture and architecture in the context of various political and social environments in Renaissance Italy: the city-state, the church, the noble court, the neighborhood. We have come to think of Renaissance sculptures, paintings and decorative objects as artworks. Yet the creators and original users of these works saw them primarily as religious or domestic furnishings, indicators of power or civic pride, gifts or aids to spiritual meditation -- many with strong political messages. Artists and patrons we will examine include Michelangelo, Raphael, Leonardo, the Medici, Pope Julius II, Isabella d’Este, and other influential artists and patrons.

…

SCHEDULE OF CLASS MEETINGS AND REQUIRED READINGS

UNIT 1 SKILLS) Understanding the frameworks: contexts, techniques, terms, art as history

Jan 20: Michelangelo’s David and its meanings over time

Focus questions: Who made it and how? Who commissioned it and why? Who saw it, used it, and how? How does it involve or appeal to viewers? How has its context and meaning changed? What happened to it over time? What does it mean to us now?

Jan 22: Places and purposes of art in daily life in Renaissance Italy

Reading: Paoletti 12-15, 43 Focus questions:

• What are some examples of religious art and architecture in contemporary cultures?

• How might examples of religious art and architecture in Renaissance culture be similar/different?

• How do we define “religious” and “Renaissance” and “art,” and how do our definitions affect the ways we study and understand the history, art and artifacts of Renaissance Italy?

Jan 27: Materials, Techniques and Conditions of Artistic production …

UNIT 2 SKILLS) Analysis and Synthesis: using artifacts, primary and secondary sources to construct the story

Feb 19: Formal Visual Analysis practice in class … Feb 26: Analysis of primary sources: Michelangelo’s Last Judgment

Reading: primary sources in class Focus questions:

• Are primary sources more reliable than secondary sources?

• What makes a source reliable?

• When primary sources contradict each other, how to judge?

• When secondary sources contradict each other, how to judge?

2. Show students your plan for building their skills in sequence

Examples for Discussion

Sample A, p. 7

Purpose

• Skills practiced relevance to students

• Knowledge gained

Task: What to do

How to do it

Criteria

• What excellence looks like (annotated)

• Criteria in advance to help students self-evaluate

}

Sample A, p. 7

Purpose


• Knowledge gained }

1) Read the assignment and look for:

2) Wave when you locate the purpose.

Sample A, p. 7

Purpose





3) (2 min) Talk to your colleagues: Do you agree?

Sample A, p. 7

Purpose





3) (2 min) Talk to your colleagues: Do you agree?

3) Report back

Sample C, p. 8

Purpose


• Knowledge gained

Task: What to do

How to do it

Criteria

• What excellence looks like (annotated)

• Criteria in advance to help students self-evaluate

}

Sample C, p. 8


• How is it different from Sample A?

• How is it similar to Sample A?

Sample C, p. 8




2) Discuss with your colleagues: Do you agree?

Sample C, p. 8




2) Discuss with your colleagues: Do you agree?

3) Report back

More Contexts for TILT

Assignment


Program/Unit (institution)


Example: UNLV


Unit: disciplinary language/guidelines

Purpose Skills practiced Knowledge gained

Task What to do How to do it

Criteria Measures for success Examples

Libraries: Dee Fink, Fink, Significant Learning Experiences (2003)

Learning GoalsTeaching/Learning

Activities

Feedback and Assessment # students served # faculty/staff served support for research

productivity

Online Ed: Quality Quality Matters rubric (QM 2014, 2014, 2017)

2. Learning objectives,competencies

4. Instructional Materials1. Course Overview

and Intro5. Course Activities,

Learner Interaction7. Learner Support

5. Assessment and Measurement # online students # student completions # online courses # hybrid courses

Registrar: Bennis 2009, Mahoney 2016, Stark 2010, 2010, AACRAO n.d.

Define how the Office Office and the employee both benefit benefit from a task that that supports student student success

Actions that support the mission mission of student student success

Employee annual annual reviews

Evidence of impact and goal goal achievement smooth progression vs.

vs. bottlenecks in students' students' path to completion

student retention student completion

Example: UNLV


Unit: disciplinary language/guidelines

Purpose Skills practiced Knowledge gained

Task What to do How to do it

Criteria Measures for success Examples

Academic Success Success Center: Center: CAS 2014

Goals Education Plan Responsibility to

meet requirements

Articulate intent, intent, curriculum

Engage Use complex info to

assess goal achievement

Academic Advising: CAS 2014

Goal settingIdentify desired courses, learning, GPA

Goal progression Self-monitoring,

awareness

Campus Life & Recreation: ACPA/NASPA 2016

Outcomes of the programs and how how those offer learning benefits for for students

What will students do?

How will they do do it?

Criteria, examples of of successful work

Reflection on student's student's learning

UNLV Faculty Development Strategic Plan


Stylus,

March 2019

How Did We Do?PURPOSE:

– Understand how transparently designed assignments

can offer equitable opportunities for all college

students to succeed; and consider applications

TASKS:

– Review: summary of research findings, cases

– Apply: to sample assignments

CRITERIA:

You’ll leave with

– Understanding of our research

– Strategies for applying TILT Framework to your

teaching materials

Please join us!

TILTHigherEd.com

WORKSHOP TOMORROW 10:00am

• Examples of TILTed teaching

materials across disciplines

• Revise your assignment to make it

more transparent for students

transparent instruction and its impact on learning...transparent instruction and its impact on...

Documents