A Framework for Evidence-based Teaching in Developmental Biology

Scott Freeman, Department of Biology

University of Washington

srf991@u.washington.edu

Why are we still lecturing?

I don’t believe that active learning can work in a large lecture. (UW professor, 8/12)

I just know that students .... (UW professor, 3/09)

Although it did not occur to us .... to collect data, we consistently observed … (Barzilai 2000)

… we feel that our junior-senior cell biology course ... works extraordinarily well …” (Lodish et al. 2005)

We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990)

We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

I don’t believe that active learning can work in a large lecture. (UW professor, 8/12)

I just know that students .... (UW professor, 3/09)

Although it did not occur to us .... to collect data, we consistently observed … (Barzilai 2000)

… we feel that our junior-senior cell biology course ... works extraordinarily well …” (Lodish et al. 2005)

We think that our objective of teaching the students to think was well-accomplished. (Miller & Cheetham 1990)

We strongly believe that they lead to deeper understanding.... (Rosenthal 1995)

Other changes to our mindset, as faculty:

“I’d like to change my lectures, but I don’t have time.” (or don’t know how)

If a new technique is sweeping my research field, do I require release time and other special support to learn it?

“Oh, I tried active learning (or clickers, or group exercises)—it doesn’t work.”

The first PCR I ever tried didn’t work. Should I conclude that PCR doesn’t work?

Why be concerned about the failure rate?

Predicted grade

Average %EOP studentsin Bio180

Previous work on Biology 180How can we lower failure rates—and help capable but underprepared students—in introductory biology courses?

Spring 2002-2003 Course design

Spr ‘02

< 1.5 18.2%

< 2.5 44.8%

2002: Modified Socratic style

Student performance (does not include drops):

Spr ‘02 Spr ‘03

< 1.5 18.2% 15.8%

< 2.5 44.8% 42.3%

; 2003: + ungraded active learning

Spring 2005, Fall 2005 Course design

Spr ’02 Spr ‘03

< 1.5 18.2% 15.8%

< 2.5 44.8% 42.3%

Socratic lecturing; Cards or clickers (daily multiple-choice questions in class); weekly, peer-graded practice exam (short-answer)

Spr ‘02 Spr ’03 Spr ‘05 Fall ‘05

< 1.5 18.2% 15.8% 10.9% 11.7%

< 2.5 44.8% 42.3% 37.9% 39.3%

Low structure Medium structure High structure

Fall 2007, 2009 Course design

Spr ‘02 Spr ‘03 Spr ‘05 Fall ‘05 Fall ‘07 Fall ‘09

< 1.5 18.2% 15.8% 10.9% 11.7% 7.4% 6.3%

< 2.5 44.8% 42.3% 37.9% 39.3% 33.9% 28.3%

“Lecture-free;” clickers in peer instruction format; weekly, peer-graded practice exam; daily reading quiz; random-call ~15 students/class

• %A’s has increased from 14.5% to 24.3%

Spr ‘02 Spr ‘03 Spr ‘05 Fall ‘05

< 1.5 18.2% 15.8% 10.9% 11.7%

< 2.5 44.8% 42.3% 37.9% 39.3%

Are exams equivalent across quarters? Approach #1: Predicted exam score

Spr ‘02 Spr ‘03 Spr ‘05 Fall ‘05 Fall ‘07 Fall ‘09

Course Average PES (100pt exam)

70.6 70.2 70.9 70.5 68.0 67.5

Approach #2: Weighted Bloom’s Index

Spr ‘02 Spr ‘03 Spr ‘05 Fall ‘05 Fall ‘07 Fall ‘09

Course Average (weighted Bloom’s index) 45.8 52.1 46.9 52.2 52.1 53.5

Are students equivalent across quarters?

Spring 2002

Spring 2003

Spring 2005

Autumn 2005

Autumn 2007

Autumn 2009

Predicted grade (mean)

2.46 2.57 2.64 2.67 2.85 2.70

n 327 338 334 328 339 691

Create a general linear model to explain actual grade, based on predicted grade and degree of structure in course.

Regression model with UW GPA (at time of entering) and SAT-V; R2 ≈ 0.63

2002, 03 2005 2007,09 Course structure

Did we reduce the achievement gap?

… without spending a lot more money? or maybe even less money?

2003-2008 (Aut/Win/Spr) averages: EOP v non-EOP final grade differences in UW gateway STEM courses

General linear mixed-effects modeling and MMI:Best models include EOP as a fixed effect; likelihood-ratio test, p = 0.0027).

Bio180: lecturing vs. high-structure

UW Regents

Low structure

High structure

What could cause a disproportionate increase in performance by disadvantaged students?

The Carnegie Hall hypothesis:

How do you get to Carnegie Hall? … and how you practice matters (deliberate practice): 1) high-level questions—new contexts/applications); 2) group work—teach others/explain yourself, challenge

and be challenged—with instructor feedback; 3) daily/weekly basis.

PRACTICE!

Dave Parichy’s questions:

• Can PIs do this and still run their labs?

• How do we balance the explosion of detail in developmental biology with big-picture concepts, and help students integrate facts into a cohesive framework?

• Does this approach transfer to upper-division courses?

Broadening the research focus: From course design in introductory biology to all of the STEM disciplines

A meta-analysis of 642 papers from across the STEM disciplines: studies that compare any active-learning intervention to traditional lecturing.

1. Exam/concept inventory/quiz performance: controlling for instructor, student, and assessment equivalence; n = 158

2. DFW (failure) rates; n = 67

Exam performance data:

Overall effect size = 0.47

• In intro STEM, 6% increase in exam scores; 0.3 increase in average grade.

Course level n Hedges’s g s.e. 95% C.I.: lower limit

95% C.I.: upper limit

Introductory 116 0.489 0.065 0.361 0.616

Upper division 38 0.480 0.120 0.245 0.715

Failure rate data:

Overall odds ratio = 1.94

• Biomed RCTs stopped for benefit: mean relative risk of 0.53 (0.22-0.66) and/or p < 0.001.

Course level n Odds ratio 95% C.I.: lower limit

95% C.I.: upper limit

Introductory 44 1.994 1.732 2.296

Upper division 17 1.762 1.372 2.263

Dave’s Second Question: The content problem

1950 1960 1970 1980 1990 2000 2010 20200

5000

10000

15000

20000

25000

Year

Page

s pub

lishe

d in

PN

AS

Apply: Can I use these ideas in a new situation?

Understand: Can I explain these ideas to someone else?

Remember: Can I recall key terms and ideas?

Analyze:Can I recognizeunderlying patternsand structure?

Synthesize:Can I put ideas and information together to create something new?

Evaluate:Can I make judgmentson the relative value of ideas and information?

Lower order thinking

Higher order thinking

Bloom’s taxonomy as a conceptual framework:

and hierarchical

Coping strategies:

• State learning objectives; use backward course design

• Reading quizzes or other “flipping” strategies

Dave’s Third Question: The 6-jobs problem

• Breaking the “Research vs. Teaching” dichotomy with RICs

• Find a colleague/mentor to help with new techniques

• Recruit grad students/post-docs who want to teach

• Start small and expect to fail (the first time)

My all-time favorite line from a course evaluation:

“Keep pushing us—we can do it!”

Bill HoeseAnne CasperKelly HoganClarissa DirksCarol PollackMegan RectorPam Pape-LindstromRoss NehmBrian CasperJenny KnightJoan SharpMichelle Smith

Peter ShafferPaula HeronLillian McDermottDavid HodgeFerric FangEmile PitreRobert HarringtonKevin MihataCathy BeyerDeb McGheeMichael Griego

Mercedes ConverseMichael FlemingIggy ChauMikhail KovalDozie OkoroaforRoddy TheobaldDavid HaakMicah HorwithChris GastRiley BrazilEunice LauHannah JordtEliza HeeryAlan SunadaChelsea MannDave HaysElli Jenkins

Sara BrownellSarah EddyJen NemhauserDave HurleyMatt CunninghamTom DanielAlison CroweBarbara WakimotoJanneke Hille Ris LambersEileen O’ConnorJohn ParksMary Pat WenderothToby BradshawBen WigginsMandy Schivell